Removing outliers for A/B search tests
How do you measure the success of a new feature? How do you test the impact? There are different ways ...
Senior Software Engineer
How do you measure the success of a new feature? How do you test the impact? There are different ways ...
Senior Software Engineer
Algolia's advanced search capabilities pair seamlessly with iOS or Android Apps when using FlutterFlow. App development and search design ...
Sr. Developer Relations Engineer
In the midst of the Black Friday shopping frenzy, Algolia soared to new heights, setting new records and delivering an ...
Chief Executive Officer and Board Member at Algolia
When was your last online shopping trip, and how did it go? For consumers, it’s becoming arguably tougher to ...
Senior Digital Marketing Manager, SEO
Have you put your blood, sweat, and tears into perfecting your online store, only to see your conversion rates stuck ...
Senior Digital Marketing Manager, SEO
“Hello, how can I help you today?” This has to be the most tired, but nevertheless tried-and-true ...
Search and Discovery writer
We are proud to announce that Algolia was named a leader in the IDC Marketscape in the Worldwide General-Purpose ...
VP Corporate Marketing
Twice a year, B2B Online brings together America’s leading manufacturers and distributors to uncover learnings and industry trends. This ...
Director, Sales Enablement & B2B Practice Leader
Generative AI and large language models (LLMs). These two cutting-edge AI technologies sound like totally different, incomparable things. One ...
Search and Discovery writer
ChatGPT, Bing, Bard, YouChat, DALL-E, Jasper…chances are good you’re leveraging some version of generative artificial intelligence on ...
Search and Discovery writer
Your users are spoiled. They’re used to Google’s refined and convenient search interface, so they have high expectations ...
Technical Writer
Imagine if, as your final exam for a computer science class, you had to create a real-world large language ...
Sr. SEO Web Digital Marketing Manager
What do you think of the OpenAI ChatGPT app and AI language models? There’s lots going on: GPT-3 ...
Search and Discovery writer
In the fast-paced and dynamic realm of digital merchandising, being reactive to customer trends has been the norm. In ...
Staff User Researcher
You’re at a dinner party when the conversation takes a computer-science-y turn. Have you tried ChatGPT? What ...
Sr. SEO Web Digital Marketing Manager
It’s the era of Big Data, and super-sized language models are the latest stars. When it comes to ...
Search and Discovery writer
Did you know that 86% of the global population uses a smartphone? The 7 billion devices connected to the Internet ...
Staff SME Business & Optimization - UI/UX
The Cloud Native Foundation is known for being the organization behind Kubernetes and many other Cloud Native tools. To foster ...
In previous blog posts, we have discussed the high-level architecture of our search engine and our worldwide distributed infrastructure. Now we would like to dive a little deeper into the Algolia search engine to explain why we implemented it from scratch instead of building upon an existing open-source engine.
We have many different reasons for doing so and want to provide ample context for each, so we have split “Inside the Algolia engine” into several posts. As you learn more about our search engine, please let us know if there’s anything you would like us to address in future posts.
If you have ever worked on a search engine with significant traffic and indexing, you are undoubtedly familiar with the problem of trying to fine-tune your indexing to avoid negatively affecting search performance. Part one of this series will focus on one of the quintessential problems with search engines—the impact of indexing on search queries—and our approach to solving it.
Indexing impacts search performance because indexing and search share two critical resources—CPU and Disk. More specifically:
The obvious way to solve this problem is to try to reduce or remove the conflicts of access to the shared resources.
There are a lot of different approaches to dealing with this issue, and the majority fall into one of the following three categories:
While complex to implement, the second approach of using different machines for indexing and search is a good solution if indexing performance is not crucial to you. The other two approaches only partially solve the issue as search remains impacted. Realistically, none of these approaches appropriately solves the problem of indexing affecting search performance because either indexing performance, search performance or both end up suffering.
By splitting the indexing and search into different application processes!
At Algolia, indexing and search are divided into two different application processes with different scheduling priorities. Indexing has a lower CPU priority than search based on a higher nice level (Nice is a tool for modifying CPU priority on Unix-like operating systems). If there is not enough CPU to serve both indexing and search, priority is given to search queries and indexing is slowed down. You can keep your hardware architecture designed to handle both by simply slowing down indexing in the case of a big spike in search queries.
As is the case with using different machines for indexing and search, separating them into different application processes introduces some complexity; for example, the publication of new data for search becomes a multi-process commit.
This problem is pretty common and can easily be solved with the following sequence:
This approach solves the problem of needing to share and prioritize CPU resources between indexing and search but is unfortunately something that most search engines on the market today cannot implement because indexing and search are executed in the same process.
The race for disk resources is a bit more complex to solve. First, we configured our kernel I/O scheduler to assign different priorities to read and write operations via the custom expiration timeout settings within the Linux deadline scheduler. (Read operations expire after 100ms, write operations expire after 10s). Those settings gave us a nudge in the right direction, but this is still far from perfect because the indexing process performs a lot of read operations.
The best way to address the contention for finite disk resources is to make sure the search process does not perform any disk operations, which means that all the data needs to remain in memory. This may seem obvious, but it is the only way to ensure the speed of your search engine is not impacted by indexing operations. It may also seem a bit crazy in terms of costs (having to buy additional memory), but the allocated memory can actually handle the vast majority of use cases without issue. We of course have some users that want to optimize costs for huge amounts of data, but this makes up a very small percentage of our users (less than 1%) and is addressed on an individual basis.
Everything at Algolia is designed with speed and reliability in mind—your data is stored in memory and synced on a high-end SSD and at least three different servers for high availability. Our ultimate goal is to remove all of the pains associated with building a great search feature, and solving the dependency between indexing and search was a very important step in getting there!
We take a lot of pride in building the best possible product for our customers and hope this post gives you some insight into the inner workings of our engine and how we got where we are today. As always, we would love your feedback. Definitely leave us a comment if you have any questions or ideas for the next blog in the series.
We recommend to read the other posts of this series:
Create a full-featured search experience in no time.
Powered by Algolia Recommend