Haystack EU 2023: Learnings and reflections from our team
If you have built search experiences, you know creating a great search experience is a never ending process: the data ...
Senior ML Engineer
If you have built search experiences, you know creating a great search experience is a never ending process: the data ...
Senior ML Engineer
Just as with a school kid who’s left unsupervised when their teacher steps outside to deal with a distraction ...
Search and Discovery writer
Back in May 2014, we added support for synonyms inside Algolia. We took our time to really nail the details ...
Technical Writer
You’re running an ecommerce site for an electronics retailer, and you’re seeing in your analytics that users keep ...
Technical Writer
What do OpenAI and DeepMind have in common? Give up? These innovative organizations both utilize technology known as transformer models ...
Sr. SEO Web Digital Marketing Manager
As a successful in-store boutique manager in 1994, you might have had your merchandisers adorn your street-facing storefront ...
Search and Discovery writer
At Algolia, our business is more than search and discovery, it’s the continuous improvement of site search. If you ...
JavaScript Library Developer
Analytics brings math and data into the otherwise very subjective world of ecommerce. It helps companies quantify how well their ...
Technical Writer
Amid all the momentous developments in the generative AI data space, are you a data scientist struggling to make sense ...
Sr. SEO Web Digital Marketing Manager
Fashion ideas for guest aunt informal summer wedding Funny movie to get my bored high-schoolers off their addictive gaming ...
Sr. SEO Web Digital Marketing Manager
Imagine you're visiting an online art gallery and a specific painting catches your eye. You'd like to find ...
Senior Software Engineer
At Algolia, our commitment to making a positive impact extends far beyond the digital landscape. We believe in the power ...
Senior Manager, People Success
In today’s post-pandemic-yet-still-super-competitive retail landscape, gaining, keeping, and converting ecommerce customers is no easy ...
Sr. SEO Web Digital Marketing Manager
There are few atmospheres as unique as that of a conference exhibit hall: the air always filled with an indescribable ...
Marketing Content Manager
To consider the question of what vectors are, it helps to be a mathematician, or at least someone who’s ...
Search and Discovery writer
My first foray into programming was writing Python on a Raspberry Pi to flicker some LED lights — it wasn’t ...
Technical Writer
How well do you know the world of modern ecommerce? With retail ecommerce sales having exceeded $5.7 trillion worldwide ...
Sr. SEO Web Digital Marketing Manager
In a world of artificial intelligence (AI), data serves as the foundation for machine learning (ML) models to identify trends ...
Director of AI Engineering
In previous blog posts, we have discussed the high-level architecture of our search engine and our worldwide distributed infrastructure. Now we would like to dive a little deeper into the Algolia search engine to explain why we implemented it from scratch instead of building upon an existing open-source engine.
We have many different reasons for doing so and want to provide ample context for each, so we have split “Inside the Algolia engine” into several posts. As you learn more about our search engine, please let us know if there’s anything you would like us to address in future posts.
If you have ever worked on a search engine with significant traffic and indexing, you are undoubtedly familiar with the problem of trying to fine-tune your indexing to avoid negatively affecting search performance. Part one of this series will focus on one of the quintessential problems with search engines—the impact of indexing on search queries—and our approach to solving it.
Indexing impacts search performance because indexing and search share two critical resources—CPU and Disk. More specifically:
The obvious way to solve this problem is to try to reduce or remove the conflicts of access to the shared resources.
There are a lot of different approaches to dealing with this issue, and the majority fall into one of the following three categories:
While complex to implement, the second approach of using different machines for indexing and search is a good solution if indexing performance is not crucial to you. The other two approaches only partially solve the issue as search remains impacted. Realistically, none of these approaches appropriately solves the problem of indexing affecting search performance because either indexing performance, search performance or both end up suffering.
By splitting the indexing and search into different application processes!
At Algolia, indexing and search are divided into two different application processes with different scheduling priorities. Indexing has a lower CPU priority than search based on a higher nice level (Nice is a tool for modifying CPU priority on Unix-like operating systems). If there is not enough CPU to serve both indexing and search, priority is given to search queries and indexing is slowed down. You can keep your hardware architecture designed to handle both by simply slowing down indexing in the case of a big spike in search queries.
As is the case with using different machines for indexing and search, separating them into different application processes introduces some complexity; for example, the publication of new data for search becomes a multi-process commit.
This problem is pretty common and can easily be solved with the following sequence:
This approach solves the problem of needing to share and prioritize CPU resources between indexing and search but is unfortunately something that most search engines on the market today cannot implement because indexing and search are executed in the same process.
The race for disk resources is a bit more complex to solve. First, we configured our kernel I/O scheduler to assign different priorities to read and write operations via the custom expiration timeout settings within the Linux deadline scheduler. (Read operations expire after 100ms, write operations expire after 10s). Those settings gave us a nudge in the right direction, but this is still far from perfect because the indexing process performs a lot of read operations.
The best way to address the contention for finite disk resources is to make sure the search process does not perform any disk operations, which means that all the data needs to remain in memory. This may seem obvious, but it is the only way to ensure the speed of your search engine is not impacted by indexing operations. It may also seem a bit crazy in terms of costs (having to buy additional memory), but the allocated memory can actually handle the vast majority of use cases without issue. We of course have some users that want to optimize costs for huge amounts of data, but this makes up a very small percentage of our users (less than 1%) and is addressed on an individual basis.
Everything at Algolia is designed with speed and reliability in mind—your data is stored in memory and synced on a high-end SSD and at least three different servers for high availability. Our ultimate goal is to remove all of the pains associated with building a great search feature, and solving the dependency between indexing and search was a very important step in getting there!
We take a lot of pride in building the best possible product for our customers and hope this post gives you some insight into the inner workings of our engine and how we got where we are today. As always, we would love your feedback. Definitely leave us a comment if you have any questions or ideas for the next blog in the series.
We recommend to read the other posts of this series:
Create a full-featured search experience in no time.
Powered by Algolia Recommend