Search by Algolia
Vector vs Keyword Search: Why You Should Care
ai

Vector vs Keyword Search: Why You Should Care

Search has been around for a while, to the point that it is now considered a standard requirement in many ...

Nicolas Fiorini

Senior Machine Learning Engineer

What is a B2B marketplace?
e-commerce

What is a B2B marketplace?

It’s no secret that B2B (business-to-business) transactions have largely migrated online. According to Gartner, by 2025, 80 ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

3 strategies for B2B ecommerce growth: key takeaways from B2B Online - Chicago
e-commerce

3 strategies for B2B ecommerce growth: key takeaways from B2B Online - Chicago

Twice a year, B2B Online brings together industry leaders to discuss the trends affecting the B2B ecommerce industry. At the ...

Elena Moravec

Director of Product Marketing & Strategy

Deconstructing smart digital merchandising
e-commerce

Deconstructing smart digital merchandising

This is Part 2 of a series that dives into the transformational journey made by digital merchandising to drive positive ...

Benoit Reulier
Reshma Iyer

Benoit Reulier &

Reshma Iyer

The death of traditional shopping: How AI-powered conversational commerce changes everything
ai

The death of traditional shopping: How AI-powered conversational commerce changes everything

Get ready for the ride: online shopping is about to be completely upended by AI. Over the past few years ...

Aayush Iyer

Director, User Experience & UI Platform

What is B2C ecommerce? Models, examples, and definitions
e-commerce

What is B2C ecommerce? Models, examples, and definitions

Remember life before online shopping? When you had to actually leave the house for a brick-and-mortar store to ...

Catherine Dee

Search and Discovery writer

What are marketplace platforms and software? Why are they important?
e-commerce

What are marketplace platforms and software? Why are they important?

If you imagine pushing a virtual shopping cart down the aisles of an online store, or browsing items in an ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What is an online marketplace?
e-commerce

What is an online marketplace?

Remember the world before the convenience of online commerce? Before the pandemic, before the proliferation of ecommerce sites, when the ...

Catherine Dee

Search and Discovery writer

10 ways AI is transforming ecommerce
e-commerce

10 ways AI is transforming ecommerce

Artificial intelligence (AI) is no longer just the stuff of scary futuristic movies; it’s recently burst into the headlines ...

Catherine Dee

Search and Discovery writer

AI as a Service (AIaaS) in the era of "buy not build"
ai

AI as a Service (AIaaS) in the era of "buy not build"

Imagine you are the CTO of a company that has just undergone a massive decade long digital transformation. You’ve ...

Sean Mullaney

CTO @Algolia

By the numbers: the ROI of keyword and AI site search for digital commerce
product

By the numbers: the ROI of keyword and AI site search for digital commerce

Did you know that the tiny search bar at the top of many ecommerce sites can offer an outsized return ...

Jon Silvers

Director, Digital Marketing

Using pre-trained AI algorithms to solve the cold start problem
ai

Using pre-trained AI algorithms to solve the cold start problem

Artificial intelligence (AI) has quickly moved from hot topic to everyday life. Now, ecommerce businesses are beginning to clearly see ...

Etienne Martin

VP of Product

Introducing Algolia NeuralSearch
product

Introducing Algolia NeuralSearch

We couldn’t be more excited to announce the availability of our breakthrough product, Algolia NeuralSearch. The world has stepped ...

Bernadette Nixon

Chief Executive Officer and Board Member at Algolia

AI is eating ecommerce
ai

AI is eating ecommerce

The ecommerce industry has experienced steady and reliable growth over the last 20 years (albeit interrupted briefly by a global ...

Sean Mullaney

CTO @Algolia

Semantic textual similarity: a game changer for search results and recommendations
product

Semantic textual similarity: a game changer for search results and recommendations

As an ecommerce professional, you know the importance of providing a five-star search experience on your site or in ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What is hashing and how does it improve website and app search?
ai

What is hashing and how does it improve website and app search?

Hashing.   Yep, you read that right.   Not hashtags. Not golden, crisp-on-the-outside, melty-on-the-inside hash browns ...

Catherine Dee

Search and Discovery writer

Conference Recap: ECIR23 Take-aways
engineering

Conference Recap: ECIR23 Take-aways

We’re just back from ECIR23, the leading European conference around Information Retrieval systems, which ran its 45th edition in ...

Paul-Louis Nech

Senior ML Engineer

What is a neural network and how many types are there?
ai

What is a neural network and how many types are there?

Your grandfather wears those comfy slipper-y shoes all day, every day, and they’re starting to get holes in ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Looking for something?

facebookfacebooklinkedinlinkedintwittertwittermailmail

When we developed the first version of Algolia Search, we put a lot of effort into developing a data update API. It worked like this: You could send us a modified version of your data as soon as the change appeared, even if it concerned only a specific part of a record. For example, this batch of information could be the updated price or number of reviews, and we would only update this specific attribute in your index.

However, this initial plan did not take into account that most of our big customers would not benefit from this API due to their existing infrastructure. If you had not planned to catch all updates in your architecture, or if you were not using a framework like Ruby on Rails, it could be very difficult to even have a notification for any of these updates. The solution in this case was to use a batch update on a regular basis. It was a good method to use if you didn’t want to change a single line of code in your existing infrastructure, but the batch update was far from a cure-all.

The problem of batch update

There are two main ways to perform a batch update on a regular basis:

  1. Scan your database and update all objects. This method is good if you have no delete operation, but if some data are removed from your database, you will need to perform an extra check to handle delete, which can be very slow.
  2. Clear the content of the index and import all your objects. With this method, you ensure that your index is well synchronized with your database. However, if you receive queries during the import, you will return partial results.  If interrupted, the whole rescan could break your relevance or your service.

So the two approaches are somewhat buggy and dangerous.

Another approach: build a new index with another name

Since our API allows the creation of a new index with a different name, you could have made your batch import in a new index. Afterward, you would just need to update your front end to send queries to this new index.
Since all indexing jobs are done asynchronously, we first need to check that an indexing job is finished. In order to do that, we return an integer (called TaskID) that allows you to check if an update job is applied. Thus, you just have to use the API to check that the job is indexed.
But then a problem arises with mobile applications: You cannot change the index name of an application as easily, since most of the time, it is a constant in the application code. And even for a website, it means that the batch will need to inform your frontend that the index name is different. This can be complex.

The elegant solution: move operation

To solve these problems, we implemented a command that is well known on file systems: move. You can move your new index on the old one, and this will atomically update the content of the old index with the content of the new one. With this new approach, you can solve all the previous update problems with one simple procedure. Here’s how you would update an index called “MyIndex”:

  1. Initialize an index “MyIndex.tmp”
  2. Scan your database and import all your data in “MyIndex.tmp”
  3. Move “MyIndex.tmp in “MyIndex”

You don’t have to do any modification on your backend to catch modifications, nor do you need to change the index name on the frontend. Even better, you don’t need to check the indexing status with our TaskID system since the “move” operation will simply be queued after all “adds”. All queries will go to the new index when it is ready.

The beauty of the move command

This command is so elegant that even customers who had been sending us realtime updates via our updates API have decided to use this batch update on a regular basis. The move command is a good way to ensure that there are no bugs in your update code, nor divergence between your database and Algolia.

This operation is supported in our twelve API Clients. We go even further in our Ruby on Rails integration: You need only use the ‘reindex’ command (introduced in 1.10.5) to automatically build a new temporary index and move it on top of the existing one.

The move command is an example of how we try to simplify the life of developers. If you see any other way we can help you, let us know and we’ll do our best to remove your pain!

About the author
Julien Lemoine

Co-founder & former CTO at Algolia

githublinkedintwitter

Recommended Articles

Powered byAlgolia Algolia Recommend

Algolia's top 10 tips to achieve highly relevant search results
product

Julien Lemoine

Co-founder & former CTO at Algolia

12 ways to improve your search index
engineering

Jon Silvers

Director, Digital Marketing

Search Index 101: Everything You Wanted to Know…
product

Peter Villani

Sr. Tech & Business Writer