Send and Update Your Data
On this page
Where Do You Push Your Data?
Pushing Your Data to an Algolia Index
There are several ways to push your data to an Algolia index:
- Using one of Algolia’s API clients to push your data (also known as indexing). You don’t have to explicitly create an index. If you try to add objects to a nonexistent index, it will be automatically created for you.
- Using the Algolia dashboard by sending a JSON file.
- Using the Algolia crawler. It lets you automatically crawl your content directly from your site and index it into Algolia. The Algolia crawler is available as an add-on.
Choose a unique, readable name and make sure you don’t use any sensitive or personally identifiable information (PII) as your index name. This includes customer names, user IDs, or email addresses. Index names appear in network requests and should be considered publicly available.
You need an Application ID and a valid API key to push your data. These credentials protect your data, and you can set different access levels for your team.
Every indexing operation (add, update, delete) needs these credentials. The App ID gives you access to your indices, and the appropriate API key gives you rights to perform all indexing operations.
Unique identifier - ObjectID
The engine requires every object to be identified by a unique
We recommend you to set your own internal IDs.
If you don’t, Algolia will generate them for you (like “228506501”), which you can retrieve by browsing the index.
Later, you’ll need to use the
objectIDs for updates and deletes.
Algolia lets you send one or more records in a single API call. When you send several records in the same call, this is called batch sending. You can batch your indexing operations via the API or the Dashboard.
For optimal indexing performance, we strongly encourage you to send your records in batches. We recommend a batch size of ~10 MB, which represents between 1,000 or 10,000 records depending on the average record size.
Batching has many benefits: it reduces network calls and speeds up indexing. Customers with the largest number of records will see the most significant impact on performance. Yet, we recommend everyone to batch indexing operations whenever possible.
Batching does not change how Algolia counts indexing operations. Every indexing operation adds to your count. If you send several records, each of them still counts as one operation.
Updating Your Data
After the initial data import, you’ll need to keep your index up-to-date with the latest additions and changes on your application or website.
The appropriate frequency for updating your Algolia indices depends on how often new data is added to your site, and how quickly it needs to be made searchable.
Let’s take the example of an e-commerce shop:
- You want to update price changes or product availability in real time
- You don’t need to update the number of sales (used for ranking) as often, so you can periodically send them in batches (every hour/day/week)
- More complex use cases include real-time updates, like accurately displaying booking information
You need to find a balance between having fresh information appear in the search as fast as possible and reducing the number of operations, because it has an impact on your pricing and performance.
To give you more control over the consistency of your Algolia data when working in a concurrent environment, the engine supports built-in operations that allow for conditional updates based on versioning and optimistic locking.
If You are Indexing from a Third-Party Platform
Some of our customers are using the below extensions and integration platforms. These platforms usually have their own databases and front ends. The task here is to send your data from these platforms to Algolia and to manage the platform/Algolia relationship, which primarily involves data updates and search configurations.
We’ve built several extensions to simplify the indexation process for popular third-party platforms. If you’re using one of the platforms listed below, we recommend installing our extension for indexing your content.
Additionally, you can follow our tutorials for indexing data from other third-party services: