Send and update your data
On this page
Pushing your data to an Algolia index
You can upload your data to Algolia using these methods:
- Algolia CLI
- Algolia’s API clients
- In the Algolia dashboard you can upload a JSON file
- Using the Algolia Crawler. It automatically crawls your content directly from your site and indexes it into Algolia. The Algolia Crawler is available as an add-on.
Naming your index
Choose a unique, readable name and make sure you don’t use any sensitive or personally identifiable information (PII) as your index name. This includes usernames, user IDs, or email addresses. Index names appear in network requests, so you can consider them as publicly available.
Passing Algolia credentials
You need an application ID and a valid API key to push your data. These credentials protect your data, and you can set different access levels for your team.
Every indexing operation requires these credentials. The application ID gives you access to your indices, and the appropriate API key gives you rights to perform indexing operations.
Using unique object identifiers
The engine identifies each object with a unique
objectID, ideally one that you set yourself. If you don’t, Algolia generates them for you. You can retrieve the
objectID by browsing the index. Later on, when you need to update or delete records, you must provide the
Sending records in batches
Algolia lets you send one or more records in a single API call, or batch records. You can batch your indexing operations via the API or the dashboard.
For optimal indexing performance, you should send your records in batches. An ideal batch size is ~10 MB, which represents between 1,000 or 10,000 records depending on the average record size.
Batching has several benefits: it reduces network calls and speeds up indexing. Users with lots of records see the most significant impact on performance, but every Algolia user should batch indexing operations whenever possible.
Batching doesn’t change how Algolia counts indexing operations. Every indexing operation adds to your count. If you send several records, each of them still counts as one operation.
Updating your data
After the initial data import, you need to keep your index up-to-date with the latest changes on your application or website. The appropriate update frequency depends on how often your content changes, and how fast you need it to be searchable.
Since the engine prioritizes search over indexing updates, you should expect some delays when updating an index.
Take the example of an ecommerce store:
- You want to update price changes or product availability in real time.
- You don’t need to update the number of sales (used for custom ranking) as often, so you can periodically send them in batches.
You need to find a balance between having fresh information in the search as fast as possible, and reducing the number of operations. This impacts your pricing and performance.
Handling multiple users
You may use Security-filtering to restrict the data users can see when searching. This allows for dedicated searchable content, which isn’t visible by unauthorized users.
When working in a concurrent environment, you might need more control to guarantee data consistency. The engine supports built-in operations that allow for conditional updates based on versioning and optimistic locking.
Integrating with third-parties
You may use a framework or platform to power your application. Most of these solutions have their own databases and front ends. The goal is to send your data from these platforms to Algolia, and manage data updates and search configurations.
Algolia offers several official integrations to simplify the indexing process for popular frameworks and platforms. If you’re using one of the solutions listed below, you can install the corresponding extension to index your content.