Elasticsearch connector

Use the Elasticsearch connector to index from Elasticsearch 7 or later without writing an indexing script. In this quickstart, you connect an existing Elasticsearch index, use an Algolia destination, transform the records, and create an on-demand synchronization task.

Before you begin

Make sure you have:

An Algolia account. Create one for free if you don’t already have one.
An Elasticsearch 7 or later deployment containing the sample data in an index named apparel. Download the Algolia apparel sample data and add it to the index:
- For Kibana, follow Elastic’s file upload instructions.
- For other ingestion methods, see Ingest: Bring your data to Elastic.

Set up the Elasticsearch connector

Create a source, destination, transformation, and synchronization task for the sample Elasticsearch index. The source identifies the Elasticsearch deployment. The destination identifies the Algolia index. The transformation enriches the records. The task identifies the source Elasticsearch index and determines when the connector runs.

Choose the Elasticsearch connector

Go to the Algolia dashboard and select your Algolia application. Open the Connectors page.Find Elasticsearch, then select Connect.

Configure the Elasticsearch source

Under Configure your data source, select Create a new source.Enter the following details:

Field	Value
Server Endpoint	The Elasticsearch endpoint URL
Authentication method	Api Key/Secret
API Key ID	The ID for an Elasticsearch API key that can access the `apparel` index
API Key Secret	The secret for that API key

In Connector name, enter Quickstart products Elasticsearch source, then select Create source.

Transform the records

This transformation adds a price_range attribute to each record. After the connector indexes your records, you can display price_range or configure it as a facet.Select Transform using the code editor and replace the placeholder Transformation code with this function:

JavaScript

async function transform(record, helper) {
    const price = Number(record.price);
    if (!Number.isFinite(price)) {
        return record;
    }
    if (price < 25) {
        record.price_range = "Under $25";
    } else if (price < 50) {
        record.price_range = "$25 to $49";
    } else if (price < 100) {
        record.price_range = "$50 to $99";
    } else {
        record.price_range = "$100 and up";
    }
    return record;
}

Select Save.

Choose the destination Algolia index

Under Configure your destination, select Create a new destination.Under Search, enter quickstart-products as the index name.

If you enter the name of an existing index, the connector overwrites the records and settings in that index.

Create the destination

Under Index credentials, select Create one for me.To use an existing API key, choose one with the addObject, deleteIndex, and editSettings ACL permissions.In Name, enter Elasticsearch quickstart products destination, then select Create destination.

Create and run an on-demand synchronization task

Under Configure your task, select On demand.Select Full reindexing.Under Which table do you want to sync?, select apparel as the source Elasticsearch index and select the objectID field as the ObjectID unique identifier.Select Create task, then Run and wait for the task to finish.

Verify the indexed records

Open the quickstart-products index in the Algolia dashboard.The index contains the imported product records, with attributes such as title, description, product_type, price, price_range, and showcase_image.

You can use this index as the data source for Build your first search experience.Before you build the UI, configure product_type as an attribute for faceting in the quickstart-products index.

Index your own Elasticsearch data

To index your own data, configure the connector with your Elasticsearch server, choose an authentication method and destination, and create a synchronization task for the Elasticsearch indices you want to import.

Configure the connector for your data

Use a populated Elasticsearch index that the Algolia connector can reach. If you restrict inbound access to your Elasticsearch server, allow the Algolia connector IP addresses. Follow the quickstart steps to create the source, destination, and task. Replace the sample endpoint, credentials, source index, and destination index with your own values. Review or remove the sample transformation so that it matches your record structure. When you configure the task, select the field to use as the ObjectID. The field must be present in every source document, and its value must be unique and stable.

Elasticsearch authentication

To create an API key and secret, use Kibana or the Elasticsearch API. Elasticsearch API keys include an ID and secret. The Algolia connector requires these values separately. If Elasticsearch gives you only an encoded credential, decode it locally to recover the ID and secret. For more information, see Elasticsearch API keys. When you configure the Elasticsearch source, choose one of these authentication methods:

None: the connector reads your data without authentication.
Api key/Secret: enter the decoded API key ID and API key secret.
Username/Password: enter the username and password the connector should use.

For Api key/Secret or Username/Password authentication, grant read-only access to the Elasticsearch indices you want to import. Use the following privileges:

Cluster: monitor
Indices: read, view_index_metadata, and monitor

Scope the index privileges to the relevant index names or patterns. For Username/Password authentication, choose a user whose roles grant the same access.

The connector reads from Elasticsearch and writes to the Algolia destination. It doesn’t need permission to create Elasticsearch indices or documents.

Transformations

Use transformations to add computed attributes or change values before the connector indexes your records. To use a transformed attribute for search, faceting, or ranking, update the relevant index settings, such as searchableAttributes, attributesForFaceting, or customRanking.

Test transformations with repeated task runs. Transformations that read from and write to the same attribute, or extract a value and then delete the source attribute, can produce different results when the task runs again.

Synchronization schedule

After you create a task, you can edit it in the Algolia dashboard to choose when the connector runs:

On demand. Run the connector manually.
Scheduled. Select a predefined schedule or enter a custom cron expression.

Indexing strategy

You can choose how the connector updates your index. For information about full reindexing, full record updates, and partial record updates, see Data synchronization strategies.

Schedule full reindexing tasks only as often as necessary. Each run reads all documents from the selected Elasticsearch indices and rewrites the destination, which increases the workload on Elasticsearch and the number of Algolia indexing operations.

Limitations

This connector is subject to the following limitations:

Search and Discovery platform

Optimization and Personalization

AI-powered experiences

Production and scale

Elasticsearch connector

Before you begin

Set up the Elasticsearch connector

Index your own Elasticsearch data

Configure the connector for your data

Elasticsearch authentication

Transformations

Synchronization schedule

Indexing strategy

Limitations

See also

​Before you begin

​Set up the Elasticsearch connector

​Index your own Elasticsearch data

​Configure the connector for your data

​Elasticsearch authentication

​Transformations

​Synchronization schedule

​Indexing strategy

​Limitations

​See also

Before you begin

Set up the Elasticsearch connector

Index your own Elasticsearch data

Configure the connector for your data

Elasticsearch authentication

Transformations

Synchronization schedule

Indexing strategy

Limitations

See also