NeuralSearch A/B testing

Before you enable NeuralSearch for all search traffic, run an A/B test to measure how it affects your search metrics. NeuralSearch uses Algolia’s standard A/B testing features. You can create tests with the A/B testing API or from the Algolia dashboard.

Choose a test type

You can run one of these test types:

Test type	Control	Variant	When to use
Keyword vs NeuralSearch	Keyword search	NeuralSearch	Compare keyword search with NeuralSearch
Adaptive Intent test	NeuralSearch	NeuralSearch with Adaptive Intent	Compare NeuralSearch to NeuralSearch with Adaptive Intent

Compare keyword search with NeuralSearch

Your index must have NeuralSearch enabled before you can launch an A/B test. NeuralSearch can be in one of these states:

Inactive: NeuralSearch isn’t configured for the index.
Preview: NeuralSearch is configured for testing but doesn’t serve all search traffic.
Active: NeuralSearch is enabled for the index’s search traffic.

To change the NeuralSearch state for an index, set neuralSearchMode appropriately. This A/B test evaluates whether NeuralSearch improves your existing search experience.

The control uses the index’s default keyword search.
The variant enables NeuralSearch.

Command line

curl --request POST \
  --url https://analytics.algolia.com/2/abtests \
  --header 'content-type: application/json' \
  --header 'x-algolia-api-key: {ALGOLIA_ANALYTICS_API_KEY}' \
  --header 'x-algolia-application-id: {ALGOLIA_APPLICATION_ID}' \
  --data '{
    "name": "Keyword vs NeuralSearch",
    "variants": [
      {
        "index": "{INDEX_NAME}",
        "trafficPercentage": 50,
        "description": "Keyword search (control)"
      },
      {
        "index": "{INDEX_NAME}",
        "trafficPercentage": 50,
        "customSearchParameters": {
          "enableNeuralSearch": true
        },
        "description": "NeuralSearch (variant)"
      }
    ],
    "endAt": "2026-12-31T00:00:00.000Z"
  }'

Compare NeuralSearch to NeuralSearch with Adaptive Intent

This A/B test measures whether Adaptive Intent improves NeuralSearch results.

The control uses NeuralSearch without Adaptive Intent.
The variant enables Adaptive Intent.

Command line

curl --request POST \
  --url https://analytics.algolia.com/2/abtests \
  --header 'content-type: application/json' \
  --header 'x-algolia-api-key: {ALGOLIA_ANALYTICS_API_KEY}' \
  --header 'x-algolia-application-id: {ALGOLIA_APPLICATION_ID}' \
  --data '{
    "name": "NeuralSearch vs NeuralSearch with Adaptive Intent",
    "variants": [
      {
        "index": "{INDEX_NAME}",
        "trafficPercentage": 50,
        "description": "NeuralSearch (control)"
      },
      {
        "index": "{INDEX_NAME}",
        "trafficPercentage": 50,
        "customSearchParameters": {
          "adiConfig": {
            "enabled": true
          }
        },
        "description": "NeuralSearch with Adaptive Intent (variant)"
      }
    ],
    "endAt": "2026-12-31T00:00:00.000Z"
  }'

Start tests from the Algolia dashboard

To start NeuralSearch A/B tests from the Algolia dashboard:

Go to the Algolia dashboard and select your Algolia application.
Select your index.
Click the NeuralSearch tab.
Click Launch an A/B test.
Select the test type: Keyword vs NeuralSearch or Adaptive Intent test.
Set the traffic split and test duration.
Click Create test.

The Algolia dashboard pre-fills the required customSearchParameters for each test type, so you don’t need to configure the API parameters manually.

Test metrics

Algolia tracks these metrics automatically for every NeuralSearch A/B test:

Conversion rate
Click-through rate
Add-to-cart rate
No-results rate
Average click position

For more information, see Click and conversion events.

Test duration

Run tests long enough to reach statistical significance:

Tests should run for at least 14 days to account for weekly traffic patterns.
Aim for a minimum increase of 3% on conversion rate.
Indices with more traffic usually reach statistical significance faster.

Stopping a test

Stop tests from the Algolia dashboard or with the A/B testing API. After you stop a test, traffic returns to the index’s active configuration automatically.

​Choose a test type

​Compare keyword search with NeuralSearch

​Compare NeuralSearch to NeuralSearch with Adaptive Intent

​Start tests from the Algolia dashboard

​Test metrics

​Test duration

​Stopping a test