Prepare Your Environment for A/B Testing

Prepare your environment

There are a couple of steps you should take to prepare your A/B testing environment.

Set-up Click Analytics

You need to implement Click Analytics to capture click and conversion events, because we use click and conversion rates to do comparative A/B testing.

Add unique user identifiers to your searches

Make sure every search is associated with a unique user identifier. This serves two purposes:

  • It ensures randomness. The engine creates two sets of users - one for variant A and another for variant B. Whenever a new user arrives, they’re randomly distributed into A or B, based on a hash of their unique user identifier.
  • It maintains the same distribution. If a user was assigned to variant A once, they continue with A and never B. This “user-to-variant stickiness” is mandatory for the A/B test conclusion to be reliable.

A unique user token is the mechanism by which Algolia ensures that the same user is always directed to the same variant. By default, the user token is the user’s IP address. However, you should not rely on this behavior. We recommend that you explicitly create a unique user identifier that you can keep track of in your own application. The user identifier can be sent to Algolia with the userToken parameter, or with the X-Algolia-UserToken request header.

A/B Testing

Once these steps are completed, you can follow the Dashboard’s process to start, stop, and view your tests.

If you want to AB test an index’s setting in the dashboard, select the same index for both variants. A prompt will appear asking you which settings you want to test.

Did you find this page helpful?