Prepare Your Environment for A/B Testing
On this page
There are a couple of steps you should take to prepare your A/B testing environment.
Set-up Click Analytics
You need to implement Click Analytics to capture click and conversion events, because we use click and conversion rates to do comparative A/B testing.
Add unique user identifiers to your searches
Make sure every search is associated with a unique user identifier. This serves two purposes:
- It ensures randomness. The engine creates two sets of users - one for variant A and another for variant B. Whenever a new user arrives, they’re randomly distributed into A or B, based on a hash of their unique user identifier.
- It maintains the same distribution. If a user was assigned to variant A once, they continue with A and never B. This “user-to-variant stickiness” is mandatory for the A/B test conclusion to be reliable.
Algolia uses unique user tokens to ensure that the same user is always directed to the same variant. By default, the user token is the user’s IP address.
However, you should not rely on this behavior. We recommend that you explicitly create a unique user identifier that you can keep track of in your own application. The user identifier can be sent to Algolia with the
userToken parameter, or with the
X-Algolia-UserToken request header.
Once these steps are completed, you can follow the dashboard’s process to start, stop, and view your tests.
If you want to AB test an index’s setting in the dashboard, select the same index for both variants. A prompt will appear asking you which settings you want to test.