> ## Documentation Index
> Fetch the complete documentation index at: https://algolia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B test implementation checklist

> Assess your A/B test implementation.

export const SearchQuery = () => <Tooltip tip="The text users enter into a search box. In the Search API, this corresponds to the query parameter. A search query is often used with filters, facets, and other parameters, but these aren't part of the query text itself.">
    search query
  </Tooltip>;

export const ClickThroughRate = () => <Tooltip tip="The percentage of searches with `clickAnalytics` set to `true` that result in at least one click on a search result." cta="Click-through rate" href="/doc/guides/search-analytics/concepts/metrics#click-through-rate">
    click-through rate (CTR)
  </Tooltip>;

Use this checklist to avoid common issues with [statistical significance](/doc/guides/ab-testing/what-is-ab-testing/in-depth/how-ab-test-scores-are-calculated#statistical-significance-or-chance) and unexpected test results.

## Check your events

Check that all events are in the right format and are valid.
For more information, see [Validate your events](/doc/guides/sending-events/guides/validate).

## Identify your users

Link click and conversion events with user profiles by assigning a unique identifier, [`userToken`](/doc/guides/sending-events/concepts/usertoken), to each user.

## Handle backend search users

If you use [backend search](/doc/guides/building-search-ui/going-further/backend-search/js), since all searches appear to come from your server's IP address, Algolia treats them as coming from a single user.

To differentiate between users, [include the `userToken` or the user's IP address](https://support.algolia.com/hc/en-us/articles/4406981881489-Will-Analytics-work-for-back-end-searches) when forwarding queries from your server to Algolia.

## Check Personalization implementation

If you're using Personalization, check it's working as expected with the [Personalization implementation help](https://dashboard.algolia.com/personalization/implementation) page on the Algolia dashboard. Your queries should have associated user tokens.

Personalization is available on the Build and Premium [pricing plans](https://www.algolia.com/pricing/).

## Handle anonymous users

Sometimes, you may have a mix of anonymous and identified users. For example, you may have assigned an [anonymous token](/doc/guides/sending-events/concepts/usertoken#persistent-user-token) to users who haven't yet accepted cookie consent.
Since anonymous users share the same user token, Algolia treats them as coming from a single user.

To avoid skewing results, turn off A/B testing for queries from anonymous users.
For example, the following checks for an anonymous user token (`null`, `undefined`, or `YOUR_ANONYMOUS_USER_TOKEN`) and
appends the `{ "[enableABTest](parameter_url('enableABTest'))": false }` parameter to the outgoing <SearchQuery />.

<CodeGroup>
  ```cs C# theme={"system"}
  namespace Algolia;

  using System;
  using System.Collections.Generic;
  using System.Net.Http;
  using System.Text.Json;
  using Algolia.Search.Clients;
  using Algolia.Search.Http;
  using Algolia.Search.Models.Search;

  class AbTestImplementationChecklist
  {
    private static string GetUserToken()
    {
      // Implement your logic here
      return "";
    }

    async Task Main(string[] args)
    {
      var client = new SearchClient(new SearchConfig("ALGOLIA_APPLICATION_ID", "ALGOLIA_API_KEY"));

      // Set the query and get the current user token
      var query = "User search query";
      var userToken = GetUserToken();

      // Set the searchParams
      var searchParamsObject = new SearchParamsObject { Query = query };

      // Is the user token anonymous?
      if (string.IsNullOrEmpty(userToken) || userToken == "YOUR_ANONYMOUS_USER_TOKEN")
      {
        // Disable A/B testing for this request
        searchParamsObject.EnableABTest = false;
      }
      else
      {
        // Set the user token to the current user token
        searchParamsObject.UserToken = userToken;
      }

      var searchParams = new SearchParams(searchParamsObject);

      try
      {
        // Perform the searchSingleIndex
        var result = await client.SearchSingleIndexAsync<Hit>("INDEX_NAME", searchParams);
        // SearchSingleIndex results
        Console.WriteLine(result);
      }
      catch (Exception err)
      {
        // SearchSingleIndex errors
        Console.Error.WriteLine(err);
      }
    }
  }

  ```

  ```dart Dart theme={"system"}
  import 'package:algolia_client_search/algolia_client_search.dart';

  String getUserToken() {
    // Implement your logic here
    return "";
  }

  void abTestImplementationChecklist() async {
    final client =
        SearchClient(appId: 'ALGOLIA_APPLICATION_ID', apiKey: 'ALGOLIA_API_KEY');

    // Set the query and get the current user token
    final query = "User search query";
    final userToken = getUserToken();

    // Set the searchParams
    final SearchParamsObject searchParams;

    // Is the user token anonymous?
    if (userToken.isEmpty || userToken == "YOUR_ANONYMOUS_USER_TOKEN") {
      // Disable A/B testing for this request
      searchParams = SearchParamsObject(
        query: query,
        enableABTest: false,
      );
    } else {
      // Set the user token to the current user token
      searchParams = SearchParamsObject(
        query: query,
        userToken: userToken,
      );
    }

    try {
      // Perform the searchSingleIndex
      final result = await client.searchSingleIndex(
        indexName: "INDEX_NAME",
        searchParams: searchParams,
      );
      // SearchSingleIndex results
      print(result);
    } catch (err) {
      // SearchSingleIndex errors
      print(err);
    }
  }

  ```

  ```go Go theme={"system"}
  package main

  import (
  	"github.com/algolia/algoliasearch-client-go/v4/algolia/search"
  )

  func getUserToken() (string, error) {
  	// Implement your logic here
  	return "", nil
  }

  func abTestImplementationChecklist() {
  	client, err := search.NewClient("ALGOLIA_APPLICATION_ID", "ALGOLIA_API_KEY")
  	if err != nil {
  		// The client can fail to initialize if you pass an invalid parameter.
  		panic(err)
  	}

  	// Set the query and get the current user token
  	query := "User search query"

  	userToken, err := getUserToken()
  	if err != nil {
  		panic(err)
  	}

  	// Set the searchParams
  	var searchParams *search.SearchParams

  	// Is the user token anonymous?
  	if userToken == "" || userToken == "YOUR_ANONYMOUS_USER_TOKEN" {
  		// Disable A/B testing for this request
  		searchParams = search.SearchParamsObjectAsSearchParams(
  			search.NewSearchParamsObject().
  				SetQuery(query).
  				SetEnableABTest(false),
  		)
  	} else {
  		// Set the user token to the current user token
  		searchParams = search.SearchParamsObjectAsSearchParams(
  			search.NewSearchParamsObject().
  				SetQuery(query).
  				SetUserToken(userToken),
  		)
  	}

  	// Perform the searchSingleIndex
  	_, err = client.SearchSingleIndex(client.NewApiSearchSingleIndexRequest(
  		"INDEX_NAME").WithSearchParams(searchParams))
  	if err != nil {
  		// SearchSingleIndex errors
  		panic(err)
  	}
  }

  ```

  ```java Java theme={"system"}
  package com.algolia;

  import com.algolia.api.SearchClient;
  import com.algolia.config.*;
  import com.algolia.model.search.*;

  public class abTestImplementationChecklist {

    private static String getUserToken() {
      // Implement your logic here
      return "";
    }

    public static void main(String[] args) throws Exception {
      SearchClient client = new SearchClient("ALGOLIA_APPLICATION_ID", "ALGOLIA_API_KEY");

      // Set the query and get the current user token
      String query = "User search query";
      String userToken = getUserToken();

      // Set the searchParams
      SearchParamsObject searchParams = new SearchParamsObject().setQuery(query);

      // Is the user token anonymous?
      if (userToken == null || userToken.isEmpty() || userToken.equals("YOUR_ANONYMOUS_USER_TOKEN")) {
        // Disable A/B testing for this request
        searchParams.setEnableABTest(false);
      } else {
        // Set the user token to the current user token
        searchParams.setUserToken(userToken);
      }

      try {
        // Perform the searchSingleIndex
        SearchResponse result = client.searchSingleIndex("INDEX_NAME", searchParams, Hit.class);
        // SearchSingleIndex results
        System.out.println(result);
      } catch (Exception err) {
        // SearchSingleIndex errors
        System.err.println(err);
      }
    }
  }

  ```

  ```js JavaScript theme={"system"}
  import { algoliasearch } from 'algoliasearch';

  import type { SearchParams } from 'algoliasearch';

  const client = algoliasearch('ALGOLIA_APPLICATION_ID', 'ALGOLIA_API_KEY');

  // Implement your logic here
  const getUserToken = () => {
    return '';
  };

  // Set the query and get the current user token
  const query = 'User search query';
  const userToken = getUserToken();

  // Set the searchParams
  const searchParams: SearchParams = {
    query,
  };

  // Is the user token anonymous?
  if (userToken === null || userToken === undefined || userToken === 'YOUR_ANONYMOUS_USER_TOKEN') {
    // Disable A/B testing for this request
    searchParams.enableABTest = false;
  } else {
    // Set the user token to the current user token
    searchParams.userToken = userToken;
  }

  try {
    // Perform the searchSingleIndex
    const result = await client.searchSingleIndex({ indexName: 'indexName', searchParams: searchParams });
    // SearchSingleIndex results
    console.log(result);
  } catch (err) {
    // SearchSingleIndex errors
    console.error(err);
  }

  ```

  ```kotlin Kotlin theme={"system"}
  import com.algolia.client.api.SearchClient
  import com.algolia.client.configuration.*
  import com.algolia.client.extensions.*
  import com.algolia.client.model.search.*
  import com.algolia.client.transport.*

  fun getUserToken(): String {
    // Implement your logic here
    return ""
  }

  suspend fun abTestImplementationChecklist() {
    val client = SearchClient(appId = "ALGOLIA_APPLICATION_ID", apiKey = "ALGOLIA_API_KEY")

    // Set the query and get the current user token
    val query = "User search query"
    val userToken = getUserToken()

    // Set the searchParams
    val searchParams: SearchParamsObject

    // Is the user token anonymous?
    if (userToken.isEmpty() || userToken == "YOUR_ANONYMOUS_USER_TOKEN") {
      // Disable A/B testing for this request
      searchParams = SearchParamsObject(query = query, enableABTest = false)
    } else {
      // Set the user token to the current user token
      searchParams = SearchParamsObject(query = query, userToken = userToken)
    }

    try {
      // Perform the searchSingleIndex
      val result =
        client.searchSingleIndex(indexName = "INDEX_NAME", searchParams = searchParams)
      // SearchSingleIndex results
      println(result)
    } catch (err: Exception) {
      // SearchSingleIndex errors
      println(err)
    }
  }

  ```

  ```php PHP theme={"system"}
  <?php

  require __DIR__.'/../vendor/autoload.php';
  use Algolia\AlgoliaSearch\Api\SearchClient;
  use Algolia\AlgoliaSearch\Model\Search\SearchParamsObject;

  $getUserToken = function (): string {
      // Implement your logic here
      return '';
  };

  $client = SearchClient::create('ALGOLIA_APPLICATION_ID', 'ALGOLIA_API_KEY');

  // Set the query and get the current user token
  $query = 'User search query';
  $userToken = $getUserToken();

  // Set the searchParams
  $searchParams = (new SearchParamsObject())
      ->setQuery($query)
  ;

  // Is the user token anonymous?
  if (null === $userToken || '' === $userToken || 'YOUR_ANONYMOUS_USER_TOKEN' === $userToken) {
      // Disable A/B testing for this request
      $searchParams->setEnableABTest(false);
  } else {
      // Set the user token to the current user token
      $searchParams->setUserToken($userToken);
  }

  try {
      // Perform the searchSingleIndex
      $result = $client->searchSingleIndex(
          'INDEX_NAME',
          $searchParams,
      );
      // SearchSingleIndex results
      var_dump($result);
  } catch (Exception $err) {
      // SearchSingleIndex errors
      error_log($err->getMessage());
  }

  ```

  ```python Python theme={"system"}
  from algoliasearch.search.client import SearchClientSync

  from algoliasearch.search.models.search_params import SearchParams


  def _get_user_token() -> str:
      # Implement your logic here
      return ""


  _client = SearchClientSync("ALGOLIA_APPLICATION_ID", "ALGOLIA_API_KEY")


  # Set the query and get the current user token
  query = "User search query"
  user_token = _get_user_token()

  # Set the searchParams
  # Is the user token anonymous?
  if user_token is None or user_token == "" or user_token == "YOUR_ANONYMOUS_USER_TOKEN":
      # Disable A/B testing for this request
      search_params = SearchParams(
          query=query,
          enable_ab_test=False,
      )
  else:
      # Set the user token to the current user token
      search_params = SearchParams(
          query=query,
          user_token=user_token,
      )

  try:
      # Perform the searchSingleIndex
      result = _client.search_single_index(
          index_name="INDEX_NAME",
          search_params=search_params,
      )
      # SearchSingleIndex results
      print(result)
  except Exception as err:
      # SearchSingleIndex errors
      print(err)

  ```

  ```ruby Ruby theme={"system"}
  require "algolia"

  def get_user_token
    # Implement your logic here
    ""
  end

  client = Algolia::SearchClient.create("ALGOLIA_APPLICATION_ID", "ALGOLIA_API_KEY")

  # Set the query and get the current user token
  query = "User search query"
  user_token = get_user_token

  # Set the searchParams
  search_params = Algolia::Search::SearchParamsObject.new(
    query: query
  )

  # Is the user token anonymous?
  if user_token.nil? || user_token.empty? || user_token == "YOUR_ANONYMOUS_USER_TOKEN"
    # Disable A/B testing for this request
    search_params.enable_ab_test = false
  else
    # Set the user token to the current user token
    search_params.user_token = user_token
  end

  begin
    # Perform the searchSingleIndex
    result = client.search_single_index("INDEX_NAME", search_params)
    # SearchSingleIndex results
    puts(result)
  rescue => err
    # SearchSingleIndex errors
    puts(err)
  end

  ```

  ```scala Scala theme={"system"}
  import scala.concurrent.Future
  import scala.concurrent.ExecutionContext.Implicits.global

  import algoliasearch.api.SearchClient
  import algoliasearch.config.*
  import algoliasearch.extension.SearchClientExtensions
  import algoliasearch.search.SearchParamsObject

  val getUserToken: String = {
    "" // Implement your logic here
  }

  def abTestImplementationChecklist(): Future[Unit] = {
    val client = SearchClient(appId = "ALGOLIA_APPLICATION_ID", apiKey = "ALGOLIA_API_KEY")

    // Set the query and get the current user token
    val query = "User search query"
    val userToken = getUserToken

    // Set the searchParams
    var searchParams = SearchParamsObject(
      query = Some(query)
    )

    // Is the user token anonymous?
    if (userToken == null || userToken.isEmpty || userToken == "YOUR_ANONYMOUS_USER_TOKEN") {
      // Disable A/B testing for this request
      searchParams = searchParams.copy(enableABTest = Some(false))
    } else {
      // Set the user token to the current user token
      searchParams = searchParams.copy(userToken = Some(userToken))
    }

    // Perform the searchSingleIndex
    client
      .searchSingleIndex(
        indexName = "INDEX_NAME",
        searchParams = Some(searchParams)
      )
      .map { result =>
        // SearchSingleIndex results
        println(result)
      }
      .recover { case err: Exception =>
        // SearchSingleIndex errors
        println(s"An error occurred: ${err.getMessage}")
      }
  }

  ```

  ```swift Swift theme={"system"}
  import Foundation
  #if os(Linux) // For linux interop
      import FoundationNetworking
  #endif

  import AlgoliaCore
  import AlgoliaSearch

  let getUserToken = { "" } // Implement your logic here

  func abTestImplementationChecklist() async throws {
      let client = try SearchClient(appID: "ALGOLIA_APPLICATION_ID", apiKey: "ALGOLIA_API_KEY")

      // Set the query and get the current user token
      let query = "User search query"
      let userToken = getUserToken()

      // Set the searchParams
      // Is the user token anonymous?
      let searchParams =
          if userToken.isEmpty || userToken == "YOUR_ANONYMOUS_USER_TOKEN" {
              // Disable A/B testing for this request
              SearchSearchParams.searchSearchParamsObject(
                  SearchSearchParamsObject(query: query, enableABTest: false)
              )
          } else {
              // Set the user token to the current user token
              SearchSearchParams.searchSearchParamsObject(
                  SearchSearchParamsObject(query: query, userToken: userToken)
              )
          }

      do {
          // Perform the searchSingleIndex
          let result: SearchResponse<Hit> = try await client.searchSingleIndex(
              indexName: "INDEX_NAME",
              searchParams: searchParams
          )
          // SearchSingleIndex results
          print(result)
      } catch {
          // SearchSingleIndex errors
          print(error)
      }
  }

  ```
</CodeGroup>

## Exclude internal searches

Exclude non-user searches (such as from a dashboard or internal page) from A/B testing by setting [`enableABTest`](/doc/api-reference/api-parameters/enableABTest) to false.

## Avoid overriding A/B test settings with query parameters

For accurate results, don't change any query parameters you initially set in the A/B test.
For example, you turn on Personalization for variant B and all queries. This wouldn't give meaningful results because all users see the personalized experience, making it difficult to tell if variant B has any impact.

## Manage bot traffic

Bots crawling your web pages and performing searches will generate outlier traffic. Although outliers are [automatically removed](/doc/guides/ab-testing/how-to-read-your-a-b-test-results#automatic-outlier-exclusion) from the A/B test results,
you should still take steps to avoid them because they can affect the <ClickThroughRate />.

To control and restrict bot activity and avoid skewing results:

* **Set rate limits.** Set [API keys with rate limits](/doc/guides/security/api-keys/in-depth/api-key-restrictions#rate-limit) to control the number of API calls per hour and IP address.
* **Use HTTP referrers.** However, don't rely solely on referrer URLs sent through the `referer` or `Origin HTTP` headers to secure your data. Malicious actors can tamper with them, compromising your security, and some browsers, like Brave, don't send these headers.
* **Update robots.txt.** Configure [`robots.txt`](https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt) to prevent bots from accessing your search pages.