> ## Documentation Index
> Fetch the complete documentation index at: https://algolia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-variant testing

> Test more than one variant at a time versus the control.

A **multi-variant test** (sometimes written A/B/n) compares one **control variant** with two or more **test variants** in a single experiment. Instead of running several A/B tests in sequence you try every idea in parallel.

## Why run multi-variant tests?

* **Speed.** You discover the best setting faster because every variant is evaluated at the same time. There will be less time to gather enough samples than if you tested 2 or more variants sequentially.
* **Fair comparison.** All variants share the same traffic window, removing calendar effects like holidays or seasonal campaigns.
* **Less engineering overhead.** One set-up, one end date, one set of results to review.

## Limitations and trade-offs

* **Traffic is divided.** With more variants each one receives a smaller share of users, so it takes longer to collect enough data. This is still less time and risk than running sequential tests, however.
* **Statistical penalty.** Every additional comparison increases the risk of a "lucky" winner. Correct for this (see below) by making the confidence threshold stricter.

### Number of variants

You can create up to five variants per test (1 control, 4 test variants). This limit is in place because a test with too many variants extends the test duration too far.

## How Algolia keeps results reliable

By default, Algolia applies the [**Benjamini-Hochberg**](https://en.wikipedia.org/wiki/False_discovery_rate#Benjamini%E2%80%93Hochberg_procedure) (BH) method. You can also choose the more conservative [**Bonferroni**](https://en.wikipedia.org/wiki/Bonferroni_correction) method:

| Method                           | Corrected threshold |
| -------------------------------- | ------------------- |
| **Benjamini-Hochberg** (default) | αᵢ = (i/m) × 0.05   |
| **Bonferroni**                   | α = 0.05 / *m*      |

where *m* is the number of comparisons and *i* is the rank of the p-value when they're sorted from smallest to largest.

A comparison is marked **Confident** when its p-value is smaller than (or equal to) its corrected α.
With Bonferroni, if there's only one test variant (*m* = 1) the threshold stays at 0.05 - just like a regular two-variant test.