> ## Documentation Index
> Fetch the complete documentation index at: https://algolia.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Deduplicate recommendations
> Remove variants from your recommendations by using Recommend deduplication.
export const Application = () =>
application
;
export const AlgoliaRecommend = () => ;
export const AlgoliaSearch = () => ;
With Recommend deduplication, you can refine your recommendations by removing item variants from your AI model training.
The feature offers several advantages:
* **Improves recommendation accuracy and speed** by removing item variants before the training process.
* **Improves recommendation quality** by using events from all item variants.
* **Separates deduplication from search** so it doesn't affect the search experience.
## Recommend deduplication versus search deduplication
Algolia offers two types of deduplication:
* **Search deduplication** [removes duplicate items from search results](/doc/guides/managing-results/refine-results/grouping). This can help improve the relevance of the results and make them more user-friendly.
* **Recommend deduplication** remove duplicate items from Algolia AI recommendations. This can help improve the diversity of the recommendations and make them more personalized.
With Recommend deduplication,
your AI models are **trained with all variants**,
and the recommendations then **filtered by the API** before sending you the results to remove variants.
## How Recommend deduplication works
Recommend deduplication adds two processes to your AI model training:
* **Pre-training process**: generates a training dataset with only one variant per item. It also merges all events from variants of the same item.
* **Post-training process**: add all the variants dropped during the pre-training process back into the final set of recommendations.
Item variants share the same recommendations.
## Set up the deduplication for a model
To deduplicate your recommendations, you must first declare an attribute for distinguishing variants, then turn on deduplication when configuring a Recommend model.
After that, verify that the recommendations have been deduplicated.
### Configure an attribute for distinguishing variants
First, choose which attribute defines records as variants:
1. Go to the [Algolia dashboard](https://dashboard.algolia.com/explorer/browse) and select your Algolia .
2. On the left sidebar, select **Search**.
3. Select your Algolia index.
4. On the [**Configuration**](https://dashboard.algolia.com/explorer/configuration/deduplication-and-grouping) tab, go to the **Deduplication and Grouping** page.
5. In the **Attribute for Distinct** box,
select or enter the attribute name you want to use to define variants.
Only use the `distinct` option if you also want to [deduplicate search results](/doc/guides/managing-results/refine-results/grouping).
### Enable Recommend deduplication on your model
1. Go to the [Algolia dashboard](https://dashboard.algolia.com/recommend) and select your Algolia application.
2. On the left sidebar, select **Recommend**.
3. Create a new Recommend model or edit an existing one for the index you used when setting the [`attributeForDistinct`](/doc/api-reference/api-parameters/attributeForDistinct).
4. In the section **Distinct and deduplication of recommendations**, select the **Deduplicate recommendations** option. The attribute you selected for defining variants is shown.
5. Continue to configure your Recommend model and click **Save**.
### Verify the recommendations
To check that the deduplication is working for your recommendations,
revisit the model configuration once the training process has finished:
1. Go to the **Preview** section.
2. Use the **Search for a record** box to search for an item that should have variants.
This displays the list of recommendations for the selected item.
They shouldn't contain any variants.
## Examples
The following examples illustrate how Recommend deduplication works.
The index has records for T-shirts in different colors and sizes:
* One **red** T-shirt in one size (*XS*)
* Two **green** T-shirts in two sizes (*S*, *M*)
* Three **blue** T-shirts in three sizes (*L*, *XL*, *XXL*)
This example uses the [**Related Items** model](/doc/guides/algolia-recommend/overview#recommend-models) to recommend the top 3 similar items with and without deduplication (with the `color` attribute configured as `attributeForDistinct`).
### Without deduplication
Without deduplication,
recommendations include variants,
such as **blue** (*XXL*) or **green** (*M*),
except for the **red** T-shirt which doesn't have any.
| Base item | Recommendation 1 | Recommendation 2 | Recommendation 3 |
| --------------- | ---------------- | ---------------- | ---------------- |
| **red** (*XS*) | **green** (*S*) | **blue** (*L*) | **blue** (*XL*) |
| **green** (*S*) | **green** (*M*) | **red** (*XS*) | **blue** (*L*) |
| **blue** (*L*) | **blue** (*XL*) | **blue** (*XXL*) | **green** (*M*) |
### With deduplication
With deduplication, the recommendations don't include any variants.
However, since this example dataset only includes three records, it can only generate two recommendations.
If you introduce a new item, such as an **orange** T-shirt, it will be added as a third recommendation.
| Base item | Recommendation 1 | Recommendation 2 | Recommendation 3 |
| --------------- | ---------------- | ---------------- | ---------------- |
| **red** (*XS*) | **green** (*S*) | **blue** (*L*) | Doesn't apply |
| **green** (*S*) | **red** (*XS*) | **blue** (*L*) | Doesn't apply |
| **blue** (*L*) | **red** (*XS*) | **green** (*M*) | Doesn't apply |
### Verify the recommendations
To check that the deduplication is working for your recommendations,
revisit the model configuration once the training process has finished:
1. Go to the **Preview** section.
2. Use the **Search for a record** box to search for an item that should have variants.
This displays the list of recommendations for the selected item.
They shouldn't contain any variants.
## Examples
The following examples illustrate how Recommend deduplication works.
The index has records for T-shirts in different colors and sizes:
* One **red** T-shirt in one size (*XS*)
* Two **green** T-shirts in two sizes (*S*, *M*)
* Three **blue** T-shirts in three sizes (*L*, *XL*, *XXL*)
This example uses the [**Related Items** model](/doc/guides/algolia-recommend/overview#recommend-models) to recommend the top 3 similar items with and without deduplication (with the `color` attribute configured as `attributeForDistinct`).
### Without deduplication
Without deduplication,
recommendations include variants,
such as **blue** (*XXL*) or **green** (*M*),
except for the **red** T-shirt which doesn't have any.
| Base item | Recommendation 1 | Recommendation 2 | Recommendation 3 |
| --------------- | ---------------- | ---------------- | ---------------- |
| **red** (*XS*) | **green** (*S*) | **blue** (*L*) | **blue** (*XL*) |
| **green** (*S*) | **green** (*M*) | **red** (*XS*) | **blue** (*L*) |
| **blue** (*L*) | **blue** (*XL*) | **blue** (*XXL*) | **green** (*M*) |
### With deduplication
With deduplication, the recommendations don't include any variants.
However, since this example dataset only includes three records, it can only generate two recommendations.
If you introduce a new item, such as an **orange** T-shirt, it will be added as a third recommendation.
| Base item | Recommendation 1 | Recommendation 2 | Recommendation 3 |
| --------------- | ---------------- | ---------------- | ---------------- |
| **red** (*XS*) | **green** (*S*) | **blue** (*L*) | Doesn't apply |
| **green** (*S*) | **red** (*XS*) | **blue** (*L*) | Doesn't apply |
| **blue** (*L*) | **red** (*XS*) | **green** (*M*) | Doesn't apply |