Dynamic Re-Ranking

The relevance of your search results often changes with time, and it can change fast. Take a drugstore as an example. Users who searched for “mask” before February 2020 (the outbreak of the COVID-19 pandemic) most probably were looking to buy facial masks to take care of their skin. During the pandemic, users were more likely to look for surgical masks.

To address users’ changing expectations, you might have to tweak your results using tools like Rules or optionalFilters. This can become time-consuming because trends come and go fast. A better, more scalable approach is to use Dynamic Re-Ranking.

What is Dynamic Re-Ranking?

Dynamic Re-Ranking is an Algolia feature that uses AI to find trends in your users’ behavior. Based on the query and the position of the result they click or convert, it can improve your relevance by boosting results that are rising in popularity. Indirectly, Dynamic Re-Ranking will also demote less relevant results by boosting other results above them.

Dynamic Re-ranking can learn from user behavior patterns. For example, if users frequently choose surgical masks when searching for “mask”, it will increase the ranking of surgical masks in future results. This leads to a more relevant search experience and can result in a higher conversion rate without you having to create rules or use optional filters to tweak relevance.

Effect on relevance

While Dynamic Re-Ranking can be a powerful tool for tweaking your relevance, you don’t want it to interfere with the relevance you’ve already set up. For this reason, Dynamic Re-Ranking kicks in after the engine has computed your results’ textual relevance and has applied your custom ranking.

Dynamic Re-Ranking only moves results that are part of the result set, following any eventual filters. It cannot insert new records in the result set.

If you use Rules to pin or hide specific records, the engine applies them after re-ranking so that they keep working as expected. The engine applies all other Rules before re-ranking. This means re-ranking can override Rules that boost and bury categories. Re-ranking may also override an eventual Geo filtering.

Re-ranking doesn’t take effect if you enable Personalization and Algolia has enough data to personalize the search. However, re-ranking kicks back in if there’s not enough data to personalize a search for first-time users. The engine doesn’t apply both Personalization and Dynamic Re-Ranking concurrently.

The engine applies the relevance strategies in this order:

1. Textual ranking criteria
2. Business relevance (custom ranking)
3. Personalization (if you’ve enabled it and Algolia has enough information to personalize results for a particular user)
4. Boosting and burying categories with rules or Automatic Boosting from Query Categorization
5. Dynamic Re-Ranking
6. Pinning and hiding specific items with rules.

By default, Algolia doesn’t apply re-ranking on empty searches but you can enable re-ranking for empty searches filtered on a particular facet value.

Simulate Dynamic Re-Ranking

You can simulate re-ranking for any index from the Dynamic Re-Ranking page in the dashboard. Once you’ve sent some click and conversion events, you can try out any query and see how Dynamic Re-Ranking affects the results (without activating re-ranking on your live index).

In the simulator, each record has an attractiveness score per query. This score is the number of events this record received last month compared to other records, and it explains why re-ranking boosts one record over another.

If you enabled the time decay option, attractiveness scores will be impacted: a record with less events may have a higher score than a record that received more events if the first record received them recently.

If you use rules to pin top items for popular queries, Dynamic Re-Ranking has a limited effect, since rules take precedence over it. Turn off the rules in the simulator with the Rules toggle to assess the benefits of Dynamic Re-Ranking. Consider removing these rules if you’re satisfied with the records that Dynamic Re-Ranking promotes.

Dynamic Re-Ranking preparation

Dynamic Re-Ranking is available on the Build and Premium pricing plans. Some options are only available if your plan includes Advanced Dynamic Re-Ranking.

Click and conversion events requirements

To use Dynamic Re-Ranking, you must send click and conversion events. Dynamic Re-Ranking uses this data to detect trends and re-rank results based on these trends.

Dynamic Re-Ranking may work with only clicks events (and no conversion events), but it will perform better if you send both click and conversion events.

Dynamic Re-Ranking refreshes each query’s ranking every 24 hours by default. Re-Ranking takes the last 30 days into account, meaning it’s based on a sliding window of the most recent analytics data. For a record to be promoted for a given query, Re-Ranking requires it to have at least 20 clicks or two conversions during the 30-day time window.

Events must contain a queryID property that corresponds to a valid search. The same userToken property needs to be passed both in the search query and in the event. Only one click or conversion event is considered per userToken per search query per record (objectID).

Events must include an objectID property for a record currently in the specified index. This objectID needs to belongs to one of the returned records of the search with the specified queryID. For example:

• A search returned the record X on the first day of the month.
• An event for record X, in the production index, is sent with the search corresponding queryID
• Record X will be used for the next Dynamic Re-Ranking ordering computations.
• On the fifth day of the month, record X is deleted from the production index
• This means record X won’t be used for Dynamic Re-Ranking ordering computations anymore.

Dynamic Re-Ranking may not be suitable for indices with frequent record deletions, such as those used by a second-hand car dealer where each car’s record is deleted upon sale. We advise you to use instead Automatic boosting from Query Categorization.

Security best practices

To prevent malicious users from manipulating the ranking of your records, Dynamic Re-Ranking uses a distinct userToken per event policy. That means when a userToken sends duplicate events for the same user-query-record combination, only one counts for re-ranking For example, if the same user (same userToken) clicks on the same record 100 times after the same query, it counts as one click. If the same user clicks on the same record after different queries, they count as separate events.

To prevent users from crafting several userTokens, use secured API keys. With secured API keys, you can include the userToken as an embedded parameter when you generate the key on your backend. This lets you give each user a unique key, but they can’t change its userToken.

Dynamic Re-Ranking settings

Configure the following settings from the Dynamic Re-Ranking page in the dashboard.

General settings

Events source index

By default, Dynamic Re-Ranking uses the events sent to the re-ranked index. You can use another index as the source of the events by changing the Events source index option. The indices must be replicas of each other.

You can use this parameter to experiment with your Dynamic Re-Ranking configuration without affecting production search results:

• Create a replica of your production index
• Enable Dynamic Re-Ranking on the replica
• Set the event source index to your production index to use its events to generate the replica’s re-ranking (which have been empty otherwise due to the lack of events on the replica)
• Test changes to your Dynamic Re-Ranking configuration in the simulator on the replica
• Eventually, start an A/B test by comparing your production index with the replica

Hourly refresh

This option is only available if your plan includes Advanced Dynamic Re-Ranking.

By default, Dynamic Re-Ranking is refreshed every 24 hours. By enabling this option, it will refresh every hours instead.

Enabling hourly refresh better handles shorter search trends.

Coverage settings

Multi-Signal Ranking

This option is only available if your plan includes Multi-Signal Ranking.

By default, Dynamic Re-Ranking uses your events to re-rank your top queries and improve their relevance.

Multi-Signal Ranking improves the ordering of less popular queries: it doesn’t affect ordering of your top queries.

For the less popular queries, an Algolia AI model is trained on your events. It considers the textual relevance of each record compared to the query, and the content of the record itself. This AI model training runs every day, and can take up to an hour. Once the training is finished, the newly re-ranked queries will affect search (as for any Dynamic Re-Ranking query).

To train the model, choose up to 20 record attributes. Choose attributes you show to users that affect the “attractiveness” of a record. For example, brand and price are good candidates but SKU isn’t.

The attributes must be boolean, a number, or a string. Arrays and objects aren’t supported. You can mark each attribute as:

• date for numbers expressed as a Unix timestamp.
• category for string or number values shared across your records. Good examples are brand and color. Don’t use category for unique fields like productDescription.
• unclassifiedfor anything else.

To see which attributes affected ranking the most, download a CSV from the dashboard. The closer the value is to 0, the less impact the attribute has on re-ranking. Negative values indicate that the attribute lowers ranking. For example, the attribute isOutOfStock is likely to be negative since out of stock items are usually not desirable. Multi-Signal Ranking will learn to demote attributes with a negative impact.

Enable re-ranking on browsing pages (Browsing Facet)

By default, Algolia doesn’t dynamically re-rank empty queries. Setting a Browsing Facet re-ranks empty queries that include facetFilters for that attribute.

For example, if you use Algolia to populate your website’s category pages by performing an empty query with a categoryPageID facet, you can set categoryPageID as a browsing facet. Re-ranking reorders records in each categoryPageID based on their popularity.

If you set the root of a nested faceting attribute as a browsing facet, Dynamic Re-Ranking reorders all its sub-attributes For example, if you have an attribute named hierarchicalCategories with several nested levels (like hierarchicalCategories.level0 and hierarchicalCategories.level1), using only hierarchicalCategories as a browsing facet re-ranks empty queries filtered on hierarchicalCategories.level0 and hierarchicalCategories.level1 as well.

You can set up to five browsing facets if your plan includes Advanced Dynamic Re-Ranking. Otherwise, you can use only one browsing facet.

Enable re-ranking on empty query by default

This option is only available if your plan includes Advanced Dynamic Re-Ranking.

By default, Algolia doesn’t dynamically re-rank empty queries. By enabling this option, Dynamic Re-Ranking will always re-rank the empty query, promoting the most popular records overall.

If you have setup browsing facets, the fallback re-ranking of the empty query will only be triggered if none of the browsing facets were passed in the search query.

Re-rank more results per query (Re-ranked Hits)

You can increase the maximum number of records pushed to the top of search results from the default of 20 to 100.

Dynamic Re-Ranking may promote fewer results than this value if Dynamic Re-Ranking doesn’t gather significant traffic on enough records for some queries, or if filters remove records that were about to be re-ranked.

Ordering settings

Give more weight to recent events (Event Freshness)

To give more weight to recent events in the ranking computation, enable the Event Freshness option. If you don’t, an event that happened four weeks ago affects re-ranking the same way as an event that happened yesterday. Enabling event freshness better handles seasonality and shorter search trends.

Group Similar Queries

By default, each query is re-ranked independently. Group Similar Queries helps to re-rank less frequent query variations.

Queries are grouped if they share the exact same words (in any order).

Setting a language for grouping allows Dynamic Re-Ranking to ignore plurals, conjugations, and stop words during the grouping phase.

Grouped queries will share their events and, hence, will have the same ordering.

Turn off re-ranking for specific records (Re-Ranking Filters)

If you set a re-ranking filter, Dynamic Re-Ranking only promotes items matching the configured filters.

For example, if you select inStock = true, Dynamic Re-Ranking only promotes items that are in stock.

Enable Dynamic Re-Ranking for production

You can enable Re-Ranking on the Dynamic Re-Ranking page in the dashboard. Select the index you want to activate Re-Ranking for and press the Activate Re-Ranking button.

A/B test Dynamic Re-Ranking

You can use A/B testing to evaluate the impact of re-ranking on your index. This way, you can accurately measure re-ranking’s effect on your search. You can create an A/B test for re-ranking directly from the Dynamic Re-Ranking page in the dashboard by clicking the Launch an A/B test button at the top right of the page. Clicking this button shows a form that lets you create an A/B test with the desired traffic split and duration, testing the search performances with and without Dynamic Re-Ranking.

Turn off Dynamic Re-Ranking for specific queries

If you want to turn Dynamic Re-Ranking off on a specific query, you can create a rule and add as a consequence {"enableReRanking" : false}.