Concepts / Getting insights and analytics / Query Suggestions
Jan. 11, 2019

Query Suggestions

Query Suggestions - What is it?

Algolia has introduced a new version of its Analytics service (v2) which doesn’t yet work with Query Suggestions. We will soon provide a newly-hosted version of this service along with the other Analytics services. In the meantime, if you have any question regarding this feature, please feel free to reach out to support@algolia.com.

Query Suggestions is a well-known UI experience that many of the best websites offer. Providing suggestions helps the user type longer queries. For mobile users, it reduces the amount of typing required to execute a search. But it’s not just about typing less. It’s far more powerful - it helps users formulate the best search. Good suggestions are proven queries that ensure the best results. The quality of the suggestions is as important as the results they generate.

At Algolia, we use Query Suggestion to extend the conversational search experience. It is as if the user, with every keystroke, is conversing with the search bar, discussing with it exactly what to send the search engine to obtain the best results. For this to work, suggestions need to be instantly responsive and intuitively relevant. If either one of these is missing, the conversation is over.

How it works

Query Suggestions precedes the actual search:

  • As the user types, an initial search processes each keystroke, searching through an index of carefully curated suggestions. This results in selecting a small set of similar queries (starting with the same prefix). The user can of course ignore the suggestions and keep typing; with every keystroke, a new list of suggestions will appear.

  • When a user selects a suggestion, that action sends the suggestion as a query off to the main index.

Check out our demo.

All Enterprise customers can have access to Query Suggestions from their dashboard, but this feature needs to be enabled first. Please reach out to the Algolia Success Team at support@algolia.com to enable Query Suggestions on your dashboard.

Getting Started with Query Suggestions

To get started with Query Suggestions, you’ll need to think about two very different subjects:

  • The Query Suggestions Index: How to populate, configure, and maintain the index behind the suggestions
  • Your front-end implementation: How to integrate it into your front end solution

We’ll first focus on configuring the Query Suggestions Index, as this is the essence of Query Suggestions - to create an index of good suggestions based on analytics.

If you want to learn how to build your Query Suggestions interface, we offer guidance below.

Query Suggestions is a separate Index

Query Suggestions uses a dedicated index to suggest the most relevant/most used queries. This is not a new API endpoint, nor an API feature, but a clever use of an index.

Configuring the index with the Dashboard

Query Suggestions is available in the Dashboard to all Enterprise customers, but you need to enable it first. Please reach out to the Algolia Success Team at support@algolia.com to enable this feature on your dashboard.

You can use our Query Suggestions dashboard to do everything you need in order to manage the data and configuration.

  • Create your index
  • Import data
  • Setup attributes or categories
  • Configure settings like languages, minimum length for prefix
  • Enable/Disable data sources
  • Status updates
  • Manage banned expressions

Query Suggestions - How it Works

Query Suggestions is entirely data-driven, suggesting the best-matched searches based on a comparison between the terms typed and a collection of stored queries.

Internally, a suggestion is based on a specific query with the count, which is the number of times it has been typed in.

1
2
3
4
{
  "query": "iphone X",
  "count": "255"
}

Whatever the source of data, the suggestion engine uses a popularity score to determine what queries to suggest. Every input uses the same format for its suggestions: the query + the number of times it has been used. If the same query comes in from more than one source, we add the values together and use the total as the score for that query.

An example

There are 2 important aspects to keep in mind with query suggestions:

  • The content of suggestions - which suggestions appear
  • The order of the suggestions - which appear at the top and which appear below

Take a look at these suggestions for the query “iph”:

  • iphone
  • iphone 6
  • iphone 8
  • iphone charger

These suggestions seem appropriate and relevant: “iph” suggests “iphone”, and the most popular queries for iphone are either models or accessories. Good so far.

However, iphone X is missing and 6 is higher than 8. This is most likely due to the current analytics data, which relies on popularity.

  • From a content point of view, X is only a recent release, so it is not yet popular enough to be included as a suggestion.
  • From an order point of view, 6 has been on the market longer than 8, so it appears above 8, even though 8 might have been the better, more up-to-date / relevant suggestion.

The solution? We regenerate your suggestions index every 24 hours, and take into account only the last 30 days of analytics data. So if more people search for 8 than 6, it will end up on top. This is still biased by the initial position though - that’s why we let our customers upload ‘external analytics’ data to feed the suggestions and their popularity scores.

Building your query suggestions index

All suggestions are stored in a separate, single Query Suggestions Index. This index is different from your main index.

There are 3 sources that can populate the Query Suggestions index:

  • Algolia Analytics
  • your own datasets of suggestions, like Google Analytics or derived from your own database
  • filters & facets from your main indices

You can use one source, or combine several sources together.

Algolia Analytics

Algolia Analytics is done behind the scene. You just agree to use Algolia’s Search Analytics (in your configuration), and we do the rest - capture the necessary metrics and build the analytic-based suggestion index. While we capture the data in real-time, we rebuild the index once every 24 hours.

Query Suggestions is a perfect use case for Analytics data. We base our suggestions on Popular Searches.

External data / Your own suggestions

Analytics data is only one source. When you get started, you’ll need some kind of starting data. You can either upload your own suggestions, or we can use your main index’s filters and facets to help build your suggestions index.

To do this, you can place your own suggestions in a JSON format and upload the content. This data can be made up of what you consider ideal queries; or you can feed in analytics data from other sources, including Google Analytics. It just needs to follow the same format as Algolia’s own analytics data.

Here is an example of JSON file you can send:

1
2
3
4
5
6
7
8
9
10
11
[
  {
    "query": "iphone",
    "count": "10031"
  },
  {
    "query": "samsung",
    "count": "731"
  }
  [...]
]

Using your index’s searchable facets

You can also use your main index to generate query suggestions by using facets. You send the query suggestions engine attributes that are set up as searchable facets. These will be resolved into very precise suggestions. For example, if you send the engine “brand + color”, you can generate a query for every brand + color combination:

  • “nike red”
  • “nike blue”
  • “adidas red”
  • “Adidas blue”

These queries will be sent along with their respective scores. Normally, they come in with low scores, being based on the amount of items that match those search terms.

This method is used primarily when you have no initial data - neither analytics nor your own suggestions. In the long run, these facet-based suggestions will most likely be less relevant (that is, less popular) than your analytics-based ones.

You’ll need to have large activity to justify using Query Suggestions. We also have a 10000 limit of suggestions by the Algolia analytics. So large customers will want to supplement the data with Google or other kinds of analytics.*

Note: the analytics on your replicas will be taken into account.

Filtering-out unwanted queries

You’ll need to be careful - by default, every search is captured and uploaded. Therefore, Query Suggestions can be misled - the data can be mistaken, irrelevant, or even completely embarrassing! One company, for example, had received a flood of phony, inappropriate searches, which, due to their apparent popularity, scored higher than other searches. This meant that other users began to see these phony searches.

We have 4 systems in place to limit such inappropriate suggestions and to prevent fraud.

Minimum number of results required

By default, the suggestions index only includes queries that have returned at least 5 results. This limit is configurable, yet keep in mind that you’ll need an Algolia Admin to assist you.

Minimum popularity required

We do not suggest queries that fall below a certain minimum popularity score.

IP restrictions

We apply a distinct IP per query: if an IP sends many requests for the same query, it will always count as 1. This creates a barrier to an accidental or abusive flood of requests from the same IP.

Blacklisted words

Perhaps the most powerful way to tell the engine what not to do, is to use our banned list functionality. With blacklisting, you can remove any unwanted queries. There are 2 methods to do this:

  • A list of exact matches: whole queries can be eliminated with an easy-to-use blacklist, which is a list of partial words, words, and phrases that the engine will ignore. The query must find an exact match in the list to be ignored.
  • You can also create one or more expressions that look for certain inappropriate or unwanted patterns in queries. Here, you can implement partial matching: any query that contains a blacklisted expression is ignored.

Displaying Query Suggestions

Query Suggestions vs. Autocomplete

Query Suggestions is essentially an autocomplete UI experience.

Algolia already offers an Autocomplete library as an alternative to the InstantSearch libraries, in which both display as-you-type search results. Autocomplete displays its results in a dropdown box, usually with a short text and small thumbnail images. InstantSearch displays its results directly on the page, with room for full-size images, more descriptive text, and a variety of filters and categories.

Query Suggestions is a different kind of autocomplete: it also uses a dropdown box and changes its contents as the user types, but it does not display results. Instead, it displays suggested queries that are executed only when the user selects one of them.

Combining InstantSearch with Query Suggestions

A particularly advanced and useful UI combines the power of Query Suggestions with InstantSearch results. This would mean that, as the user types, two areas of the screen will be changing: (1) queries will be suggested while (2) new results appear. Seeing both can help users make a decision. Check out our InstantSearch + Query Suggestions demo to see how this could look.

Combining suggestions and results in the same drop box

Another choice is to combine suggestions and results within the same dropdown box. This is an especially good UI for mobile, which simplifies the UX by having suggestions + results. Check out our mobile android guide and mobile ios guide.

Adding scope

A useful enhancement to the list of suggestions is to add categories / scope into the results. So instead of, or in addition to, suggesting queries, you suggest filters. For example:

If someone types in “iphone”, the standard approach is to present useful queries: “iphone 7” “iphone charger” “iphone X cases” “iphone X speakers”

But you can also present categories:

iphone

  • in phones
  • in accessories
  • in electronics

Or a combination of both.

This can be done by configuring Query Suggestions with categories from your main index.

Did you find this page helpful?