In the past decade, artificial intelligence and predictive search has made our lives easier and faster in many areas. Predicting keystrokes, words, products, and phrases is one of the areas where artificial intelligence has made its mark. Under the hood of many ecommerce and media sites, you’ll find high-end AI technologies that are used to improve sales with advanced natural language processing. The science behind NLP for understanding user’s queries and predicting the best search results was made possible through machine learning. Today, Google uses BERT, which helps return meaningful results of queries that users haven’t previously typed, no matter how they spell or combine the words in the query.

Predictive search, in particular, refers to a useful user experience (UX) tool that suggests queries as users type. Query suggestions help people formulate the best query to find the products they are looking for. In this article, you’ll gain a complete understanding of how predictive search works, common pitfalls of predictive search, and how to avoid those pitfalls, along with the benefits and best practices for implementing a common predictive search experience – namely, autocomplete.

What is predictive search?

Most of us have experienced the wonderful world of autocomplete, where you type the first few letters of what you’re looking for into a search engine and click on one of the search suggestions from the drop-down, but there’s a lot that goes on in the background to make this possible. The ability to select from a list of suggestions without having to type the complete term in the search box is the core functionality of predictive search.

Google set the standard, being the first organization to implement an autocomplete feature. Very quickly, users grew to expect predictive search. Amazon, Shopify, Spotify, and Walmart are some examples of sites and applications that have implemented predictive search capability to show popular queries, offer contextual suggestions, avoid typos, and filter suggestions based on a user’s location or preferences. Today, organizations, big and small, offer some form of autocomplete.

How does predictive search work?

Predictive search technology attempts to anticipate search terms based on the behavior, previous searches, geolocation, and other attributes of the end user, as well as trending searches across all user sessions, and displays the terms as suggestions in or under the search field. Autocomplete, autosuggest, and predictive search are often used interchangeably in the tech industry, especially in the UI/UX department.

A search query is the term that a user types in the search bar of a page. Predictive search analyzes each letter of the search query as they are being typed and provides a set of suggested queries in a drop-down menu before the user has even finished typing. These suggestions are generated by a series of algorithms. Multiple machine learning and natural language processing algorithms and models are used to generate matches, starting with a simple string in order to identify, match, and predict the outcome of the unfinished search query.

These suggestions help users to get results faster with less typing, which helps prevent typos and create queries that will return the desired results. The algorithms behind these suggestions begin with simple string matching from static lists or existing data, and get increasingly complex by using natural language processing to manage typos and synonyms to help users get the desired results.

Predictive search – some UI/UX fundamentals

Here are some best practices to create a good user experience with autocomplete searches:

• Add visual focus when active: ensure that the search bar and autocomplete suggestions are focused when in use. You can see how it works in the video above — note how when the search bar is selected, the rest of the page is dimmed. Especially on smaller mobile devices, reduce the icons, shortcuts, or menu items that are displayed beside the search bar to help in minimizing visual distraction on screen.
• Use text wrapping for suggestions: if the suggestions contain lengthy text, avoid using a horizontal scroll bar. Ensure the lengthy keywords and terms are large enough for readability, but are constrained to a single screen. If necessary, use text wrapping and allow suggestions to spread across multiple rows, especially on mobile devices.
• Avoid scrollbars: it’s important to ensure that suggestions aren’t hidden on the screen. This means avoiding scroll bars wherever possible. Scrollbars increase the time users spend searching, making it more likely that they’ll abandon the search, and making it more difficult for them to find the content that they want.
• Provide visual hints that the list is scrollable: sometimes it’s impossible to avoid using scrollbars. In cases like this, give enough hints and instructions that it’s clear the list is scrollable.
• Enable easy removal of queries: provide clear visual icons and options for the user to clear the search bar and autocomplete suggestions. The most widely used indicator is the “X” icon, which is provided to help the user remove a search query. This design solution drastically improves the overall user experience.
• Optimize suggestions: part of the appeal of autocomplete suggestions is faster results, and these results should be displayed in real time, as shown below. Suggestions should appear almost as soon as the user types the first character, and should update with each character added to the query. Typos and other errors should be filtered so as to return relevant suggestions.
• Limit the number of options being listed: it’s important to limit the number of suggestions shown in response to a user query. Too many suggestions can result in irrelevant results and distracted users. It’s a best practice to keep the suggestions to ten items or less.

Instant search

Instant search is a cousin of predictive search. As a user types a query, results are displayed in near real-time. Instant search is executing a new search with every keystroke, which improves the search experience by driving discovery visually. Visual sites, such as ecommerce sites with product photos, can really benefit as instant search enables buyers to scan the results faster.

Instant search can be built on the back of autocomplete or other predictive search technologies, but at Algolia, we took a different approach using radix trees (also called radix tries or compressed tries… more on this below), a data structure that’s already sorted and works well for typo-tolerance.

Handling typos with prefix search

The radix tree is used for something else: prefix search. Prefix search helps us retrieve results as someone types each character; it starts document sorting and matching using the beginning of a term. If you type “t”, prefix search will retrieve results for “toast” or “toaster”; the engine must know that both of these terms contain the prefix “t”.

This approach also helps with alternate spellings (e.g., “gray” or “grey”) and typos (“iphxne” instead of “iphone”). Also called typo tolerance, it allows users to make mistakes while typing and still find the records they’re looking for. This kind of predictive matching works for words that are close in spelling — the  difference in spelling between a typed word and its exact match in the index. A perfect match is distance = 0. When there is a perfect match, or the distance is low (one or two letters mistakenly typed), then a match is made, and the record is added to the results.

It’s extremely fast to calculate, and often faster than more sophisticated AI models. If you’re familiar with vector search, you know it can help with typos, too. While vectors may contain embeddings for misspellings and provide a degree of typo tolerance, the radix tree can compute prefixes on the fly. Keyword search is actually much faster than vector search for this use case and can work for a much broader array of misspellings. It can help someone find an Apple device with an “aplp” prefix.

Best practices for complex autocomplete UI/UX

Predictive search results are not only influenced by matching strings, but also popular searches and user data that get fed into machine learning algorithms. These algorithms are continually learning from user behavior, improving the accuracy with which they predict what the user is searching for and what results might be relevant.

Regardless of the algorithm used to implement predictive search, the relevance, structure, and design of the search suggestions are critical for user interaction and overall user experience. Having a well-designed predictive search that provides relevant and responsive search suggestions is crucial. Here are some more best practices:

• Use ranking to determine display order: there are many factors that contribute to how the suggestions provided are ranked. Generally speaking, queries related to the user’s location, language, recent search history, previous order history, popular searches, and ongoing promotions should be ranked highly. For example, if it’s near a holiday, suggestions related to that holiday should be ranked higher to ensure that they’re displayed in the first few options of the suggestions dropdown.
• Style different types of suggestions differently: in some instances, there may be search results and suggestions that include both products and categories. In these situations, ensure that the different categories of suggestions are clearly differentiated. The text could be styled differently, or different colors could be used for different types of suggestions.
• Highlight potential outcomes: when displaying a number of suggestions, make sure to highlight potential outcomes of the search term provided by the user in the autocomplete suggestions. This helps the user determine the differences and decide faster. If the user has typed “cou”, for example, you’ll want to show different potential outcomes, such as couch, countertop, and counter.
• Personalize autocomplete suggestions: consider the queries based on the user’s location, language, and search history in order to personalize the results. If a user located in a cold place like Oslo or Toronto searches for “shoes” in December, the suggestions presented to them should be useful to someone in those locations: boots or warm shoes, not fabric athletic shoes or sandals.

Benefits of predictive search

So, why spend the time implementing predictive search? What is the business value? Predictive search increases conversion rates and saves time for users when searching for an item by providing helpful auto suggestions. It’s an important functionality when shopping from a mobile phone that has a compact keyboard. Having the predictive search feature in the application reduces friction for users, making it less likely that they’ll abandon their search.

Here are a few of the benefits of using predictive search:

• It reduces search time by offering suggestions, which means the user will rarely have to enter their entire query
• Predictive search offers a more personalized experience, informed by users’ previous queries, location, and preferred categories
• It educates users about your site and product, and helps users find what they’re looking for by providing relevant images and descriptions for certain products
• It increases sales by showing users appropriate and useful suggestions based on popular searches and bestselling products
• By making content available in a more optimized fashion, it increases and improves the overall user experience
• It increases the number of relevant queries, and offers optimized results

Predictive search helps to avoid confusion caused by typos and errors, and offers faster results by filtering the data.

Conclusion

Predictive search offers many advantages, including an increased conversion rate and a better customer experience, and is a crucial component in the success of many businesses. Using the best practices described in this article will help you make sure that your site search works as hard as you do.