Semantic search and why it matters for e-commerce

With the economic downturn, I hear from customers all the time about the need to do more with less. How, I’m asked, can Algolia help them not only grow sales and improve margins, but with fewer resources at their disposal?

The answer I’m most excited about today is semantic search. Semantic search is one of the most versatile sets of technologies you can add to site search today. Retailers, for example, can make it vastly easier for their customers to find the products they’re looking for. Semantic search improves search relevance, ranking, and customer experience. Moreover, it can automate or at least drastically reduce the effort and resources required to improve search relevance. 

By the end of this article, I hope to have explained what semantic search is, how it reduces resources and expands top-line growth, and where it’s headed. 

Results without effort

Whereas keyword-based search only looks up keywords (and their alternative forms), semantic search understands the meaning and often the intention of what people are searching for. 

Take for example a search for a “prom dress.” Keyword search engines look for the words “prom” and “dress” to find the best match. Behind the scenes, keyword engines have some built-in features — typo tolerance, word agglutination, stemming, and many other natural language techniques – for finding common alternatives. 

However, if your customer types in “homecoming dress” or “sequin gown”, they won’t find the prom dress. You’d have to create synonyms that state that “homecoming dress” and “prom dress” are similar ideas. 

Semantic search solves this for you. It understands that these terms are synonymous or closely related — no additional hacks required to make it work. 

Where did semantic search come from? 

Semantic search was once only the provenance of entrenched behemoths like Google, Amazon, and Microsoft (Bing). These companies invested heavily to enable customers to type in just about any query to get great results. Amazon, for example, employs nearly 2,000 engineers and data scientists to perfect on-site search. 

What took Amazon 20+ years to engineer can now be done by anyone instantly at a fraction of the cost. Today, thanks to newer machine learning models, any business can add semantic search to its website to drive better search results, higher revenue, and happier customers.

Semantic search includes multiple technologies and processes including vector embeddings. Vectors power some of the most popular services we use today, such as Netflix recommendations, Google image search, and chatbots. Vectors are mathematical models that represent words. Machine learning models produce vectors from data, and those vectors are used to measure the distance between the data. They can contain thousands of points, but to simplify it, we can look at simple three-dimensional models like the ones below. 

The training set for these machine learning models can be massive, often composed of billions of documents which help capture the relationships between words. For example, with our own vector solution (more on this below), we use a variety of AI based language models including Universal Sentence Encoder and various sentence transformers. It enables our search engine to know that “gown” and “dress” are closely-related ideas. 

That’s how vector search is able to understand the relationship between terms to return good results even without the use of matching keywords. 

Vector search is amazing, but it works even better with keyword search than by itself!

Hybrid search: keyword and semantic search 

Vector search works well for longer queries, descriptive queries, or questions. But vector search is slow and expensive to scale. Keyword search is faster than vector search and often still much better for certain kinds of queries like single-word queries or exact phrase matches. What if you were able to get the best of both without the tradeoff of speed or accuracy?

This is where hybrid search comes in. A hybrid search engine can deliver better results than either keyword search or vector search alone. The trick is in making it fast and performant, even for vector queries. At Algolia, we do that with a technology we call neural hashing. We can convert vectors into binary hashes — a smaller, more portable data set that can be run on commodity hardware rather than specialized GPUs. 

For your customers, Algolia-powered hybrid search means faster, more accurate search, discovery, and recommendations. Better search results mean a better user experience and a higher likelihood of on-site conversion and brand loyalty.

For you, it means an ability to do more with less — there’s no more need to create lists of synonyms, write if/then rules, or stuff your product pages with additional keywords and tags. New products added to your catalog are instantly searchable. You have more time to devote to other areas of your business.

There’s nothing else like it on the market: incredible search out of the box with significantly less — often zero — effort. Hybrid search is coming soon!