Search by Algolia
How personalization boosts customer engagement
e-commerce

How personalization boosts customer engagement

You land on your favorite retailer’s website, where everything seems to be attractively arranged just for you. Your favorite ...

Jon Silvers

Director, Digital Marketing

What is retail analytics and how can it inform your data-driven ecommerce merchandising strategy?
e-commerce

What is retail analytics and how can it inform your data-driven ecommerce merchandising strategy?

There is such tremendous activity both on and off of retailer websites today that it would be impossible to make ...

Catherine Dee

Search and Discovery writer

8 ways to use merchandising data to boost your online store ROI
e-commerce

8 ways to use merchandising data to boost your online store ROI

New year, new goals. Sounds positive, but looking at your sales data, your revenue and profit aren’t so hot ...

John Stewart

VP, Corporate Communications and Brand

Algolia DocSearch + Astro Starlight
engineering

Algolia DocSearch + Astro Starlight

What is Astro Starlight? If you're building a documentation site, your content needs to be easy to write and ...

Jaden Baptista

Technical Writer

What role does AI play in recommendation systems and engines?
ai

What role does AI play in recommendation systems and engines?

You put that in your cart. How about this cool thing to go with it? You liked that? Here are ...

Catherine Dee

Search and Discovery writer

How AI can help improve your user experience
ux

How AI can help improve your user experience

They say you get one chance to make a great first impression. With visual design on ecommerce web pages, this ...

Jon Silvers

Director, Digital Marketing

Keeping your Algolia search index up to date
product

Keeping your Algolia search index up to date

When creating your initial Algolia index, you may seed the index with an initial set of data. This is convenient ...

Jaden Baptista

Technical Writer

Merchandising in the AI era
e-commerce

Merchandising in the AI era

For merchandisers, every website visit is an opportunity to promote products to potential buyers. In the era of AI, incorporating ...

Tariq Khan

Director of Content Marketing

Debunking the most common AI myths
ai

Debunking the most common AI myths

ARTIFICIAL INTELLIGENCE CAN’T BE TRUSTED, shouts the headline on your social media newsfeed. Is that really true, or is ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

How AI can benefit the retail industry
ai

How AI can benefit the retail industry

Artificial intelligence is on a roll. It’s strengthening healthcare diagnostics, taking on office grunt work, helping banks combat fraud ...

Catherine Dee

Search and Discovery writer

How ecommerce AI is reshaping business
e-commerce

How ecommerce AI is reshaping business

Like other modern phenomena such as social media, artificial intelligence has landed on the ecommerce industry scene with a giant ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

AI-driven smart merchandising: what it is and why your ecommerce store needs it
ai

AI-driven smart merchandising: what it is and why your ecommerce store needs it

Do you dream of having your own personal online shopper? Someone familiar and fun who pops up every time you ...

Catherine Dee

Search and Discovery writer

NRF 2024: A cocktail of inspiration and innovation
e-commerce

NRF 2024: A cocktail of inspiration and innovation

Retail’s big show, NRF 2024, once again brought together a wide spectrum of practitioners focused on innovation and transformation ...

Reshma Iyer

Director of Product Marketing, Ecommerce

How AI-powered personalization is transforming the user and customer experience
ai

How AI-powered personalization is transforming the user and customer experience

In a world of so many overwhelming choices for consumers, how can you best engage with the shoppers who visit ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

Unveiling the future: Algolia’s AI revolution at NRF Retail Big Show
algolia

Unveiling the future: Algolia’s AI revolution at NRF Retail Big Show

Get ready for an exhilarating journey into the future of retail as Algolia takes center stage at the NRF Retail ...

John Stewart

VP Corporate Marketing

How to master personalization with AI
ai

How to master personalization with AI

Picture ecommerce in its early days: businesses were just beginning to discover the power of personalized marketing. They’d divide ...

Ciprian Borodescu

AI Product Manager | On a mission to help people succeed through the use of AI

5 best practices for nailing the ecommerce virtual assistant user experience
ai

5 best practices for nailing the ecommerce virtual assistant user experience

“Hello there, how can I help you today?”, asks the virtual shopping assistant in the lower right-hand corner ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

Add InstantSearch and Autocomplete to your search experience in just 5 minutes
product

Add InstantSearch and Autocomplete to your search experience in just 5 minutes

A good starting point for building a comprehensive search experience is a straightforward app template. When crafting your application’s ...

Imogen Lovera

Senior Product Manager

Looking for something?

facebookfacebooklinkedinlinkedintwittertwittermailmail

It can be confusing to hear search companies explain how search and AI work. Bing has added ChatGPT which uses large language models (LLMs), but even before that they had deep learning capabilities. Google recently announced new image search capabilities and its own LLM service. At Algolia, we’re also about to introduce our own AI-powered technology that uses neural hashing to scale intelligent search for any application. All of these terms can be confusing. 

Let’s fix that by breaking down the technologies involved with search.

Let’s start from the start… what is keyword search?

Keyword search engines have been around for decades. The Apache Lucene project is one of the best well-known open source search engines which offers keyword search functionality. This type of search engine uses statistical techniques to match queries to items in the index. They work much like the index at the back of a book by pointing to all the places in the book where information is located. Query processing technologies like typo tolerance, word segmentation, and stemming are also used to help search engines digest and make sense of spelling and query understanding. 

understand ai search banner

Keyword search tends to be very fast, and works well for exact query-keyword matches. However, they often struggle with long tail queries, concept searches, question-style searches, synonyms, and other phrases where the query doesn’t exactly match the content in the index. For this reason, many companies have added additional features such as AI synonym generation to help. 

What is semantic search?

Semantic search involves understanding the meaning of words and phrases in a search query and returning results that are semantically related to the query. Semantic search engines use natural language processing (NLP) techniques to understand the meaning of words and phrases and to find related concepts, synonyms, and other related information that may be relevant to the search query.

How is that different from AI search? Are semantic search and AI search the same thing?

AI search is a general and broader term that includes semantic search as well as other machine learning techniques for delivering search results. AI search typically involves several steps, including query processing, retrieval, and ranking. 

  • Query processing: This step involves analyzing the user’s query to understand its intent, scope, and constraints. Query processing may include tasks such as parsing the query into its constituent parts, semantic understanding of keywords and phrases, normalizing the query to a standard format, and more.
  • Retrieval: Once the query has been processed, the system retrieves a set of documents or data items that match the query criteria. AI search typically uses machine learning algorithms to determine similarity and measure relatedness between terms to deliver relevant results.
  • Ranking: After the documents or data items have been retrieved, the system ranks them based on their relevance and importance to the user’s query. Learning-to-rank models such as reinforcement learning are used to continuously optimize results.

Is search engine AI different from generative AI?

OpenAI’s ChatGPT, Google’s Bard, Midjourney and other similar AI technology are what’s called generative AI. These general purpose solutions attempt to predict the results based on input, and will actually generate a fresh response. They use pre-existing text and visual content to generate something new. 

On the other hand, search engines can use AI to improve search results. Just like generative AI, search AI can be used to understand natural language inputs. Unlike generative AI, search engines are not creating any new, novel content. Both technologies can be used together or independently. Generative AI technologies can be used to aid with creative output, and search is used to filter and rank order results. Someone looking for new fashion ideas might ask a chat bot what the latest trends are, get results, and then use search to find results. Or, you might use search to find products and then ask chat to explain pros and cons of each result. 

Both generative chat AI and search AI often provide a better user experience through the understanding of natural language. 

Does AI search also use LLMs?

Large language models (LLMs) have been around for a while now, but GPT has put them in the spotlight. LLMs are artificial intelligence models that are trained to process and generate natural language text. These models are typically built using deep learning techniques and require vast amounts of data and computational resources for training. At Algolia, we use LLMs, too, but to aid in machine understanding. We use LLMs to create vectors which we can use to compare queries to results. 

What is vector search?

Vectorization is the process of converting words into vectors (numbers) which allows their meaning to be encoded and processed mathematically. You can think of vectors as groups of numbers that represent something. In practice, vectors are used for automating synonyms, clustering documents‍, detecting specific meanings and intents in queries, and ranking results. Embeddings are very versatile and other objects — like entire documents, images, video, audio, and more — can be embedded too.  

Vector search is a way to use word embeddings (or image, videos, documents, etc.,) to find related objects that have similar characteristics using machine learning models that detect semantic relationships between objects in an index. 

semantic vectors
Image via Medium showing vector space dimensions. Similarity is often measured using Euclidean distance or cosine similarity.

There are many different approximate nearest neighbor (ANN) algorithms for calculating vector similarity. Techniques such as HNSW (Hierarchical Navigable Small World), IVF (Inverted File), or PQ (Product Quantization, a technique to reduce the number of dimensions of a vector) are some of the most popular ANN methods to find similarity between vectors. Each technique focuses on improving a particular performance property, such as memory reduction with PQ or fast but accurate search times with HNSW and IVF. It is common practice to mix several components to produce a ‘composite’ index to achieve optimal performance for a given use case.

One of the challenges for working with vectors is their size. They tend to be very large strings that require specialized databases and GPU management. Neural hashing is a new process that uses neural networks to compress vectors so they can be processed up to 500 times faster than standard vector calculations and run on commodity hardware. 

Keywords + vectors… what is hybrid search?

Hybrid search is the combination of vector search with keyword search. Vector search is terrific for fuzzy or broad searches, but keyword search still rules the roost for precise queries. For example, when you query for “Adidas” on a keyword engine, by default you will only see the Adidas brand. The default behavior in a vector engine is to return similar results — Nike, Puma, Adidas, etc., because they are all in the same conceptual space. Keyword search still provides better results for short queries with specific intention. 

Hybrid search offers the best of both worlds providing speed and accuracy for exact matches and simple phrases, while vectors improve long tail queries and open the door to new search solutions.  Learn more about our very own hybrid AI solution — Algolia NeuralSearch — right here.

About the author
Jon Silvers

Director, Digital Marketing

linkedin

Recommended Articles

Powered byAlgolia Algolia Recommend

What is end-to-end AI search?
ai

Abhijit Mehta

Director of Product Management

How neural hashing can unleash the full potential of AI retrieval
ai

Bharat Guruprakash

Chief Product Officer

What is vector search?
ai

Dustin Coates

Product and GTM Manager