Search by Algolia
Feature Spotlight: Query Rules
product

Feature Spotlight: Query Rules

You’re running an ecommerce site for an electronics retailer, and you’re seeing in your analytics that users keep ...

Jaden Baptista

Technical Writer

An introduction to transformer models in neural networks and machine learning
ai

An introduction to transformer models in neural networks and machine learning

What do OpenAI and DeepMind have in common? Give up? These innovative organizations both utilize technology known as transformer models ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What’s the secret of online merchandise management? Giving store merchandisers the right tools
e-commerce

What’s the secret of online merchandise management? Giving store merchandisers the right tools

As a successful in-store boutique manager in 1994, you might have had your merchandisers adorn your street-facing storefront ...

Catherine Dee

Search and Discovery writer

New features and capabilities in Algolia InstantSearch
engineering

New features and capabilities in Algolia InstantSearch

At Algolia, our business is more than search and discovery, it’s the continuous improvement of site search. If you ...

Haroen Viaene

JavaScript Library Developer

Feature Spotlight: Analytics
product

Feature Spotlight: Analytics

Analytics brings math and data into the otherwise very subjective world of ecommerce. It helps companies quantify how well their ...

Jaden Baptista

Technical Writer

What is clustering?
ai

What is clustering?

Amid all the momentous developments in the generative AI data space, are you a data scientist struggling to make sense ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What is a vector database?
product

What is a vector database?

Fashion ideas for guest aunt informal summer wedding Funny movie to get my bored high-schoolers off their addictive gaming ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Unlock the power of image-based recommendation with Algolia’s LookingSimilar
engineering

Unlock the power of image-based recommendation with Algolia’s LookingSimilar

Imagine you're visiting an online art gallery and a specific painting catches your eye. You'd like to find ...

Raed Chammam

Senior Software Engineer

Empowering Change: Algolia's Global Giving Days Impact Report
algolia

Empowering Change: Algolia's Global Giving Days Impact Report

At Algolia, our commitment to making a positive impact extends far beyond the digital landscape. We believe in the power ...

Amy Ciba

Senior Manager, People Success

Retail personalization: Give your ecommerce customers the tailored shopping experiences they expect and deserve
e-commerce

Retail personalization: Give your ecommerce customers the tailored shopping experiences they expect and deserve

In today’s post-pandemic-yet-still-super-competitive retail landscape, gaining, keeping, and converting ecommerce customers is no easy ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Algolia x eTail | A busy few days in Boston
algolia

Algolia x eTail | A busy few days in Boston

There are few atmospheres as unique as that of a conference exhibit hall: the air always filled with an indescribable ...

Marissa Wharton

Marketing Content Manager

What are vectors and how do they apply to machine learning?
ai

What are vectors and how do they apply to machine learning?

To consider the question of what vectors are, it helps to be a mathematician, or at least someone who’s ...

Catherine Dee

Search and Discovery writer

Why imports are important in JS
engineering

Why imports are important in JS

My first foray into programming was writing Python on a Raspberry Pi to flicker some LED lights — it wasn’t ...

Jaden Baptista

Technical Writer

What is ecommerce? The complete guide
e-commerce

What is ecommerce? The complete guide

How well do you know the world of modern ecommerce?  With retail ecommerce sales having exceeded $5.7 trillion worldwide ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Data is king: The role of data capture and integrity in embracing AI
ai

Data is king: The role of data capture and integrity in embracing AI

In a world of artificial intelligence (AI), data serves as the foundation for machine learning (ML) models to identify trends ...

Alexandra Anghel

Director of AI Engineering

What are data privacy and data security? Why are they  critical for an organization?
product

What are data privacy and data security? Why are they critical for an organization?

Imagine you’re a leading healthcare provider that performs extensive data collection as part of your patient management. You’re ...

Catherine Dee

Search and Discovery writer

Achieving digital excellence: Algolia's insights from the GDS Retail Digital Summit
e-commerce

Achieving digital excellence: Algolia's insights from the GDS Retail Digital Summit

In an era where customer experience reigns supreme, achieving digital excellence is a worthy goal for retail leaders. But what ...

Marissa Wharton

Marketing Content Manager

AI at scale: Managing ML models over time & across use cases
ai

AI at scale: Managing ML models over time & across use cases

Just a few years ago it would have required considerable resources to build a new AI service from scratch. Of ...

Benoit Perrot

VP, Engineering

Looking for something?

facebookfacebooklinkedinlinkedintwittertwittermailmail

Have you had to make a big decision recently? 

Maybe you needed to choose where to move or were in the market for a big-ticket item like a mattress. Maybe you did a ton of research online and asked your friends or nearest neighbors their opinions to try to land on a definitive answer about which choice would be the absolute best, the one that would bring you 100% satisfaction: the city recommended as the #1 place to live, with no air pollution and great schools. Or the product with all five-star reviews (not just 4.5) and exactly the right features for you; the crystal-clear winner.

If you’re like millions of people weighing decisions, you got overwhelmed. You had trouble deciding on the size, color, make, model, the optional features. You felt yourself sinking into mental quicksand, with your clarity about the best choice growing muddier by the minute.

More choices, less happiness

You can end up feeling “decisioned out” to the point that you don’t even have the energy to make smaller, less consequential choices, either. 

If you’re like most people who want to make perfect choices, you’ve been afflicted by “choice paralysis,” an inability to choose anything at all because there are simply too many options with too many variations to sift through. Maybe you’ve ended up walking away from buying a new item after all, or giving up and randomly picking from seemingly popular items. You don’t have all day.

The phenomenon of “too much choice” sounds like a joke, but it’s a problem that has been widely documented. Hick’s Law states that the time required to reach a decision increases logarithmically with the number of choices available. And authors have written whole books about it, including The Paradox of Choice: Why More Is Less, by Barry Schwartz. “The fact that some choice is good doesn’t necessarily mean that more choice is better,” he says.

Bleak and potentially depressing as it can be for people to have too many choices, the good news is that using data science, recommender systems can simplify your process of having to choose and make the selection process easier. 

What’s a recommendation engine?

What exactly is a “recommender system”? It’s a data-filtering tool that uses machine-learning algorithms to determine consumer behavior patterns, predict user ratings or preferences they’d have, and recommend relevant items to them. In short, it’s software that filters information to help users choose, and it provides them with the most relevant user-based suggestions.

A recommender system expertly narrows the number of items a particular user could consider so that they can continue calmly browsing and eventually make the right choice, confidently select an item, and feel good about the user experience to the point that they feel like they could — or even want to — do it again.

Recommender systems are all the rage. Most major tech companies that have an online presence are taking advantage of the ability to make “intelligent” recommendations based on user profiles, which, in addition to improving customer satisfaction, helps companies optimize by boosting their user engagement and retention rates. 

For example:

  • YouTube suggests videos to watch, as well as which ones to automatically start playing next, based on the person’s user history
  • Facebook suggests adding new Facebook friends that users may know in the real world (offline)
  • LinkedIn suggests jobs that people may be interested in applying for based on various factors

Classic online retailers are also big fans of recommender systems. The most notable example and pioneering leader in product recommendations driven by deep learning is Amazon.com: In 2021, the site accounted for half of all U.S. ecommerce sales (Statista).

Big tech is the primary arena for personalized recommendations, but this technology is also being used for more-specific and focused applications, such as restaurant choices, online dating, social media, and financial services.

How recommendation engines works

What goes on in the inner workings of a recommender system? A recommendation engine does its work in four phases:

1. Data collection

 Data is gathered from people in two ways: implicitly, without their knowledge; and explicitly.

Implicit data gathering consists of the software essentially following the user around online and noting what they do. When they go to the same page three times in a row or put an item in their cart, the data collection functionality takes note and, judging by what’s collected, starts to infer the user’s typical behavior.

Explicit data gathering takes place when a user is asked to or voluntarily offers information, such as when they write a review, rate a product, like a social media post or leave a comment, or follow an author or musician. This type of data collection also infers user preferences.  

One drawback of explicit data gathering is that since people don’t always take the time to rate or review products they buy or provide information in other ways, the available data may be incomplete.

2. Storage

All of this interesting user data is filed away (for instance, in a NoSQL or standard SQL database) for future reference.

3. Analysis

Through filtering (such as batch, real time, near real time), the system looks at data relationships and identifies items that show similar user engagement data.

4. Filtering

Finally, the data is filtered to collect relevant information so that intelligent, appropriate recommendations can be made to users. Different algorithms can be used when recommending products. Content-based and collaborative algorithms recommend similar content, whereas a cluster algorithm makes recommendations that could also be based on what other users are doing.

The importance of relationships in a recommendation system

The relationships between elements in the collected data are the “glue” that gives recommender systems an understanding of customers’ preferences and helps them know what people want. 

Three types of relationship between users and items are looked at in data analysis:

  • The user-item relationship: Users have preferences for certain items and types of products
  • The item-item relationship: People may like items that are similar looking or described in a similar way, such as books in the same genre or foods in the same type of cuisine
  • The user-user relationship: people with similar backgrounds (for example, in the same age group) may have similar taste

In addition to focusing on these relationships, recommender systems look at data about:

  • User behavior: details about people’s online activity, collected when they do things like go to particular web pages, give product ratings, click on items, engage with content (e.g., by watching a movie no Netflix), and buy things
  • User demographics: This is what it sounds like — information about age, ethnicity, gender, education, career, family structure, income, and religious beliefs
  • Product attributes: product-related information such as item genre or type of cuisine

As long as we’re making lists of three…there are also three different types of recommender system: those that do collaborative filtering, those that do content-based filtering, and those that do both types.

Collaborative filtering

A collaborative filtering recommender engine focuses on collecting and analyzing data on user behavior, activities, and preferences in order to predict what a user likes based on their similarity to other users. It does not analyze or understand the content itself (the product, book, or video).

This type of filtering:

  • Uses a matrix-style formula to suggest items based on what it knows about the user
  • Applies the logic that if a user likes item A and another someone likes it too, as well as item B, then the first user might also like item B
  • Predicts new interactions based on earlier ones 

There are two ways of filtering collaboratively: memory based and model based.

Memory-based collaborative filtering methods

This type of collaborative filtering:

  • Identifies clusters of users; utilizes user interactions to predict those of similar users
  • Identifies clusters of items rated by user A, then uses them to predict interaction of user A with similar item B
  • Can be challenging with matrix sparsity because the number of user-item interactions may be too low for generating high-quality clusters

Model-based collaborative filtering methods

This type of collaborative filtering:

  • Relies on machine learning and data-mining techniques
  • Strives to train models to make predictions; for example, it might use existing interactions a user has had with an item to help a model predict the person’s top-5 liked items 
  • Has an advantage over memory-based collaborative filtering in that it can recommend larger numbers of items to larger numbers of users

One drawback of collaborative filtering is that this type of processing takes time and has to start somewhere. A new user on the site or the introduction of a new product is a cold-start problem. You need enough user-item interactions for the system to work; it can’t give product recommendations until the person has interacted with enough items. Companies may be able to decrease the lag time by directly asking people for data (such as their age or interests) when they sign up on a site, and pairing that collected information with item metadata that’s already available to relate the new user to the existing items.

Content-based filtering

With filtering according to content, similar items are grouped together based on their features. Recommendation algorithms take into account the customer’s preferences plus descriptions of items (e.g., their genre, product type, color, or word length). You know a system is filtering based on content when the interface tells you “If you like that, you might also like this.” Or when it shows you an item that it remembers you were interested in the past, thinking you could still be interested in it.

What are the drawbacks of a recommendation engine using content-based filtering? 

  • You have limited insight; you can only recommend what’s similar to what a person is viewing, buying, or using
  • If user interaction shows that someone is interested only in certain categories, the system has no way to reliably recommend items in other categories
  • There’s not enough data available about new users when they first get started

Hybrid approaches

A recommendation engine that uses hybrid filtering utilizes both collaborative and content-based data. Not surprisingly, its superior filtering produces the best information.

Netflix is a good example of a company using hybrid recommender filtering on its website. It accounts for users’ interests (collaborative filtering) and also for movie descriptions and features (content-based filtering).

Decisions, decisions

Recommender systems are a sanity saver that can help people break through choice paralysis when making decisions, confidently choose what’s best, and leave them with enough energy and brain cells to focus on making those bigger, tougher decisions.

Will we get to the point where we delegate even our “big” decisions to online expertise? You might ask yourself a big question, such as “Where shall we go on vacation?” or “Who’s the best plumber to fix my overflowing sink?” only to end up checking the web for low plane fares or looking on Yelp for plumber reviews. 

So for all intents and purposes, that time is already here…

Want a great customer experience provided by the right recommendation engine? Contact us today.

For more information

The anatomy of high-performance recommender systems (part 1)

About the author
Catherine Dee

Search and Discovery writer

linkedin

Recommended Articles

Powered byAlgolia Algolia Recommend

The anatomy of high-performance recommender systems - Part 1
ai

Ciprian Borodescu

AI Product Manager | On a mission to help people succeed through the use of AI

What are personalized recommendations and how can they boost engagement and conversion?
ux

Catherine Dee

Search and Discovery writer

What is a product recommender (or product recommendation engine)?
product

Catherine Dee

Search and Discovery writer