Algolia’s Image Recommendation API opens the door to a host of innovative possibilities, providing users with an efficient image vector retrieval system where they can carefully blend images with textual and business signals like price and availability. An expressive tool, this technology makes it easy for companies to deliver a rich visual search experience that is particular to each business case and individually tailored to each online user.

We’ve all used text-based search looking for information on the internet or a product on an ecommerce site. Many of us even use audio-based song recognition and recommendation software to help us identify music that we like. But image retrieval and recommendation technology has been far more difficult to implement and operationalize.

Algolia’s Image Recommendation API changes that.

Until now, digital devices have never managed images particularly well. The key to our AI-based solution is its ability to understand, see, and sort through images quickly and accurately. For businesses, this translates into context sensitive image retrieval that works in harmony and in parallel with features like text or keyword search.

The Image Recommendation API delivers a new multimodal experience to online search and discovery. Our innovative solution relies on an array of new technologies to lead image retrieval at a speed, scale, and relevance that is applicable to a wide range of different business use cases.

How does the system work?

Product images are critical for ecommerce and online retailers. Customers need to see what they are buying, and online merchandisers need to be able to convey color, texture, style, place, and mood. Merchandisers also need those elements to work together to generate the selling propositions and meaningful connections that convert online users into paying customers.

To accomplish multimodal image retrieval and search at scale, Algolia engineers use a series of three core technologies to power our Image Recommendation API. The new system uses image vectorization, vector hashing and vector retrieval, as well as our powerful natural language processing (NLP) capabilities, to generate instant, accurate responses and seamless interactions.

Image vectorization

Image vectorization is a powerful core technique that converts an image’s pixel value into numerical values called vectors. The process of vectorizing an image involves running it through an advanced deep learning network called a convolutional neural network (CNN). The CNN analyzes and extracts the image’s relevant features.

The deeper the model, the more features get extracted. Ironically, experiments determined that generating too many layers ended up degrading the model and introducing new errors. As a result, our engineers decided to employ a deep learning model that uses a CNN called ResNet 18 to convert images into fixed length vectors – an approach that doesn’t add significant errors or training time to the data analysis process. 

Once plotted into multi-dimensional graphical space, the next issue to consider is if you have millions of vectors, how do you scale this process? And how do you ensure that classification, matching, comparison, and retrieval are efficient and timely?

Vector hashing and retrieval

Algolia engineers solve this issue by using a concept called vector hashing or semantic hashing. This method groups similar items together so that retrieving them is easier when dealing with very large data sets.

Employing a variant on vector hashing called Locality Sensitivity Hashing (LSH) also allows deep training models to learn to distinguish between features that are similar or different from one another. Once the model understands which vectors offer a good representation of the different partitions in the data, they become faster and easier to retrieve.

What sets Algolia’s API apart is its ability to work extremely well with a very large dataset, without diminishing accuracy and speed. The image vectors are retrieved and filtered using a Hierarchical, Navigable, Small World (HNSW) graph system. They are then linked to relevant metadata such as image, price and description or other assigned rules, before being delivered instantly to the end user in the form of a lightning-fast search result.

Context sensitive image retrieval and usability

The combination of image vectorization, binary hashing, and vector retrieval using HNSW enables fast and relevant recommendations. In turn, these recommendations help drive a powerful, versatile, and context-sensitive image retrieval API that can be adapted to a wide range of different use cases:

• Suggest similar looking products to customers when the item they’re looking for is unavailable
• Encourage users to explore your catalog by displaying similar looking items
• Promote specific product images associated with certain locations or topics
• Offer brand alternatives that resemble initial choice with lower cost option, boosting margins

In addition to being able to index close to 50K images in less than 60 minutes and up to half-a-million in a few hours, Algolia’s Image Recommendation API enables users to have the ability to control a range of image and search result parameters.

Some of the unique features include:

• Critical fallback parameters to avoid the risk of delivering users any null results
• Providing a flexible rule system that allows merchandisers and marketers to boost and/or bury images for promotional purposes
• Easily adding rules for non-technical users and letting them filter results to implement merchandising strategies and improve accuracy, delivering an enhanced customer experience

Picturing a range of possibilities

For businesses to generate the maximum value from the Image Recommendation API it’s important to consider both the dynamic relationship between the elements on the search results page and ways to blend image signals alongside key textual and business elements.

Striking the right combination is different for each use case, each customer, and can change from one query to another, opening up a new area of creative business and merchandising possibilities.

Algolia’s Image Recommendation API gives businesses another tool in their palette of creative options. The technology has been designed to enable users to leverage contextual-image search in the broadest way possible, allowing you to apply objective level properties, to filter, re-rank and post-process results as necessary.

Designed to be an expressive tool, Algolia’s Image Recommendation API offers merchandisers a powerful enabling technology. More than just an efficient image vector retrieval system that delivers fast, relevant search results, it also shapes the customer journey, delivering a unique, personalized search experience that helps drive conversions and delight end users.

Some next steps:

• Learn more about Algolia’s Image Recommendation API with the eBook Image Recommendation API for Business-Contextual Search
• Or click here to learn why we believe this is a key feature of modern web experiences
• Or click here to learn about inspiring use-cases

Cheers!