I search through your content to help you find answers to your questions, fast.
Listen to this blog as a podcast:
What if I told you an AI configured search better than I did as a new hire at Algolia?
We built an experimental CLI tool that uses AI to analyze your Algolia data and generate relevance configurations. It produces near-expert quality suggestions in ~10 seconds, handling searchable attributes, custom ranking, faceting, and sorting options.
At Algolia, we've solved the hard parts of search. Upload your catalog, and you can search it immediately, with lightning fast results. Typos? Handled. Infrastructure? Scales automatically. Search-as-you-type? It just works. We've spent years perfecting the complex algorithms and infrastructure required for state-of-the-art search, so you don't have to even think about it.
But here's the thing: even with all that complexity handled, creating great search still requires decisions. Which attributes should be searchable? How should popularity influence ranking? What sorting options do users need?
These aren't technical limitations—they're the difference between generic search and search that understands your business. Amazon knows to prioritize Prime items. Netflix and YouTube factor in your viewing history. Your search needs to understand what matters to your users.
The challenge is that setting these configurations right can stretch to days. You need to understand concepts, read documentation, experiment, and iterate—all before you can build the UI to show your end users actual value. And, let’s face it, relevance configuration isn’t fun. It can get tedious, error-prone, and it’s easy to rush or overlook.
Common mistakes we often seen:
Here’s one thing you realize after spending years at Algolia and looking at many different indices and relevance configurations: patterns repeat. Most e-commerce sites need similar sorting options. Most media platforms rank content in predictable ways. The best practices we document aren't random—they're the accumulated wisdom of thousands of implementations.
This realization led to a question: if these patterns are so consistent, could AI learn them? Could we encode our expertise into something that analyzes your data and suggests the right configuration automatically?
We built a CLI tool that uses LLMs to analyze your data and generate Algolia configurations. The approach is straightforward:
⚠️ Privacy note: This tool sends data to OpenAI/Anthropic. Only use non-sensitive datasets.
Here are the recommendations we got when analyzing a standard e-commerce dataset, using Anthropic’s Claude Haiku 3.5 model:
.webp)
.jpg)
namebranddescriptioncategories.lvl0categories.lvl1categories.lvl2colormaterialThe agent's reasoning was spot-on. It identified the right attributes that a user would search for (with name as the highest priority given it is the primary identifier for products, brand as highly important for product discovery, etc.)
It also explicitly excluded numeric attributes like price, rating, inventory, technical attributes like objectID, product_url, image_url, or boolean attributes like in_stock, which users don’t search for with a query.
desc(rating_bayesian)desc(rating_count)desc(inventory)Notice how it chose the Bayesian rating over the raw rating. The shared rationale was exactly what it was taught:
“This is a processed metric that provides a normalized, statistically adjusted rating
Represents a more sophisticated quality signal compared to raw rating
Helps surface high-quality products with more reliable scoring
Values range from 4.4 to 4.7 in the sample, indicating meaningful differentiation”
This is exactly what an expert would recommend—avoiding the common mistake of using raw ratings that can be gamed with a few 5-star reviews.
categoriessearchable(brand)colormaterialin_stockratingpriceFacets can be made searchable when they have many different possible values, which can’t all be displayed at once in the UI. The agent correctly identified that brand should be searchable (high cardinality), while color and material don't need to be.
desc(price)asc(price)desc(rating_bayesian)desc(rating_count)desc(inventory)Another example of nuanced reasoning, the agent decided to provide two options for price (High to Low and Low to High, which are common sorting options in e-commerce search) but only descending options for rating_bayesian (it’s useful to see most popular products first, but not the other way around), rating_count (indicating user engagement) and inventory (useful for checking product abundance).
Getting good results required careful prompt engineering. Let’s take a look at our searchable attributes prompt:
const prompt = `
Analyze these sample records and determine which attributes should be searchable in an Algolia search index.
Sample records:
${JSON.stringify(sampleRecords, null, 2)}
Step 1: Identify potential searchable attributes from the sample records
Step 2: Order attributes by search importance and user intent
Step 3: Determine modifier configuration (ordered vs unordered)
Step 4: Format final result with appropriate modifiers
CRITICAL RULES:
- Only suggest attributes that actually exist in the provided sample records, don't invent ones
- Only suggest attributes truly suitable for search. If no attributes are clearly searchable, return an empty array.
Rules for selecting searchable attributes:
INCLUDE text attributes that users search for:
- Names, titles, descriptions, summaries
- Brands, manufacturers, creators, authors
- Categories, types, genres
- Features, ingredients, cast, tags
- Locations, addresses
- Any text users might query
EXCLUDE attributes that are:
- URLs, IDs, dates, timestamps, booleans
- Numeric values for ranking/sorting
- Display-only or internal metadata
Rules for ordering attributes by search importance:
Order matters - first attributes have higher search relevance.
1. Primary identifiers (name, title) rank highest
2. Secondary identifiers (brand, creator) come next
3. Content attributes (description, features) follow
4. Consider user search patterns for this data type
Rules for equal ranking attributes:
To make matches in multiple attributes rank equally, combine them in comma-separated strings:
- "title,alternate_title" - treats both title fields equally
- "name,display_name" - treats both name fields equally
- "brand,manufacturer" - treats both brand fields equally
Rules for modifier configuration:
- Use "unordered(attribute)" for most cases (position doesn't matter)
- Use "ordered(attribute)" only when early words are more important
- For array attributes: ordered may make sense when early entries are more important (cast of actors) but not for equal importance (tags)
- Default to unordered unless position specifically matters
- Note: ordered modifier cannot be used with comma-separated attributes
Explain your answer step-by-step.
`;
Some key insights from crafting and refining this prompt:
Even with careful prompting, LLMs sometimes suggest non-existent attributes.
We add a validation phase to strip out any attribute that isn’t present in the dataset.
const { object, usage } = await generateObject({
model,
maxTokens: 1000,
temperature: 0.1,
schema,
prompt,
});
// Validate that all suggested attributes actually exist in the records
const searchableAttributes = validateAttributes(
object.searchableAttributes,
sampleRecords,
);
The CLI will still tell you what attributes have been filtered out, which is useful to recognize patterns in hallucinations: poor quality datasets, scarce number of attributes, or insufficient model power.
The context window of an LLM is the amount of content (translated in tokens) that a given model can consider at once. The longer the prompt, the closer you get to the limit.
In the experiment, we always pass full records, not just attribute names. This matters because attribute names alone don't always provide enough context. But this creates challenges with prompt size:
| Number of records | Processing time | Status |
|---|---|---|
| 10 | ~10 seconds | ✅ Optimal |
| 100 | ~15 seconds | ✅ Works well |
| 500 | ~45 seconds | ⚠️ Slower |
| 1000 | - | ❌ Context window exceeded |
While this could be handled in batches, the insight here is that you actually don't need all records, just a representative sample. Even 10 records consistently produced quality suggestions, while controlling token consumption.
Although we initially built the CLI with Claude 3.5 Haiku as the default model, it was essential to compare how different models perform when given the same task.
I tried:
| Model | Cost | Quality | Notes |
|---|---|---|---|
| Claude 3.5 Haiku | 10 records: $0.0174 | Excellent | Best value, handles nuance well |
| 100 records: $0.0863 | Excellent | ||
| Claude 3.5 Sonnet | 10 records: $0.0636 | Excellent+ | Slightly better on attribute ordering |
| 100 records: $0.2390 | Excellent+ | ||
| GPT-4.1 nano | 10 records: $0.0012 | Good | Very cheap, misses nuances, more hallucinations |
| 100 records: $0.0082 | Good |
Claude Haiku emerged as the sweet spot—nearly as good as Sonnet, at 27% of the cost.
There were no noticeable quality differences between 10 and 100 records.
AI loves to be helpful, but this sometimes comes in the way of accuracy. For example, in one test, GPT-4.1 nano suggested faceting on category when the dataset didn’t contain such attribute.
I noticed that such hallucinations happened more frequently with speed-optimized models, and worsened with poor quality data.
The validation step is here to catch such mistakes, but it shows that AI can’t be blindly trusted: it needs edge case handling and refinement over time.
Inconsistent schemas and ambiguity can lead to weird suggestions:
When we tested with datasets having inconsistent schemas, hallucinations increased dramatically. This proves the need for health checks before analysis—AI can't fix fundamental data issues.
Some patterns are better handled with code or human intervention:
Another interesting use case was when the agent had to deal with multiple, similar attributes for custom ranking. In the products dataset, we added a rating_bayesian attribute to nuance the rating attribute, which was an average of all ratings, using the rating_count. This is what we recommend in the Algolia documentation, because the ranking algorithm uses tie-breaking: it compares records on each criterion, in their specified order, but only moves on to the next if it can’t break the tie.
But the catch is, we left rating and rating_count in the records to stick to what we usually see in customer datasets—it’s not uncommon to leave more data than you need. And depending on the model and the precision of the prompt, this is where suggestions quality would drastically differ.
When you have repetitive data, you must be extremely explicit in prompting to avoid the agent including both—and even then, it’s tough to guarantee it will comply. This further indicates that keeping only search-relevant data in your records becomes critical when processing them with a systematic tool like an AI assistant.
To test effectiveness, I decided to run the tool—this time using Claude Sonnet for optimal accuracy—on an index I created when I was a new hire at Algolia. This was 2018, and as the FIFA World Cup was about to start, I indexed every game with information on what teams were playing, on what day, in what stadium, what TV channels were broadcasting it, what phase of the tournament it was, and I updated scores live as the games would end.
This was my first “real” production Algolia implementation, and I configured it like I expect any customer would: following the documentation, testing settings out, until things “looked good”.
The AI actually did better than me.
My configuration was riddled with rookie mistakes:
Here’s how AI improved it:
Everything that was right about my manual configuration was also picked up by the agent, only better, by fixing my beginner mistakes.
Note that many improvements could be further made at the dataset level (e.g., providing a list of all teams instead of home_team and away_team for better faceting, storing scores as numbers, etc.), which would result in even better relevance settings. Here again, AI is doing its best with the data it has.
The code is available on GitHub for you to try: https://github.com/algolia/generative-relevance
# Install and setup npm install echo "ANTHROPIC_API_KEY=your_key" > .env # Analyze your data npm start -- analyze your-data.json --verbose # Compare with existing index npm start -- compare YOUR_APP_ID YOUR_API_KEY your_index --verbose # Try different models npm start -- analyze data.json --compare-models claude-3-5-haiku-latest,gpt-4.1-nano
More importantly, we want your feedback! We're especially interested in:
See anything we could improve? Open an issue and let us know.
While AI handles systematic patterns well, human judgment is still crucial for:
AI is not magic—it does what you ask. Use this as a foundation that takes you close to the finish line, so you can focus your efforts on what only you can do.
This experiment taught me that AI excels at systematic knowledge. When best practices are consistent and well-documented, AI can apply them effectively. It won't capture every nuance of your specific use case, but it provides a solid foundation to refine rather than starting from scratch.
The future isn't AI replacing search experts, but helping every Algolia user configure search like an expert. By commodifying the routine parts of configuration, we can free up developer time to focus on what makes their search unique: understanding their users and crafting exceptional experiences.
I presented on this topic at Algolia DevBit 2025, which you can watch below.
Powered by Algolia AI Recommendations
%20(2).svg)
