Troubleshoot Advanced Personalization

Advanced Personalization can help deliver personalized results to your users. If you don’t fix issues, users grow less happy and your results can suffer. This guide covers issues you may face with Advanced Personalization, their symptoms, and steps to fix them. Use it to keep Advanced Personalization accurate and relevant for your users.

​

Troubleshoot events

​

Non-persistent user tokens

A user token identifies users consistently across sessions. If the token isn’t persistent, it can reset or change often. Each session or event then looks like it comes from a different user.

​

Symptoms

• A low returning user rate warning appears in the Advanced Personalization dashboard.
• The dashboard might show user profiles, but they often vary a lot.

​

Recommendation

To track user activity, use persistent user tokens that remain consistent across sessions and events. This builds accurate user profiles over time.

​

Incorrect object IDs

All must send object IDs that match object IDs in your . If they don’t match, Advanced Personalization can’t link user behavior to the right data. This causes problems linking events to indices.

​

Symptoms

• The User Timeline displays events connected to “deleted records.” This suggests mismatched object IDs.
• Errors such as mismatched object IDs and no object IDs occur when you save the settings.

​

Recommendation

Ensure all events contain valid object IDs that match the in your index. This match lets Advanced Personalization build an accurate, complete .

​

Batch event processing

Processing events in batches isn’t an issue on its own. But using the same or adjusted timestamps can cause problems. It can distort session duration and produce inaccurate user profiles.

​

Symptoms

quality is poor.

​

Recommendation

Avoid manually changing timestamps to get around API checks. Make sure each event in a session has a unique timestamp that shows the actual time of the user action.

​

Incorrect event mapping

Sometimes, user events aren’t assigned to the correct event collection. This mapping error stops Advanced Personalization from reading user behavior. As a result, it can’t generate accurate user profiles.

​

Symptoms

• User profiles are poor quality because Advanced Personalization can’t read user behavior well.
• Your metric, or other key metrics, might look wrong if some events are tracked more than others, or not tracked at all.

​

Recommendation

Review the event mapping. Make sure user events match the recommended event collections. This ensures Advanced Personalization captures and analyzes every user action.

​

Obsolete events

Say you add subtypes like addToCart and purchase to your events. If the settings still point to the old events, Advanced Personalization keeps listening for events that no longer arrive. The dashboard flags these outdated events with a warning. User profiles miss the signals from the renamed events until you update the settings.

​

Symptoms

In the events configuration of the Advanced Personalization dashboard, a warning icon with the message “Event not received in the past 24 hours” appears next to each outdated event. This means the event is still configured, but Algolia hasn’t detected it for the index or its replicas in the last 24 hours.

​

Recommendation

Update the event mapping. Replace the outdated events with the new ones that include subtypes, then save the settings to apply the changes.

​

Incomplete event collection

Issues can arise if you only collect events from search activity and ignore other actions, like clicks on the home page. This leaves event collection incomplete, so user profiles rely on partial data and drop in quality.

​

Symptoms

User profiles are poor quality because Advanced Personalization builds them only from search activity and ignores other actions.

​

Recommendation

Make sure to capture and map all user actions, not just those related to search. Gathering complete event data ensures more accurate and effective personalization.

​

Troubleshoot indices

​

Excessive affinities

Problems can arise when user profiles have many values, such as several product categories or brands linked to one user. This excess data can make personalization less accurate and less effective.

​

Symptoms

• Search results are less relevant because Advanced Personalization struggles to rank the most important affinities.
• The sheer number of less relevant affinities can bury the highly relevant ones.
• A/B test performance drops, and personalized results don’t show a clear improvement over non-personalized results.

​

Recommendation

Limit the number of affinity values in user profiles to include only the most relevant data. This ensures more focused and accurate search results.

​

Using SKU attributes as affinities

Stock keeping unit (SKU) attributes hold details for each product. Using them as affinities causes issues, because affinities should be categorical, like product categories or brands. SKU attributes are unique, so Advanced Personalization ignores them. This makes them useless as attributes for affinities.

​

Symptoms

The user profiles Advanced Personalization generates don’t include the SKU-based affinities.

​

Recommendation

Use categorical attributes for affinities instead of specific SKUs.

​

Attributes with filter-only modifier

Using attributes with the filterOnly modifier as affinities causes an issue. Advanced Personalization relies on faceting to handle affinities, but attributes with this modifier don’t support faceting. This means it can’t use these attributes.

​

Symptoms

You can’t select attributes with the filterOnly modifier as attributes for affinities when you personalize an index.

​

Recommendation

Check whether an attribute is filterOnly with the dashboard. Don’t use filterOnly attributes as attributes for affinities.

​

Troubleshoot search results

​

Missing user tokens

Advanced Personalization needs a userToken to give personalized results. Without it, you can’t tailor the search experience to each user.

​

Symptoms

• Search results aren’t personalized, even though user profiles are visible in the dashboard.
• The dashboard shows errors or warnings about missing user tokens.

​

Recommendation

Make sure all search queries include the user token. Also check that you turned on the enablePersonalization flag in the index settings in the dashboard.

​

Mismatched or non-persistent user tokens

Your user tokens might not persist across sessions, or might not match the tokens you use when sending events. Either case can cause personalization issues in search results.

​

Symptoms

• Personalization doesn’t apply evenly, so some searches show personalized results while others don’t.
• A/B test results may be unreliable, because the control and variant groups may not show the expected differences. This signals an A/A test scenario.
• The Advanced Personalization dashboard shows warnings such as:
  • Low queries with personalization applied
  • Low queries with personalization found
  • Low users with personalization applied
  • Low users with personalization found

​

Recommendation

Make sure user tokens from search queries match sent events. Also persist user tokens, including anonymous ones.

​

Ranking criteria issue

If the ranking criteria for search results are too narrow, personalization has little room to affect the outcome. This can happen when the criteria limit results to a specific set, such as a single product or category.

​

Symptoms

• User profiles are generated and appear in the dashboard.
• The Advanced Personalization dashboard doesn’t display any errors or warnings.
• The Search Simulator shows little to no change in search result rankings, even when personalization is on and you choose different user profiles.

​

Recommendation

Broaden the ranking criteria so personalization can influence search results more. This lets personalized affinities change how search results rank.

​

Troubleshoot A/B tests

​

Premature A/B tests

Stopping A/B tests too soon, typically after a couple of days, can produce unreliable results.

​

Symptoms

• A/B test results show false positives or negatives, with marginal differences in metrics (such as for conversion rates and revenue)
• The test fails to reach statistical significance. This makes it hard to draw meaningful conclusions.

​

Recommendation

Run A/B tests for at least 2 to 3 weeks, or until you reach statistical significance. This ensures the results are robust and reflect actual user behavior.

​

Running an A/A test

Both the control and variant groups in your A/B test get the same treatment, typically due to non-persistent tokens or wrongly configured personalization settings. This makes both groups identical and the test useless.

​

Symptoms

• The A/B test fails to reach significance, because there’s no difference between the control and variant groups.
• Reported metrics may show little or no variation. This suggests personalization wasn’t applied as intended.

​

Recommendation

Check that personalization is on and that user tokens persist for both the variant and control groups. This ensures the variant group gets the intended personalization settings.

​

Mixing queries with and without personalization

If some search queries in the A/B test variant have personalization turned on while others don’t, the results vary. This skews the results, so it’s hard to measure the true impact of personalization.

​

Symptoms

• The A/B test results vary, with some queries showing personalization effects while others don’t.
• The number of queries analyzed may be much lower than typical traffic. This suggests the queries aren’t set up for personalization.

​

Recommendation

Ensure that personalization is turned on for all queries in both the control and variant groups. Use persistent user tokens across all queries to maintain consistency in the test.