Delivering personalized experiences in the era of heightened privacy

Online retailers build websites and mobile apps that guide shoppers from point A (the intention to buy something) to point B (finding and eventually purchasing what they need). This necessarily involves a fast and easy user interface, attractive product pages, smooth checkout processes, and more.

These elements guide your customers to find exactly what they are searching for and encourage greater discovery (extending their original intentions) through to customer loyalty. The goal is to ensure the shopping experience resonates with the buyer and keeps them engaged until they 'add to the cart' and eventually 'checkout'.

Equally important to retail is to personalize the online shopping experience.

You can achieve even greater revenue and loyalty by offering products and a user experience that speak directly to your consumers’ preferences and needs.

Personalized Recommendations Works:

• 37% Shoppers who placed an online order after clicking a recommendation
• 12.9 Minutes Shoppers who clicked on a recommendation spent an average of 12.9 minutes on the site, compared with just 2.9 minutes for those who didn’t
• 4.2 Times Greater conversions for mobile shoppers who searched and chose a recommendation

Automating a personal experience

Capitalizing on these benefits requires automation. In order to provide an experience based on an individualized and non-invasive understanding of customer needs, such automation comes from collecting data that feeds machine learning (ML) to develop artificial intelligence (AI) models.

When you personalize recommendations, customers engage at every touchpoint.

This leads to:

• 150% increase in order rate
• 20% increase in add to basket
• 24% increase in overall site conversion
• 13% lower bounce rate

A successful strategy for personalization balances three factors:

• Providing real value to the customer
• Offering powerful insights and control to the business
• Respecting the privacy expectations of the customer

These factors are not at odds with each other. As you’ll see, there is a win-win approach to providing machine-driven recommendations and personalized navigations that also respect privacy.

Think of it like a funnel in which two lines converge: one line is the customer’s satisfaction, the other is the business’s control. Privacy enables these lines to continuously move towards each other in the most rewarding way for both the customers and the business. While this perfect balance may be unique for every business, industry, and customer base, getting it right can make or break a company’s online success. 

Piyush Patel
Chief Biz Dev. Strat. Officer for Algolia

The numbers illustrate how providing personalized shopping increases discovery, where a customer arrives on a site to buy one item and finishes with a cart full of products. The numbers also demonstrate the increased customer belief that your business is their go-to shop. 

Personalizing recommendations creates a one-stop shopping experience with a future value as well – because tailoring your products and website to a customer’s personal intentions (as detected by their past search and buying activities) will sharpen and fine-tune their long-term buying decision-making. 

There’s no turning back from the trust built when the customer feels understood by its preferred business

Today’s consumers want personalized recommendations.
83% of consumers expect personalization within moments and during their whole buying journey. 
Source: Forbes

Finding the right balance. Powerful use cases.

A combined pharmacy and superstore may suggest shampoos and other items based on a customer’s past medical prescriptions. For example, their prescriptions may indicate allergies incompatible with certain shampoos. Going one step further, the store may start recommending items for a customer’s whole family based on similar private information. 

Another real-world example is a grocery store that proposes a healthy diet based on a customer’s food-buying and pharmaceutical history. While medical histories might be too much for some customers to reveal, other customers might welcome the opportunity – given the health and efficiency benefits. But would those same customers be happy if, every time they pass this grocery store, they receive a text message telling them about a promotion on their favorite items?

There are many powerful retail scenarios to consider as you explore the right balance for your business.  In these examples you can see what's at stake – great, useful information based on gathering and processing personal data.

 

The tenets of personalization

The age of the personalized online experience is here

Personalization is evident on Google Maps, Spotify playlists, Twitter feeds, or Amazon’s landing pages. Your retail business can offer the same quality of experience by implementing the four tenets of any system that offers a personalized experience.

Build a data first culture within your company

Identify data signals - from your transaction sources and from real-time customer engagements and feedback - to understand where and how your customers engage with your products. Data may also suggest what promotions, pricing, or marketing strategies are working. In essence, you must run your business according to the story your data is telling you.

The first step to connecting to all of these data sources, adding meaning to this data, and creating a smooth pipeline to update in near real-time is the first step.

Build customer preferences and segments

Employ AI algorithms to transform your company-wide data into meaningful personalized preferences and profiles using machine learning, and to detect consumer buying 2 patterns and industry trends. The personalized interface that you build is just as important as the accuracy of your personalized data.

With these two tenets in place, you'll have the foundation on top of which to build unique customer experience. The next two are the the technology execution and best practices.

Design the UI/UX

Personalized recommendations and merchandising need to fit seamlessly into the buying journey – they should not be overbearing nor marginalized. Thus, to find the right balance, you need to design the experience as you implement it.

Build it with an API-first composable architecture

Another consideration that facilitates the best implementation of the first three tenets. Your system architecture.

• You want software that provides programmatic access (APIs) to its data and functionality. APIs break down system and data silos, thus enabling the centralization and multi-functionality required to build the personalized dataset.
• The same is true for the user interface – your front-end engineers should use composable components that plug, play, and combine to more easily adapt to different parts of your websites.
• Finally, composability adds transparency and maintenance simplicity to your system by enabling your engineers to manage the privacy of your customer data 4 in each component separately.

Ok, so you’ve built the experience. Now, let’s look at the two different kinds of intent that users follow:

• Search: One customer searches for a specific product in mind – a well-known running shoe.
• Browse: Another customer browses with the intent to (maybe) buy a running shoe.

You want to create a personalized discovery journey, where the user starts with a specific need and discovers more than they had anticipated. Let’s see..

There are two kinds of intent: searching and browsing. 

For both, you need to display detailed search results and product information, promotions, facets for categorizing the information and drilling down, and even FAQs on the subject. All of this satisfies the search intent. But the browsing experience needs more detail and interactivity than a search. 

Customers, while pursuing their different intent, directly benefit from personalized recommendations, which encourages them to discover more of your catalog. Reshma Iyer Dir. of Product Marketing at Algolia

Advanced buyer journeys for the customer looking to buy running shoes.

• Encourage them to browse through related sports items. For example, guide them towards exploring related category pages (Marathon Runners or Athletic Shoes).
• Encourage them to go beyond sneakers to discover other sporting items (stopwatches, water bottles, sweatpants), or even to go beyond sports and start thinking about dress shoes, cooking utensils, or even home improvements.
• Finally, you can group runners into the same profile, thus combining the actions of many runners to fine tune the recommendations.

When driven by an accurate understanding of these shoppers, dynamic experiences can be sure wins.

The most common features of high quality personalized experiences

There are 3 types of personalizations:

• Item-to-item level
  • Display related items or items frequently bought together
• User-to-item level
  • Display items based on user preferences and previous buying patterns
• Group-to-item level (user profiles)
  • Display items based on customer profiles and segmentation

Personalization	Display search results and product suggestions on landing pages, product, and category pages, and purchase screens – derived from user-activity analytics that help discern affinities, preferences, and future interests.
Recommendations	Display product or category-based suggestions based on user and group profiles, derived from user-activity analytics as well as product relationships within the catalog.
Trend Identification	Adapt the experience to current user and industry trends based on top sellers, most viewed, and other such customer-activity signals that occur in the last 24 hours or last week.
Outbound personalization	Reach out to your customers with personalized emails, text messages, and newsletters, with personalized information based on the customer’s recent purchases and overall preferences. Pushing emails is tricky in terms of privacy expectations, but when done right, it establishes a more personalized relationship between you and your customers.

Where technology comes in …

Because automation can impact every screen, navigation path, and marketing campaign, it must be supported by robust technology and intuitive UX. For example, if your website is slow or difficult to use, you’ll lose your customers.

The best sites keep customers shopping with:

• uncluttered, dynamic multi-column layouts,
• components like carousels,
• non-intrusive banners and promotions,
• and intelligently-placed product recommendations.

The best sites do all that in milliseconds. Search at the blink of an eye is the difference between keeping an end-user’s attention and bouncing off.

Personalization must-have practices

Show products that appeal personally to the user.

There’s the notion of the paradox of choice in retail, where limiting options reduces buyer anxiety and makes it painless and therefore easier for consumers to weigh their options and make choices quickly. And because of this speed and reduced anxiety, they tend to stay longer and discover more.

What that means for your personalization practice is: limit your shoppers' options. This is where the role of personalization becomes critical. You want to be able to show the right set of products, which can be any number, but not a large set of irrelevant products.

Therefore, you want to (a) build an interface that displays less items in an uncluttered (but not too sparse or limiting) way, and (b) display some items that are personally relevant to their preferences and current intentions. (We say some, because a great discovery experience must combine preferences with other items in your catalog, to allow your customers the freedom of choice to go beyond their current preferences.)

Turning data into intelligence – from analytics to machine learning

To understand the power of data, it’s worth seeing the amount of information you can collect. There are many kinds of signals and data points used to find buying patterns. It all starts with capturing user behavior in the form of analytics data.

Fundamental to all this is the need for a personalization engine to learn continuously. AI algorithms must be designed to keep that dynamic and adaptive learning capability in mind, so that personalization remains a journey to be improved on continuously and not a destination or one and done approach. Here is a large but non-exhaustive list:

Session-based signals	Views Clicks Purchases
Defining significant actions (for conversion)	Viewing a product’s characteristics Multiple clicks within product pages Clicks on search results and banners Buying or adding to a wish list Performing certain searches, using common keywords Making comments or rating products
Monitoring duration	How long a consumer interacts with a website (not just sitting on an unmoving screen, but actually scrolling and clicking on the content) How long they spend viewing similar products
Geo-based signals	Store and restaurant locations
Grouping users	Discover Trends – dynamically change relevance and recommendations based on customer buying patterns in the last 10 or 24 hours
A/B testing	Improve and tailor the algorithms to your business ➔ Test their effectiveness ➔ Iterate to improve their quality and pertinence of personalization to your business

But what about privacy?

There’s nothing inherently invasive in capturing all these events. For example, in the runner’s example above, patterns can be discovered without knowing the runner’s name, address, or any other identifying details. Personal questions, like are you married, what’s your gender, what do you do on a day off, and so forth, need not be asked.

But a system can easily cross a line. For example, when customers log into a system, it’s quite easy to trace who they are, where they live, and what they’ve done in previous visits. However, that may be more than the customer has agreed to. That’s where anonymizing data comes in, by replacing the real-world identity (name or email) with an anonymous number. In that way, no one can use or misuse the data to find out who the user really is.

So – Does respecting privacy sound impossible? A false promise? Read on.

 

How does a positive privacy model increase your customer base

People want personalized experiences with no strings attached

Violations of your users’ personal spaces can make them feel threatened, vulnerable, manipulated, and not free.

People don’t want to feel like they are being watched or followed. They want to browse and shop in privacy. It’s their time to be alone and make decisions in the comfort of their own personal space.

They want to consent or not. They want to know what you are doing, what information you have. They want to set the boundaries and be able to say “stop” at any moment.

Privacy is precious: You never want to push the boundaries of customer trust. Businesses that underestimate the importance of privacy will lose their customers.

Privacy is respect: And respect engenders trust.

Privacy supports business goals: It is not a constraint. You are not losing opportunities or putting yourself at a competitive disadvantage if you don’t collect unnecessary data riddled with questionable information or worse, personal data.

Privacy is not a constraint.

Respecting customer privacy can add value to your business, and it involves important business considerations:

Fair use of personal information ensures data quality.

You don’t need more data than is necessary. This is a basic machine learning precept – feed your system with only the most informative and reliable data. Quality input brings about the most pertinent results. Big data is not the basis of personalization. You need only small amounts of smart data.

Transparency inspires trust and loyalty.

You establish more trust by telling your users what data you are collecting, how the data will be used, and for how long will it be stored, than by secretly capturing private information.

Not knowing the identity of a person can still deliver conversion.

To personalize an experience, you only need to know a user’s behavior. Knowing behavior does not also require knowing a user’s name and address or what’s in their wardrobe. You are only interested in capturing buying and viewing habits – i.e., what they click on, view, and purchase.

Removing user identities enables sharper profiling based on behavior not people.

Not knowing identities leads to fruitful generalizations. Grouping people with similar buying patterns helps your customer see an even more complete picture of what they can do as they shop. Even in the cases of providing 1:1 personalization, session based data is all that is needed. Personally identifiable information actually doesn't add any value here. Grouping should be based on privacy-respectful data collection that combines habits and preferences not identities.

The hard facts about respecting privacy

So the overriding question is: How can an online retail business respect end-user privacy and comply with privacy laws, while still optimizing and maximizing a fully personalized end-user experience. We’ve already discussed many ways to respect your customers and to gain their trust. We’ve outlined the benefits to your business. Now we’ll add another two key elements: security and transparency.

Laws and industry standards

Here are some relevant laws and industry standards that pertain to privacy and collection of data:

• GDPR - Legal consumer protections
• FIPPS - Best practices
• SOC2 - Industry security certifications

Securing your system

Let’s imagine you’ve stored your user data in a “closed room” in which only machines and your customers have the key to access. As the machines collect data, they should remove all identifying data and replace it with an identifying number. The machines should save the analytics data without it being traced back to the real customer’s name or email.

Customers who visit your site are given a key to this room. There, they find the products that interest them using only an anonymous identifying number. There’s no way for the machine or a customer to trace back to find out who they or another user is in the real world. Nobody else is allowed into the room.

But is the room really secure? Some concerns come to mind:

• The engineers come and go in this room as they develop and maintain the system
• It is nearly impossible to avoid all breaches and leaks – whether intentional or accidental

It’s indeed hard to create a 100% secure room. It requires continuously monitoring, upgrading, and proactively looking to address any vulnerability, even by designing a bounty program. Security best practices respect privacy. There are many technologies on the market at low cost that can help your business create a secure and reliable machine learning environment. And, the law and industry standards provide clear guidelines that actually help not hinder your technology.

Transparency and Consent — Understanding user expectations

Ask your users if they want to opt-out or opt-in. Offer them the option for informed consent, by telling them exactly what you are doing with their personal data.

Opt out

This means the customer does not want personalization. Or they do want personalization but are not (yet) ready to supply personal information.

Opt in (consent)

Their concern is to know exactly what they are opting into. You’ve got their trust, and they’re indicating openness to personalization. But they want to know what you are collecting and whether this data will be shared with other parties. Transparency needs to be simple to understand and the data collected needs to be reasonable. No hidden clauses or tricks when asking for consent. But keep it positive: you should try to encourage your customers to opt-in by informing them of the benefits they receive from personalizing their search and recommendations.

Opt in (for logged-in customers)

Known customers probably don’t mind a maximum of personalized services, such as for customer support or being guided in large projects like planning weddings or buying medical products. But it’s worth distinguishing a personalized service, which comes with its own set of privacy expectations, and a personalized buying experience, which should not differ much from the more circumscribed expectations of all visitors.

Transparency is paramount: known customers might be willing to give more information than others to enhance personalization. But they need to know what you are doing with their data, and the experience needs to match their expectations. For example, It’s probably not a good idea to surprise them with a phone call from a doctor who has just learned (from your pharmacist) that you need special care.

Moving towards explicit permission.

In 2021, Apple announced a major privacy change with the launch of iOS14 that would completely change the way 3rd party developers on the Apple Store would be able to reach/contact users of their app. The biggest change being apps are now required to explicitly ask users for their permission when they want to track them across apps and websites owned by other companies.

 

The low costs and high gains of getting it right

Do not spend time and money collecting more data than you need.

Let’s assume the biggest players in the industry have a lot of data. Does this give them an advantage? No. Any company — small, mid-size, or enterprise — can now easily collect data. Thus, the real question is not quantity but quality. Quality means collecting only the data you need, which brings down costs in terms of storage space, maintenance, and infrastructure.

On this point, using reputable cloud services or software companies that specialize in capturing analytics can bring down costs in the long run, because they take care of all storage, maintenance, and infrastructure costs.

Automation does not have to convert your retail business into a tech company. Your engineers can shop around for the best algorithms and then customize and optimize these algorithms for your particular organizational needs.

Piyush Patel

Chief Biz Dev. Strat. Officer for Algolia

Do not reinvent the wheel – avoid homegrown algorithms.

As a rule you should keep your machine learning models as simple as possible. Luckily, you can find simple yet powerful, low cost algorithms and technologies on the market. But the quality varies a lot. it's important to keep the process agile - easy to set up, implement, and update (based on learnings), and flexible for the business to fine-tune. The focus should be on usage, not on building a new model every time from scratch.

Machine learning does not have to be rocket science. Think of what spreadsheets did in the 90s and continue to do today. They improved our way of doing business by leaps and bounds, with simplicity: rows, columns, and functions, with pivoting, cross-referencing, and charts. And no company has ever had to code their own spreadsheet. It’s the same with simple machine learning algorithms – your engineers can shop around for the best algorithms and then customize and optimize the algorithms for your particular organizational needs.

Don’t spend money on more engineers.

You no longer need 100s of developers to build a fully personalized experience. To understand this, it’s important to list the engineering skills needed to build the components of a personalized experience. No one reinvents the wheel these days. The creativity lies elsewhere – it’s in how these engineers tailor their applications to your catalog, and business, and customer needs. And the team should be small enough to function within a successful agile environment.

Engineering skills needed:

• Data scientists who work data and models
• ML engineers who work closely with data scientists to design and implement the models
• Front and back end developers who code the UI/UX that your customers see

 

Last word – Don’t overreach. Give people the option.

So, are the largest online retailers the only ones who can offer a robust and powerful personalized experience? No

Can small, mid-size, and enterprise companies build ecommerce experiences as powerful as the biggest players? Yes

Simply minimize overreach. Doing too much is costly and will not add more value than a less costly system. It follows the law of diminishing returns.

Leave space for personal choice

Equally important, overreach needlessly pushes the privacy boundaries and can make your customers uncomfortable. That’s when the shopper drops out. Today’s customer is ultimately the one in control. Personalization should be an option. Do only what is necessary to create a reasonable but not overwhelming personalized experience. 

And don’t forget – you always want to leave space for the user to explore beyond their preferences. A successful online retail strategy leaves room for discovery both with and without the use of AI and machine learning algorithms.