Search by Algolia
What is online retail merchandising? An introduction
e-commerce

What is online retail merchandising? An introduction

Done any shopping on an ecommerce website lately? If so, you know a smooth online shopper experience is not optional ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

5 considerations for Black Friday 2023 readiness
e-commerce

5 considerations for Black Friday 2023 readiness

It’s hard to imagine having to think about Black Friday less than 4 months out from the previous one ...

Piyush Patel

Chief Strategic Business Development Officer

How to increase your sales and ROI with optimized ecommerce merchandising
e-commerce

How to increase your sales and ROI with optimized ecommerce merchandising

What happens if an online shopper arrives on your ecommerce site and: Your navigation provides no obvious or helpful direction ...

Catherine Dee

Search and Discovery writer

Mobile search UX best practices, part 3: Optimizing display of search results
ux

Mobile search UX best practices, part 3: Optimizing display of search results

In part 1 of this blog-post series, we looked at app interface design obstacles in the mobile search experience ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Mobile search UX best practices, part 2: Streamlining search functionality
ux

Mobile search UX best practices, part 2: Streamlining search functionality

In part 1 of this series on mobile UX design, we talked about how designing a successful search user experience ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Mobile search UX best practices, part 1: Understanding the challenges
ux

Mobile search UX best practices, part 1: Understanding the challenges

Welcome to our three-part series on creating winning search UX design for your mobile app! This post identifies developer ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Teaching English with Zapier and Algolia
engineering

Teaching English with Zapier and Algolia

National No Code Day falls on March 11th in the United States to encourage more people to build things online ...

Alita Leite da Silva

How AI search enables ecommerce companies to boost revenue and cut costs
ai

How AI search enables ecommerce companies to boost revenue and cut costs

Consulting powerhouse McKinsey is bullish on AI. Their forecasting estimates that AI could add around 16 percent to global GDP ...

Michelle Adams

Chief Revenue Officer at Algolia

What is digital product merchandising?
e-commerce

What is digital product merchandising?

How do you sell a product when your customers can’t assess it in person: pick it up, feel what ...

Catherine Dee

Search and Discovery writer

Scaling marketplace search with AI
ai

Scaling marketplace search with AI

It is clear that for online businesses and especially for Marketplaces, content discovery can be especially challenging due to the ...

Bharat Guruprakash

Chief Product Officer

The changing face of digital merchandising
e-commerce

The changing face of digital merchandising

This 2-part feature dives into the transformational journey made by digital merchandising to drive positive ecommerce experiences. Part 1 ...

Reshma Iyer

Director of Product Marketing, Ecommerce

What’s a convolutional neural network and how is it used for image recognition in search?
ai

What’s a convolutional neural network and how is it used for image recognition in search?

A social media user is shown snapshots of people he may know based on face-recognition technology and asked if ...

Catherine Dee

Search and Discovery writer

What’s organizational knowledge and how can you make it accessible to the right people?
product

What’s organizational knowledge and how can you make it accessible to the right people?

How’s your company’s organizational knowledge holding up? In other words, if an employee were to leave, would they ...

Catherine Dee

Search and Discovery writer

Adding trending recommendations to your existing e-commerce store
engineering

Adding trending recommendations to your existing e-commerce store

Recommendations can make or break an online shopping experience. In a world full of endless choices and infinite scrolling, recommendations ...

Ashley Huynh

Ecommerce trends for 2023: Personalization
e-commerce

Ecommerce trends for 2023: Personalization

Algolia sponsored the 2023 Ecommerce Site Search Trends report which was produced and written by Coleman Parkes Research. The report ...

Piyush Patel

Chief Strategic Business Development Officer

10 ways to know it’s fake AI search
ai

10 ways to know it’s fake AI search

You think your search engine really is powered by AI? Well maybe it is… or maybe not.  Here’s a ...

Michelle Adams

Chief Revenue Officer at Algolia

Cosine similarity: what is it and how does it enable effective (and profitable) recommendations?
ai

Cosine similarity: what is it and how does it enable effective (and profitable) recommendations?

You looked at this scarf twice; need matching mittens? How about an expensive down vest? You watched this goofy flick ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What is cognitive search, and what could it mean for your business?
ai

What is cognitive search, and what could it mean for your business?

“I can’t find it.”  Sadly, this conclusion is often still part of the modern enterprise search experience. But ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Looking for something?

How we tackled internationalization in our Zendesk integration
facebookfacebooklinkedinlinkedintwittertwittermailmail

Zendesk customers are worldwide, coming from every continent. It’s no surprise that their Help Centers support multiple languages out-of-the-box. When we launched our Algolia for Zendesk integration, it shipped with English support by default, and you could extend it to handle other languages. Today, we’re proud to announce that our integration supports 30 languages.

Algolia has always been language-agnostic. You can search in an English, Arabic or Chinese text without touching the parameters; but each integration comes with some specific features. Searching in help articles is a specific use-case, and since we’re providing some front-end features (e.g. an autocompletion menu), we also had some text that needed to be translated ( e.g. “10 results found in 2ms”).

Our initial release had some flaws that we quickly discovered by talking with our first integration users. We got great feedback from pretty big Zendesk users that needed multiple language support out of the box, like Dashlane, whose Help Center is available in English, French and Spanish.

TL;DR

I’ve learned the hard way that there’s no magic bullet. Languages are too different to expect being able to simply replace parts of your text easily. Some things aren’t obvious – for example, how one thousand is written numerically in English (1,000) vs. French (1 000)) – some languages have multiple plural forms and you can’t expect that the sentence construction will be the same in any other language. As soon as you want to have dynamic content in a sentence, you’ll need to use some form of templating logic.

How did it work before?

When you call our front-end library, you can pass some parameters. We had exposed a simple translations parameter, in which each key held either a string with the desired value for all languages or an object associating a locale with a translation.

translations: {
  placeholder_autocomplete: 'Search',
  found_in: {
    'en-us': 'Found in',
    'fr': 'en'
  }
}

We then decided that we wanted to embed a few languages directly inside the application by default, to ease our users’ life. That’s when we’ve learned that the simple solution I had developed wasn’t sufficient at all.

Flaws of the previous solution

We used Gengo to get our sentences translated in a first batch of 5 languages. Satisfied with the results, we ordered a total of 30 translations (most of the ones supported by Zendesk). Some languages had a grammar similar enough to the English one that the integration was straightforward. Others brought up issues with the current solution, at three levels.

Different ways of displaying the same information (dates, numbers)

We have some great tools to display the same information in multiple languages.

Your browser ships with Number.toLocaleString and Date.toLocaleDateString. Unfortunately, those methods by default use the user’s localization. ECMA2015 has added the support for a new locales parameter, but its support by browser vendors is still too low for us to confidently use it for an integration targeting our customers’ end-users.

At that time, we simply ignored the different ways of displaying numbers depending on the language. However, for dates, we used the great moment.js library.

Plural forms

This one is a very basic one, but I didn’t think about cases where a plural form was not needed in English but needed in other languages (and I’m French):

Form English French
Singular Found in Trouvé en
Plural Found in Trouvés en

Something that came as a real surprise to me though was that languages also have multiple plural forms:

Amount English Czech
1 result výsledek
2 and 3 results výsledky
more than 4 results výsledků
Sentence construction

Sometimes it’s just the sentence construction which is completely different. In some languages, the words positioning can be totally different:

Language Translation
English No results found for “query”
German Keine Ergebnisse für “query” gefunden
Japanese “query” の結果が見つかりませんでした。

Another small difference between languages can simply be on the punctuation side. You have different quotes in different languages:

English Czech
“query” „query“

It’s with all those cards in hand that we’ve started realizing we needed a better framework.

What we ended up with

We kept the same logic for the translation object, but this time added the ability to have logic on top of it.

The static standalone sentences are still simple translated strings:

filter: {
  en: 'Filter results',
  ru: 'фильтр'
  th: 'ผลลัพธ์จากตัวกรอง'
}

Others now are functions. Those function are called with access to the other translations (this) and take the dynamic parts as parameters:

stats: {
  en: function (nbHits, processing) {
    return this.nb_results(nbHits) + ' found in ' + processing + ' ms';
  },
  es: function (nbHits, processing) {
    return this.nb_results(nbHits) + ' encontrado' + (nbHits > 1 ? 's' : '') + ' en ' + processing + ' ms';
  }
}

For each locale (e.g. en-us), you can either override the root translation by using the en key, which will change the translation for every english-speaking locale (i.e. en-au, en-ca, en-gb and en-us), or by providing a locale specific translation by using the full locale as a key (e.g. 'en-us').

The only exception for this is Chinese, where Simplified Chinese and Traditional Chinese are too different to have a common root.

Going further

Once this was done, we started thinking about how we could improve the relevance in each languages.

The first thing we thought about is related to the type of queries the users might send. On a Help Center, it’s not unusual to have users type “how to do …”. The issue with that type of query is the potential noise related to words as common as “how”, “to”, “and” or “my”, that are called stop words in natural language processing.

A simple example speaks thousand words. Consider two articles:

Change your password How to delete your account
Just follow this link to change your password Click on “Delete my account”. In case you change your mind, your old login/password will still work during 14 days.

For the query “How to change my password ?”, without any special handling of those frequent words, the Change your password article would not even show up because of the lack of the words “how”, “to” and “my” in the article. How to delete your account would instead show up.

That’s where stop words handling kicks in. Algolia offers an automatic stop words removal feature, with the query parameter removeStopWords. You can either use it with true to remove all the stop words in all languages or limit yourself to a specific language. Activated with the example above, Change your password would come back first in the results list and How to delete your account second.

While this works great for full queries, its behavior can be a bit strange when used with prefix search, because the words are used as part of the query while they’re used as a prefix (the query “ho” for instance would match “how”, but the query “how to delete” would only match “delete”). That’s why we chose another solution.

Algolia also provides another query parameter called optionalWords, which can be used to specify which words aren’t required for a record to match. At each keystroke, we’re now looking at which stop words in the current language the query contains, and add them as optional words to the query. How to delete my account ? would still show up first because it matches more query words, but Change your password would show up in the results list.

This solution in the end brings a good balance between no stop words handling and the usage of removeStopWords, in terms of relevance but also user experience. This works especially well because the dataset we’re searching in is usually in the 100s of articles, so there aren’t that much relevant results for a query.

Next steps

Now that we have this framework, what could we improve upon?

The first thing would be to have a way to use language specific tags. We’re indeed displaying all the tags on the search result page by default, you can hide them with three lines of CSS, but having only localized tags could be an even better solution.

Another big improvement we will do will be to have one index per locale. This would allow us to move the whole stop words logic from the front-end to the Algolia index settings instead, which would save some space in the front-end library.

Feel free to contribute, the whole code is open-source, and we’d be happy to look at any feature you’d like!

About the author
Matthieu Dumont

Senior Software Engineer

github

Webinar: Breaking down content silos

Zendesk uses Algolia to drive collaboration and productivity.

Image of Chris WittmanImage of Britt Atilano
Chris Wittman - Customer SuccessBritt Atilano - Program Manager, IT Operations at Zendesk
Image of Chris Wittman
Chris WittmanCustomer Success
Image of Britt Atilano
Britt AtilanoProgram Manager, IT Operations at Zendesk
Webinar: Breaking down content silosSee how

Recommended Articles

Powered byAlgolia Algolia Recommend

Handling Natural Languages in Search
engineering

Léo Ercolanelli

Software Engineer

Algolia's top 10 tips to achieve highly relevant search results
product

Julien Lemoine

Co-founder & former CTO at Algolia

Inside the Algolia Engine Part 3 — Query Processing
engineering

Julien Lemoine

Co-founder & former CTO at Algolia