For engineers, an online marketplace means ecosystem – the front-end and back-office systems that run a business. Modern ecosystems are composed of third-party software components built by experts in their fields. For example, search and AI specialists build search and discovery components, financial companies provide payment functionality, and business experts create the transactions and ordering processes.

Ecosystem engineers don’t have to code the business logic of a composable architecture. Instead, they integrate the third-party API components into their existing infrastructure, and focus only on managing the exchange of data and business logic between the components.

In our previous article on matchmaking and white-labeling, we showed that the components and the composable ecosystem itself can be shared, where one business shares its infrastructure and software components directly with other businesses, or shows them how to run the same components on their own infrastructures. This article focuses in on the “matchmaking”, or the parts involving one marketplace introducing expertly-written API components to other marketplaces.

Typical matchmaking scenarios

To illustrate matchmaking, we described in the first article how two entrepreneurs built a successful marketplace ecosystem. Their story begins here. They started out by sharing their interface and infrastructure, which is essentially what Amazon, Etsy, and all other marketplaces do. Then came the spin-offs, matchmaking, and white-labeling.

Multiple UIs

One of their vendors, a coin collecting business, wanted a different interface but didn’t want to build or purchase its own infrastructure. The marketplace architecture (API-composed, headless) enabled the engineers to decouple the user interface from the back end components. In this way, the coin collector was able to access the API services of the ecosystem from their own website. 

From Ecommerce to Media

Another vendor wanted to stream music. They created a hybrid infrastructure: some components ran on the marketplace’s infrastructure; other components (for media streaming) ran on the vendor’s infrastructure.

White-labeling

Other vendors saw the need to copy the infrastructure and build their own website using their own infrastructure. At that point, the original marketplace engineers saw an opportunity to white-label their marketplace ecosystem, which enabled the external vendors to build their own marketplace by doing the following: 

• Buying the white-labeled version of the original marketplace ecosystem and installing it on their own infrastructure
• Buying the third-party components, and then integrating and configuring them to their needs.

So how’s this done? An quick overview [spoiler alert]

The business that shares its ecosystem (A) provides a data-exchange API that allows another company (B) to send its data to A’s index. Company A’s data-exchange API receives, validates, and, if there is no error, updates the data on its infrastructure. This presents an expansion opportunity for Company A, as they introduce these partnership and integration channels. We’ll see how this all works with Algolia’s Search and Discovery API.

Going one step further, A can share its ecosystem via white-labeling, allowing B to copy its ecosystem software to its own servers. In this scenario, B manages its own ecosystem and becomes a direct client of the companies that provide the third-party API components.

Let’s get into the details.

The software components behind the ecosystem and customer journey

There are several key driving forces at work here: 

• Search
• Discovery
• Ordering
• Payment
• Delivery

Of course there’s more. For example, on the back end, there’s inventory management, CRMs, pricing tools, and other such applications that run the business from A to Z. And on the front end, there are user comments, recommendations, images – just to name a few. 

The marketplace ecosystem offers the following services to create a customer journey: 

1. First, customers search and discover, then decide what they want
2. Next, they order, pay, and receive a delivery (or stream, read, download)

We won’t cover the second part. You can explore the large number of APIs and platforms that manage the ordering process (carts, wish lists, customer accounts, CMSs), payment (general payments, one-click, deferred), and delivery (links – fast, pickup/collect, stream, read). 

In this article, we’re interested in the first part of the customer journey, the search and discovery, where consumers search or browse for what they want to buy, stream, read, or download. As you’ll see, search and discovery is more than a search bar, it’s about the display and navigation of content as the end-user interacts with a digital marketplace. 

3 ways to share a Search and Discovery API

Algolia provides the software components for the Search and Discovery journey. As already stated, search and discovery involves more than a search bar, it’s about menu navigation, searching, browsing, faceting & filtering, and the relevance and visual layout of the search results, category pages, and product views. Essentially, Search and Discovery covers every part of the front-end user journey before a customer selects the item(s) they want to buy (or watch, read, download). It also continues after the purchase, with the Continue to Shop button.

Underlying all ecosystem sharing is a flexible API-first, feature-rich product. It starts with Company A creating a search-led ecosystem, using a Search and Discovery API like Algolia. Company A shares its ecosystem with Companies 1 .. N, either as partners or spin-offs. 

To understand how sharing works, it’s important to understand what an index is. Algolia’s Search and Discovery technology relies mainly on a specially designed index and fast indexing process. Once a company extracts and formats its data, an indexing API is used to add, remove, or update the data to Algolia’s servers. The data is thus located on Algolia’s hosted servers, which are located in all regions of the world, to minimize latency in the transactions. The company can choose any region that best suits their user base. The API can access any server in any region.

Here are 3 ways you can share a Search and Discovery API:

• Same index, same website – Companies use the same website and index. This is the first step, no spin-off, no white-labeling. This is what Amazon and Etsy do. All vendors send their data to the same, shared index, and the marketplace provides the website.
• Same index, different website – Companies share the same index but not necessarily the same website. In this scenario, Companies 1 .. N send their data directly to Company A’s index, but they build their own website and access the shared index from their own, separate website. This is a UI-matchmaking scenario, where the website’s features (such as searching, navigating, merchandising, and everything else) are introduced to the vendors as components for their own websites.
• Same index structure, different servers – Companies separate 100% from the marketplace and create their own marketplace. But first they obtain a copy of the index structure (without the data) and recreate it on their own Algolia account/servers. They then populate it with their own data. The two indexes are now split and there’s no more connection between the two. These companies must build their own website and become direct Algolia clients. Here’s where we see matchmaking and white-labeling, on both front and back ends, where the vendor buys, integrates, and configures the components they have been introduced to.

How to share the Search Index, and what makes up the Index

The challenge for indexing is how to manage the data that vendors send to the marketplace index. This is done via a data-exchange layer which takes in the data and validates it. For this, the marketplace exposes a data-exchange API to its vendors. This API accepts the vendor’s data, validates it, and then sends it to Algolia, if there’s no problems with the data. 

Thus, there are two APIs:

• The Algolia API that updates the search indexes
• The marketplace API that receives and validates the incoming data

We won’t call the marketplace API an ETL. An ETL normally Extracts, Transforms, and Loads data. But as defined here, there is no need for any Extraction or Transformation, because the marketplace will ask for the data already structured and formatted in the same JSON that will be sent to Algolia’s API. 

The job, therefore, for the marketplace API is to receive-validate-load the data. The validation process should ensure that:

• the data is complete, correctly formatted, and won’t break the Algolia index
• the data is not too large or sent too often, thus slowing down the system

The contents of the index

The marketplace must create an index structure that drives its business and website’s search needs. An index contains attributes which have four purposes:

• Attributes for search (keywords and textual information used to match what users type in the search bar)
• Attributes for display (images or additional on-screen information, like pricing, author, brand)
• Attributes for filtering and faceting (brand, size, genre)
• Attributes for sorting (prices, date) and ranking (number of sales, ratings)

Additionally, some marketplaces will offer attributes that can be used for merchandising and content management, like banners or item promotions.

The data structure

Sharing an index only works when the marketplace sets a standard. Algolia accepts a schema-less JSON file, which is flexible enough to allow any structure or content. The marketplace uses this flexibility to create a precise structure for its unique needs. However, the outside vendors are more restricted, in that they must conform their data to the structure the marketplace has designed. Here are some of the guidelines:

• Some attributes will need to be common for all vendors, like name of item, price, description. Every vendor needs to supply these required attributes.
• Other attributes will be optional. While every optional attribute has a clearly-defined purpose (for search, display, faceting, or ranking), the vendor will be free to ignore it. Of course, for the best representation of their items on the shared website, vendors should supply most, if not all, attributes. 
• The most advanced marketplace websites contain dynamic front-end logic so that if a vendor supplies a certain optional piece of information, the website can show something different for that vendor.

Content management, merchandising

• Some attributes can be used for promoting individual items. The actual use of those attributes for merchandising and content management will be managed by the marketplace: the engineers set up on-screen placeholders for promotions, which the vendors can make use of by supplying the right content.

Relevance & Index settings

• There’s not much to be said here: The same index settings will be shared by all vendors. This means that all configurations like typo tolerance, ranking, searchable and facet attributes, and many other settings that impact relevance, will be configured by the marketplace and not the vendors. 

More about the data-exchange API

The marketplace API should have the following features:

• Takes the same JSON file that will be sent to Algolia.
• Sends a response – success or error, with details.
• Enforces batch processing, which means that if the data is very large, it will break the records into small batches, and send the batches in regular intervals (2 minute each, or thereabouts).
• Is backwards compatible, like any API.

Migrating the index to a vendor’s own infrastructure

• This requires no structural changes by the marketplace or the vendor. The vendor simply sets up its own Algolia account and applies the same marketplace settings to its own index. The vendor then starts sending data to its own index using the same JSON.

How to share the Search UI

One thing to note at the outset regarding the front end: the same index can power multiple user interfaces. This is key to ecosystem sharing: different vendors create their own UIs while using the same indexing API methods. This is possible because the API method used for searching can point to the same index and server. 

The search UI experience, and its look and feel

• The basics of search are: search bar, search results, faceting, pagination, sorting. 
• These functionalities can look very different from one UI to another. They can be placed anywhere on the screen. The UI can differ vastly from one website to another. But the basics don’t change and the underlying index can remain the same.

The search results

• The index settings drive the relevance. Relevance can be summed up as the accuracy and order of the search results. 
• That said, the data in the index can respond differently to these settings depending on its content and formatting. Therefore, when vendors send data, they need carefully follow the content guidelines by the provider of the index. 
• When the vendor moves its data to its own Algolia account and servers, they may keep the same structure and settings initially. However, over time, they should probably change the index configuration and format of their data to suit their changing needs, to stay competitive. And this is the main reason for copying the ecosystem and white-labeling – more control over the search experience.

Discovery

• Discovery is about going beyond the search and encouraging your users to explore the entirety of your catalog. It includes navigation & browsing, category pages, carousels, merchandising, and any such content management. Whether discovery for ecommerce or media, there is no limit to how you can configure and use these features, and they can all look very different from one UI to another. 
• Whatever feature you use, the underlying index remains the same. When sharing an infrastructure, you need to follow the guidelines of the index provider.

Sharing your expertise

Any discussion about technology is invariably about the people responsible for creating it. While the APIs provide the financial, business, and user-interface expert functionality, the story is about the people who design and develop the software, and who eventually expose their expertise in the form of APIs. These experts are both technical and non-technical. Their grit and sharing is what runs our digital economy. 

—

Algolia Search and Discovery integrates closely with headless ecosystems. To find out how it can transform your digital strategies, sign up for free and see it for yourself. Or get a customized demo from our search experts today.