Search by Algolia
How personalization boosts customer engagement
e-commerce

How personalization boosts customer engagement

You land on your favorite retailer’s website, where everything seems to be attractively arranged just for you. Your favorite ...

Jon Silvers

Director, Digital Marketing

What is retail analytics and how can it inform your data-driven ecommerce merchandising strategy?
e-commerce

What is retail analytics and how can it inform your data-driven ecommerce merchandising strategy?

There is such tremendous activity both on and off of retailer websites today that it would be impossible to make ...

Catherine Dee

Search and Discovery writer

8 ways to use merchandising data to boost your online store ROI
e-commerce

8 ways to use merchandising data to boost your online store ROI

New year, new goals. Sounds positive, but looking at your sales data, your revenue and profit aren’t so hot ...

John Stewart

VP, Corporate Communications and Brand

Algolia DocSearch + Astro Starlight
engineering

Algolia DocSearch + Astro Starlight

What is Astro Starlight? If you're building a documentation site, your content needs to be easy to write and ...

Jaden Baptista

Technical Writer

What role does AI play in recommendation systems and engines?
ai

What role does AI play in recommendation systems and engines?

You put that in your cart. How about this cool thing to go with it? You liked that? Here are ...

Catherine Dee

Search and Discovery writer

How AI can help improve your user experience
ux

How AI can help improve your user experience

They say you get one chance to make a great first impression. With visual design on ecommerce web pages, this ...

Jon Silvers

Director, Digital Marketing

Keeping your Algolia search index up to date
product

Keeping your Algolia search index up to date

When creating your initial Algolia index, you may seed the index with an initial set of data. This is convenient ...

Jaden Baptista

Technical Writer

Merchandising in the AI era
e-commerce

Merchandising in the AI era

For merchandisers, every website visit is an opportunity to promote products to potential buyers. In the era of AI, incorporating ...

Tariq Khan

Director of Content Marketing

Debunking the most common AI myths
ai

Debunking the most common AI myths

ARTIFICIAL INTELLIGENCE CAN’T BE TRUSTED, shouts the headline on your social media newsfeed. Is that really true, or is ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

How AI can benefit the retail industry
ai

How AI can benefit the retail industry

Artificial intelligence is on a roll. It’s strengthening healthcare diagnostics, taking on office grunt work, helping banks combat fraud ...

Catherine Dee

Search and Discovery writer

How ecommerce AI is reshaping business
e-commerce

How ecommerce AI is reshaping business

Like other modern phenomena such as social media, artificial intelligence has landed on the ecommerce industry scene with a giant ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

AI-driven smart merchandising: what it is and why your ecommerce store needs it
ai

AI-driven smart merchandising: what it is and why your ecommerce store needs it

Do you dream of having your own personal online shopper? Someone familiar and fun who pops up every time you ...

Catherine Dee

Search and Discovery writer

NRF 2024: A cocktail of inspiration and innovation
e-commerce

NRF 2024: A cocktail of inspiration and innovation

Retail’s big show, NRF 2024, once again brought together a wide spectrum of practitioners focused on innovation and transformation ...

Reshma Iyer

Director of Product Marketing, Ecommerce

How AI-powered personalization is transforming the user and customer experience
ai

How AI-powered personalization is transforming the user and customer experience

In a world of so many overwhelming choices for consumers, how can you best engage with the shoppers who visit ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

Unveiling the future: Algolia’s AI revolution at NRF Retail Big Show
algolia

Unveiling the future: Algolia’s AI revolution at NRF Retail Big Show

Get ready for an exhilarating journey into the future of retail as Algolia takes center stage at the NRF Retail ...

John Stewart

VP Corporate Marketing

How to master personalization with AI
ai

How to master personalization with AI

Picture ecommerce in its early days: businesses were just beginning to discover the power of personalized marketing. They’d divide ...

Ciprian Borodescu

AI Product Manager | On a mission to help people succeed through the use of AI

5 best practices for nailing the ecommerce virtual assistant user experience
ai

5 best practices for nailing the ecommerce virtual assistant user experience

“Hello there, how can I help you today?”, asks the virtual shopping assistant in the lower right-hand corner ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

Add InstantSearch and Autocomplete to your search experience in just 5 minutes
product

Add InstantSearch and Autocomplete to your search experience in just 5 minutes

A good starting point for building a comprehensive search experience is a straightforward app template. When crafting your application’s ...

Imogen Lovera

Senior Product Manager

Looking for something?

facebookfacebooklinkedinlinkedintwittertwittermailmail

I teamed up with Starschema Full Stack Engineer Soma Osvay to write about a project near and dear to my heart: developer documentation. Soma and I both hope you enjoy this article where we cover the use-case and challenges and the implementation Soma came up with.


Here at Starschema, we have a lot of projects we’ve done as part of a consultation with markdown documentation. When we’re supporting these existing solutions or wanting to develop a new project, we often want to search through all of the documentation. We currently don’t have a solution, costing us man-hours since we have to do this work manually.

Use Case

We need to be able to search for projects related to specific topics to check implementation specifics, get ideas from the code, and much more. The Sales team also needs a solution to determine what kind of projects our team has completed for certain topics and be able to quickly communicate it back to potential clients. It would also be great to have something like an internal Stack Overflow for our Developers since we are doing a lot of deep technical work in Tableau. Project Managers also need to be able to determine which employees have worked with certain technologies so they can get questions answered quickly and easily.

In short, we need to be able to search for our own projects based on the following attributes:

  1. Technologies used (coding languages, databases, etc)
  2. Project keywords (tags)
  3. Project documentation itself such as install instructions, technical documentation, etc.
  4. Timeframe of when the project occurred

All of these attributes exist within the internal documentation markdown files – we just need a way to search them.

Implementation Plan

The plan is to use a NodeJS CLI as a proof of concept which will:

  1. Scrape the top GitHub public repositories and grabs the README markdown file (which will ultimately represent our internal projects)
  2. Stores the documentation files into Algolia alongside the project’s basic information (title, programming language(s), tags, etc)

The CLI will contain advanced logging and command line arguments for ease of use. We also want to host it on the web so people can try it out.

Challenge

The biggest challenge at hand is the record size — Algolia only allows our records to be 100KB at the most. However, most markdown documentation files are much larger. The solution is that we need to split up the markdown files into multiple pieces within the Index. We also then need to make sure that when we search for something, a single project only appears once—even though it is split into multiple records.

Fortunately, Algolia has a distinct feature so we can de-duplicate these results very easily.

Indexer Implementation

To make using the Indexer as easy as possible, I opted to create a CLI for it as mentioned above. After supplying the arguments required, the tool will automatically initialize the repository, remove any existing records, and configure the relevancy settings.

Powering the tool is a straightforward GitHub API that grabs the requested number of top repositories and extracts all of their metadata and downloads the README file. It’ll also filter out repositories that are missing an owner or missing a README file, giving us the best results. Finally, it’ll also convert the markdown content to HTML for easier rendering in the frontend.

To keep the record size in check, the tool will automatically split up READMEs over 50k characters into additional records. This way the records won’t get too large but one record should still serve almost all repositories. It’ll then sync this information over to Algolia 100 records at a time so we can meet their batching recommendations as described here in the documentation.

Frontend Implementation

To serve as my starting point, I took advantage of the create-instantsearch-app library released by Algolia and launched an InstantSearch.js boilerplate. From here, I was able to add additional widgets provided by InstantSearch.js such as the pagination and the page size selector which worked great.

As we also collected repository metadata with the markdown, we also needed to customize the hits component to include this additional information. Often times the metadata is as important as the library description so developers can at a glance see if it’s a popular library, who released it, the tags, and much more. I also added facets so Users could filter by the programming language, tags, or how many times it has been forked.

The final piece to the puzzle was adding the ‘Open documentation’ button allowing you to quickly and easily read the markdown content for the repository in a pop-up without leaving the application. If the record we are clicking on has multiple rows, it’ll automatically load the additional records and concatenate them in the display—awesome!

Conclusion

This project was a fun test and really showed me how flexible Algolia is for different use cases such as ours. The ready-to-go widgets saved me a lot of time during prototyping and having relevant results from the first few keystrokes is super impressive. I also think it would be super interesting if we were able to harness the power of Algolia Recommend if we were able to generate enough events from Users clicking on projects internally.

You can view a live demo of the GitHub test project here, with a button that will set you up with the default demo credentials to view our index. Interested in the backend indexing code? You can find that here on GitHub!

Algolia Code Exchange


We hope that you enjoyed this in-depth article from Soma! If you are looking for more content like this, we have many more topics that we’ve covered on the Algolia Engineering Blog! If you’re new to Algolia, you can try it out by signing up for a free tier account.

About the authors
Michael King

Developer Advocate

githublinkedinmediumtwitter
Soma Osvay

Full Stack Engineer, Starschema

Recommended Articles

Powered byAlgolia Algolia Recommend

Good API Documentation Is Not About Choosing the Right Tool
engineering

Maxime Locqueville

DX Engineering Manager

API Documentation 2.0: From Browsing to Search
product

Peter Villani

Sr. Tech & Business Writer

Introducing our new navigation
product

Craig Williams

Director of Product Design & Research