Blogcast – Building search for education and media, with Alex Patterson from CodingCatDev

Alex Patterson is founder and CEO of CodingCatDev, an online learning platform for beginning and experienced developers who want to learn about different technologies. Alex also produces podcasts and videocasts, interviewing and mentoring over 50 engineers worldwide. He is also a Cloud architect, building headless architecture.

We met Alex at Jamstack Conf 2021. He joined our booth and introduced himself as a long-time Algolia user. We were intrigued to learn more, so I contacted him the next day and he agreed to sit (zoom) with me and discuss his projects and Algolia solution. We start our blogcast discussing why he has chosen a career in three unique domains – education, podcasting, and engineering. We then go into his technology choices where he gives us a code-eye view into his Algolia implementation. We finished the  interview looking at his solution and suggested several ways he can enhance the overall Search experience.

Tell us about yourself and your online business

My primary background is in web development with a strong interest in front-end dev. I also spent 10+ years as a SAP ABAP developer specializing in RFC’s for API’s. In more recent years, it’s been difficult to stay away from Cloud hyper scalers, so I switched over to more of a Solution Architect background, specializing in microservices.

I design headless architectures and build enterprise applications. On my website, I use numerous headless services for data storage, CMS, emailing, authentication, search – pretty much everything. I also build my own services. Headless lets me change the underlying tech (say from Gatsby to NextJS) without changing the service-based middle and back layers.

How did you get into Education?

I started coding for a non-profit and discovered over the years that I wanted to teach and give back to the community. I also head local dev groups. Finding the right space to educate has always been the challenge. That’s why I built my website, a free and subscription-based online learning platform. In some ways, I’m compensating for what I feel was missing in my own tech education. 

My teaching tools are tutorials, full courses, and podcasts. Podcasting is a great educational and nurturing tool. My cohosts and I talk to specialists and go deep into a wide variety of technologies. We also use the podcasts as a mentoring opportunity. For example, we interviewed a fry cook who is now a Google Dev Advocate, and a high school teacher who has become a very skilled technology educator.

What role does Search play in online learning?

Search might be the most vital component for me. People come to our platform looking to learn something specific, and so they go to our search bar to find their tech. It’s as simple as that: search is central to how our student’s engage with our content.

We provide a variety of content that requires a good search bar. Search exposes our visitors to that variety and helps them build their own learning program. We have:

• 4 full courses with 21 tutorials
• 50 Podcasts/videocasts
• 38 blogs

Search shows them which combinations of materials they can use to pursue their studies. It also helps them discover our Purrfect Picks, which we add at the end of each podcast.

Before getting into your Algolia implementation, tell us more about your website’s tech

I started CodingCat.Dev with Angular, Hugo, then Gatsby, and now Next.js. I really found a career need to move towards a React background and at the time Gatsby didn’t have a great solution for mixed rendering requirements (SSR, SSG, CSR…). I also listened to some great talks happening within the Google space around web performance and how the chrome team was working directly with Vercel on Next.js, which more recently has been formalized as Aurora.

We’ve used Algolia for 5 years. Algolia has remained my only search engine throughout every technology change. I’ve tested the competitors, but they can’t beat the speed of the search and the simplicity of getting it up and running. There’s no coding required to build the search engine, it‘s 100% plug and play. 

As for our headless services, here’s a complete list:

• Firebase
• Sanity.io
• Algolia
• Cloudinary – media
• Youtube / anchor
• SocialBee.io – broadcasting and social
• Jamstack to plug in the tools
• Sendinblue for mailing
• Discord

Tell us about your Algolia search implementation

I found out about Algolia because of Firebase. I store everything (in each and every site iteration) on Firebase Cloud Firestore. Firestore doesn’t have a great full text search solution. At the time, Firebase had one solution for this and suggested Algolia. 

Algolia fits great for me because of the size of data that my site needs. Algolia has an amazing JavaScript library for client-side integration. I didn’t want to spend hours and hours on getting search set up and I found the documentation and pre-built React components ideal for what I was trying to accomplish. I have researched both Elastic and Typesense and still find Algolia to be the easiest solution.

Show us the code

Firebase has Cloud Functions that allow for “triggering” whenever data is updated within the Firestore database. I can then easily use algoliasearch Node.js.

export const onPostWriteSearch = functions.firestore
  .document('posts/{postId}')
  .onWrite((snap, context) => {
    console.log('Adding Data for', context.params.postId);
    const post = snap.after.data();

    if (!post) {
      console.log('post missing data');
      return;
    }

    if (post.type === 'lesson') {
      console.log('post type is lesson, skipping');
      return;
    }

    post.objectID = context.params.postId;

    // Write to the algolia index
    const index = client.initIndex(ALGOLIA_INDEX_NAME);
    return index.saveObject(cleanTimestamp(post));
  });

export const onPostDeleteSearch = functions.firestore
  .document('posts/{postId}')
  .onDelete((snap, context) => {
    // Write to the algolia index
    console.log('Deleting Data for', context.params.postId);
    const index = client.initIndex(ALGOLIA_INDEX_NAME);
    return index.deleteObject(context.params.postId);
  });

Next steps

Thank you, Alex, for getting together and sharing your story!

After the interview, Alex and I spoke about his Algolia solution. Similar to what Alex does with his podcasts, we like to help our guests improve their search. As his business grows, he’s eager to improve his search features, so he was glad to see what else he can do with Algolia. What follows are some of the suggestions I made. They include adding more variety to his search results (with federated search), and to start adding more personalized results and browsing functionality.

Relevance

We dug into his data and index configurations. We added new searchable attributes and changed his ranking from a “date sorting” to “ranking by relevance”. We then added a custom ranking on popularity, and created a replica to allow users to switch to “sort by date”. 

Autocomplete / Federated search

Alex was surprised to learn that we have a front-end plugin that displays multiple results from different datasources. This means he can display his tutorials, courses, and podcasts as three different result sets as his users type their queries. Our Autocomplete plug-in requires only the credentials and names of the various indices. He can brand it with his own CSS, and, of course, he can clone the library and customize it as he sees fit.

Full text searching

We spoke about how Algolia performs full text searching of large texts, which requires a different indexing strategy. The strategy is to break up the text into smaller chunks, one record per chunk. Each chunk is usually a paragraph. This enhances in-text search dramatically. I sent him an article on searching large texts, written by our CTO and this tutorial.

Indexing updates

A big challenge for Alex is to continue building his index on the fly without affecting the posting date. For example, right now when he has a post in a “draft” status, or a “publish date” in the future, his code doesn’t perform the update to remove these posts. We showed him both the DeleteObjects and PartialUpdates methods to delete or only update certain attributes. The other part of the solution is to manage the Firebase events and updates.

// Configure this to match an existing Algolia index name
  const algoliaIndex = algolia.initIndex(algoliaConfig.index);
  try {
    let result;
    if (req.body?.deleted) {
      // remove if doc removed
      result = await algoliaIndex.deleteObject(req.body?.deleted);
    } else if (req.body?.updated && !publish) {
      // remove if not published after update
      result = await algoliaIndex.deleteObject(req.body?.updated);
    } else if (publish) {
      // create when in publish state
      result = await algoliaIndex.saveObject(req.body);
    }
    console.log(result ? `Result: ${JSON.stringify(result)}` : 'Nothing Done.');
    res.status(200).end();
  } catch (error) {
    console.log(error);
    res.json({ message: 'Error check logs' });
    res.status(500).end();
  }

Algolia TalkSearch app

Another big winner. Since Alex has a lot of video and podcast content on YouTube, we suggested he use our TalkSearch tool that scrapes Youtube metadata and its transcriptions or captions, and builds an Algolia index that his users can search. TalkSearch allows his visitors to go directly to any moment of any video or podcast. He was really surprised by this and wanted to implement it right away. Along with the scraper, TalkSearch also comes with a front-end solution that he can modify to fit into his website.

Analytics, Insights, Recommendations, and Personalization

Finally, Alex saw immediately how he can make use of analytics and AI-powered recommendations and personalizations. Alex can use analytics to track success metrics, to improve his content, grow his offerings, and build a personalized search / learning experience. Algolia can build a set of recommendations to students that suggest next or related courses based on their previous testing scores and education choices.