Search by Algolia
How to increase your ecommerce conversion rate in 2024
e-commerce

How to increase your ecommerce conversion rate in 2024

2%. That’s the average conversion rate for an online store. Unless you’re performing at Amazon’s promoted products ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

How does a vector database work? A quick tutorial
ai

How does a vector database work? A quick tutorial

What’s a vector database? And how different is it than a regular-old traditional relational database? If you’re ...

Catherine Dee

Search and Discovery writer

Removing outliers for A/B search tests
engineering

Removing outliers for A/B search tests

How do you measure the success of a new feature? How do you test the impact? There are different ways ...

Christopher Hawke

Senior Software Engineer

Easily integrate Algolia into native apps with FlutterFlow
engineering

Easily integrate Algolia into native apps with FlutterFlow

Algolia's advanced search capabilities pair seamlessly with iOS or Android Apps when using FlutterFlow. App development and search design ...

Chuck Meyer

Sr. Developer Relations Engineer

Algolia's search propels 1,000s of retailers to Black Friday success
e-commerce

Algolia's search propels 1,000s of retailers to Black Friday success

In the midst of the Black Friday shopping frenzy, Algolia soared to new heights, setting new records and delivering an ...

Bernadette Nixon

Chief Executive Officer and Board Member at Algolia

Generative AI’s impact on the ecommerce industry
ai

Generative AI’s impact on the ecommerce industry

When was your last online shopping trip, and how did it go? For consumers, it’s becoming arguably tougher to ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

What’s the average ecommerce conversion rate and how does yours compare?
e-commerce

What’s the average ecommerce conversion rate and how does yours compare?

Have you put your blood, sweat, and tears into perfecting your online store, only to see your conversion rates stuck ...

Vincent Caruana

Senior Digital Marketing Manager, SEO

What are AI chatbots, how do they work, and how have they impacted ecommerce?
ai

What are AI chatbots, how do they work, and how have they impacted ecommerce?

“Hello, how can I help you today?”  This has to be the most tired, but nevertheless tried-and-true ...

Catherine Dee

Search and Discovery writer

Algolia named a leader in IDC MarketScape
algolia

Algolia named a leader in IDC MarketScape

We are proud to announce that Algolia was named a leader in the IDC Marketscape in the Worldwide General-Purpose ...

John Stewart

VP Corporate Marketing

Mastering the channel shift: How leading distributors provide excellent online buying experiences
e-commerce

Mastering the channel shift: How leading distributors provide excellent online buying experiences

Twice a year, B2B Online brings together America’s leading manufacturers and distributors to uncover learnings and industry trends. This ...

Jack Moberger

Director, Sales Enablement & B2B Practice Leader

Large language models (LLMs) vs generative AI: what’s the difference?
ai

Large language models (LLMs) vs generative AI: what’s the difference?

Generative AI and large language models (LLMs). These two cutting-edge AI technologies sound like totally different, incomparable things. One ...

Catherine Dee

Search and Discovery writer

What is generative AI and how does it work?
ai

What is generative AI and how does it work?

ChatGPT, Bing, Bard, YouChat, DALL-E, Jasper…chances are good you’re leveraging some version of generative artificial intelligence on ...

Catherine Dee

Search and Discovery writer

Feature Spotlight: Query Suggestions
product

Feature Spotlight: Query Suggestions

Your users are spoiled. They’re used to Google’s refined and convenient search interface, so they have high expectations ...

Jaden Baptista

Technical Writer

What does it take to build and train a large language model? An introduction
ai

What does it take to build and train a large language model? An introduction

Imagine if, as your final exam for a computer science class, you had to create a real-world large language ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

The pros and cons of AI language models
ai

The pros and cons of AI language models

What do you think of the OpenAI ChatGPT app and AI language models? There’s lots going on: GPT-3 ...

Catherine Dee

Search and Discovery writer

How AI is transforming merchandising from reactive to proactive
e-commerce

How AI is transforming merchandising from reactive to proactive

In the fast-paced and dynamic realm of digital merchandising, being reactive to customer trends has been the norm. In ...

Lorna Rivera

Staff User Researcher

Top examples of some of the best large language models out there
ai

Top examples of some of the best large language models out there

You’re at a dinner party when the conversation takes a computer-science-y turn. Have you tried ChatGPT? What ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What are large language models?
ai

What are large language models?

It’s the era of Big Data, and super-sized language models are the latest stars. When it comes to ...

Catherine Dee

Search and Discovery writer

Looking for something?

facebookfacebooklinkedinlinkedintwittertwittermailmail

Most of our users are technical. They love writing code, and we love providing API clients in the major programming languages to them (we are currently supporting 10 platforms).

They are doers. They love prototyping. Just like us, they work for startups which need to move fast, and get things done, keeping in mind that done is better than perfect. It is very important that they don’t want to waste time. In this post, I will explain how one would have used our API up to now, and how we introduced SQL and MongoDB connectors for easier onboarding, integration and testing.

Before: The first steps with our API

Up until now, our onboarding process asked you to try the API by uploading your data. We emphasized our documentation, and we made sure our users would not need more than a few minutes to integrate our REST API. Nevertheless, exporting your application’s data to a JSON or CSV file is often more complex than it appears, especially when you have millions of rows – and especially because developers are lazy 🙂 No worries, that’s totally OK. It is something you may not be willing to do, especially just to try a service, so we decided to try something else.

Initial import

90% of our users are using a SQL or MongoDB database. Exporting a table or a collection to a JSON file can be easy if you’re using a framework, for example Ruby on Rails:

File.open("/tmp/export.json", "w") do |f|
  f << MyActiveRecordModel.all.to_json
end

…or more annoying, for example when using PHP without any framework:

mysql_connect('localhost', 'mysql_user', 'mysql_password');
mysql_set_charset('utf8');
$results = array();
$q = mysql_query("SELECT * FROM YourTable");
if ($q) {
  while (($row = mysql_fetch_assoc($q))) {
    array_push($results, $row);
  }
}
$fp = fopen('/tmp/export.json', 'w');
fwrite($fp, json_encode($results));
fclose($fp);

Anyway, in both cases it gets harder if you want to export millions of rows without consuming hundreds GB of RAM. So you will need to use our API clients:

index = Algolia::Index.new "YourIndex"
MyActiveRecordModel.find_in_batches(1000) do |objects|
  index.add_objects(objects)
end
# that's actually what `MyActiveRecordModel.reindex!` does
mysql_connect('localhost', 'mysql_user', 'mysql_password');
mysql_set_charset('utf8');
$limit = 1000;
$start = 0;
$index = $client->initIndex('YourIndexName');
while (true) {
  $q = mysql_query("SELECT * FROM YourTable LIMIT " . $start . "," . $limit);
  $n = 0;
  if ($q) {
    $objects = array();
    while(($row = mysql_fetch_assoc($q))) {
      array_push($objects, $row);
      ++$n;
    }
    $index->addObjects($objects);
  }
  if ($n != $limit) {
    break;
  }
  $start += $n;
}

Incremental updates

Once imported, you will need to go further and keep your DB and our indexes up-to-date. You can either:

  • Clear your index and re-import all your records hourly/daily with the previous methods:
    • non-intrusive,
    • not real-time,
    • not durable,
    • need to import your data to a temporary index + replace the original one atomically once imported if you want to keep your service running while re-importing

Or

  • Patch your application/website code to replicate every add/delete/update operations to our API:
    • real-time,
    • consistent & durable,
    • a little intrusive to some people, even though it is only a few lines of code (see our documentation)

After: Introducing connectors

Even if we did recommend you to modify your application code to replicate all add/delete/update operations from your DB to our API, this should not be the only option, especially to test Algolia. Users want to be convinced before modifying anything in their production-ready application/website. This is why we are really proud to release 2 open-source connectors: a non-intrusive and efficient way to synchronize your current SQL or MongoDB database with our servers.

SQL connector

title=”Algolia SQL JDBC Connector”>algolia/jdbc-java-connector (MIT license, we love pull-requests :))

The connector starts by enumerating the table and push all matching rows to our server. If you store the last modification date of a row in a field, you can use it in order to send all detected updates every 10 seconds. Every 5 minutes, the connector synchronizes your database with the index by adding the new rows and removing the deleted ones.

jdbc-connector.sh --source "jdbc:mysql://localhost/YourDB"  
  --username mysqlUser --password mysqlPassword             
  --selectQuery "SELECT * FROM YourTable" --primaryField id 
  --updateQuery "SELECT * FROM YourTable WHERE updated_at > _$"
  --updatedAtField updated_at 
  --applicationId YourApplicationId --apiKey YourApiKey --index YourIndexName

If you don’t have an updated_at  field, you can use:

jdbc-connector.sh --source "jdbc:mysql://localhost/YourDB"  
  --username mysqlUser --password mysqlPassword             
  --selectQuery "SELECT * FROM YourTable" --primaryField id 
  --applicationId YourApplicationId --apiKey YourApiKey --index YourIndexName

The full list of features is available on Github (remember, we ♥ feature and pull-requests)!

MongoDB connector

 Github project: algolia/mongo-connector

This connector has been forked from 10gen-lab’s official connector and is based on MongoDB’s operation logs. This means you will need to start your mongod  server specifying a replica set. Basically, you need to start your server with: mongod –replSet REPLICA_SET_IDENTIFIER. Once started, the connector will replicate each addition/deletion/update to our server, sending a batch of operations every 10 seconds.

mongo-connector -m localhost:27017 -n myDb.myCollection 
  -d ./doc_managers/algolia_doc_manager.py              
  -t YourApplicationID:YourApiKey:YourIndex

The full features list is available on Github (we ♥ feature and pull-requests).

Conclusion: Easier Onboarding, Larger Audience!

Helping our users to onboard and try Algolia without writing a single line of code is not only a way to attract more non-technical users; It is also a way to save the time of our technical but overbooked users, allowing them to be convinced without wasting their time before really implementing it.

Those connectors are open-source and we will continue to improve them based on your feedback. Your feature requests are welcome!

About the author
Sylvain Utard

VP of Engineering

linkedintwitter

Algolia documentation

It's extensive, clear, and, of course, searchable.

Read the docs
Algolia documentation

Recommended Articles

Powered byAlgolia Algolia Recommend

Introducing the new Algolia no-code data connector platform
engineering

Keshia Rose

Staff Product Manager, Data Connectivity

Keeping Data in your Search Engine Up-to-Date
engineering

Julien Lemoine

Co-founder & former CTO at Algolia

Part 2: Supercharging search for ecommerce solutions with Algolia and MongoDB — Proposed solution and design
engineering

Soma Osvay

Full Stack Engineer, Starschema