Search by Algolia
An introduction to transformer models in neural networks and machine learning
ai

An introduction to transformer models in neural networks and machine learning

What do OpenAI and DeepMind have in common? Give up? These innovative organizations both utilize technology known as transformer models ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What’s the secret of online merchandise management? Giving store merchandisers the right tools
e-commerce

What’s the secret of online merchandise management? Giving store merchandisers the right tools

As a successful in-store boutique manager in 1994, you might have had your merchandisers adorn your street-facing storefront ...

Catherine Dee

Search and Discovery writer

New features and capabilities in Algolia InstantSearch
engineering

New features and capabilities in Algolia InstantSearch

At Algolia, our business is more than search and discovery, it’s the continuous improvement of site search. If you ...

Haroen Viaene

JavaScript Library Developer

Feature Spotlight: Analytics
product

Feature Spotlight: Analytics

Analytics brings math and data into the otherwise very subjective world of ecommerce. It helps companies quantify how well their ...

Jaden Baptista

Technical Writer

What is clustering?
ai

What is clustering?

Amid all the momentous developments in the generative AI data space, are you a data scientist struggling to make sense ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

What is a vector database?
product

What is a vector database?

Fashion ideas for guest aunt informal summer wedding Funny movie to get my bored high-schoolers off their addictive gaming ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Unlock the power of image-based recommendation with Algolia’s LookingSimilar
engineering

Unlock the power of image-based recommendation with Algolia’s LookingSimilar

Imagine you're visiting an online art gallery and a specific painting catches your eye. You'd like to find ...

Raed Chammam

Senior Software Engineer

Empowering Change: Algolia's Global Giving Days Impact Report
algolia

Empowering Change: Algolia's Global Giving Days Impact Report

At Algolia, our commitment to making a positive impact extends far beyond the digital landscape. We believe in the power ...

Amy Ciba

Senior Manager, People Success

Retail personalization: Give your ecommerce customers the tailored shopping experiences they expect and deserve
e-commerce

Retail personalization: Give your ecommerce customers the tailored shopping experiences they expect and deserve

In today’s post-pandemic-yet-still-super-competitive retail landscape, gaining, keeping, and converting ecommerce customers is no easy ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Algolia x eTail | A busy few days in Boston
algolia

Algolia x eTail | A busy few days in Boston

There are few atmospheres as unique as that of a conference exhibit hall: the air always filled with an indescribable ...

Marissa Wharton

Marketing Content Manager

What are vectors and how do they apply to machine learning?
ai

What are vectors and how do they apply to machine learning?

To consider the question of what vectors are, it helps to be a mathematician, or at least someone who’s ...

Catherine Dee

Search and Discovery writer

Why imports are important in JS
engineering

Why imports are important in JS

My first foray into programming was writing Python on a Raspberry Pi to flicker some LED lights — it wasn’t ...

Jaden Baptista

Technical Writer

What is ecommerce? The complete guide
e-commerce

What is ecommerce? The complete guide

How well do you know the world of modern ecommerce?  With retail ecommerce sales having exceeded $5.7 trillion worldwide ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Data is king: The role of data capture and integrity in embracing AI
ai

Data is king: The role of data capture and integrity in embracing AI

In a world of artificial intelligence (AI), data serves as the foundation for machine learning (ML) models to identify trends ...

Alexandra Anghel

Director of AI Engineering

What are data privacy and data security? Why are they  critical for an organization?
product

What are data privacy and data security? Why are they critical for an organization?

Imagine you’re a leading healthcare provider that performs extensive data collection as part of your patient management. You’re ...

Catherine Dee

Search and Discovery writer

Achieving digital excellence: Algolia's insights from the GDS Retail Digital Summit
e-commerce

Achieving digital excellence: Algolia's insights from the GDS Retail Digital Summit

In an era where customer experience reigns supreme, achieving digital excellence is a worthy goal for retail leaders. But what ...

Marissa Wharton

Marketing Content Manager

AI at scale: Managing ML models over time & across use cases
ai

AI at scale: Managing ML models over time & across use cases

Just a few years ago it would have required considerable resources to build a new AI service from scratch. Of ...

Benoit Perrot

VP, Engineering

How continuous learning lets machine learning  provide increasingly accurate predictions and recommendations
ai

How continuous learning lets machine learning provide increasingly accurate predictions and recommendations

What new data points have you learned lately? Learning is never ending (hence the phrase “lifelong learning”), so chances are ...

Vincent Caruana

Sr. SEO Web Digital Marketing Manager

Looking for something?

facebookfacebooklinkedinlinkedintwittertwittermailmail

In the previous article of this series, we talked about how unit tests can help us test our functions easily, as well as how getting into the mindset of using unit tests can help us write purer, simpler, and more modular functions in the first place. But what happens when we need to test a system that doesn’t fit in a function? What if we can’t depend on a function’s output to tell us if it’s working? What if our function depends on state set by the UI? In these cases, it’s difficult to accurately model the complexity of how the user will navigate our application — if there isn’t a useful abstraction or simplification that we can test on, we might just have to spin up a live instance of the site and pretend to be a user.

Surely we can’t automate that process, right? It seems too manual and human-brain-dependent to do that. As it actually turns out, the solution lies not in automating anything inside your application, but in automating the input to the browser, simulating your mouse and keyboard so that the application that you’re testing can’t tell the simulator apart from a real human. This process is called browser automation. And what better tool to automate a browser than a headless version of the browser itself — the Chromium project maintains an API called Puppeteer that we’re going to use for this.

Puppeteer — a headless browser

The first step is to get our site up and running with a dev instance, since we’ll want to test these things before we go to production. Since we’re just demonstrating here, I’m going to run this test on Algolia’s production homepage.

Next, we’ll set up a Node.JS program. Choose a logical location for this (probably a single folder inside the root of your project so that you can integrate it with your build process). Inside this folder, you’ll want to create an index.js file that calls all of the tests. This way, just running node test from the root directory of the repo will run all of your tests, and you can bury down into the folder to run more specific ones.

Now, there are some situations where it might make sense for you to make that a post-build command at the end of the build process, but these are going to be more sophisticated tests that you’ll probably want to run manually, so if you’re developing and testing locally, it’s probably best to just leave it there and let your devs manually run node test or node test/specific-test.js when they actually want to run the Puppeteer tests. If your application has a more complex build process that involves less testing locally and more dev instances in the cloud, it might be easier to set up a serverless project (something both Vercel and Netlify excels at), set it to read the tests as lambda functions, and then modify the tests to run when you ping that endpoint. The tests don’t actually need to be run from the same place as the application (since the tests are full-on clients), so you could make a little GUI with some buttons to trigger each test in the cloud.

After running npm install puppeteer, you’ll need some boilerplate code. Here’s the structure around a sample test that’ll log into Algolia as me and get the credentials of some application:

const puppeteer = require('puppeteer');

const login = async ({width, height}) => {
	const browser = await puppeteer.launch();
	const page = await browser.newPage();
	await page.goto('<https://algolia.com>');
	await page.setViewport({
		width,
		height,
		deviceScaleFactor: 1
	});

	// test logic here

	await browser.close();
};

module.exports = async () => {
	login({
		width: 375,
		height: 667
	});
	login({
		width: 1150,
		height: 678
	});
};;

A quick rundown:

  1. The first line just imports Puppeteer. Puppeteer is run by the Chromium DevTools team, so it’s essentially a native part of that ecosystem. Importing it here links in a headless version of Chromium (that’s a version that just pretends that it has a GUI), and because it was developed in tandem with Chromium, that version is guaranteed to work with Puppeteer.
  2. The first function is called login and that runs our test. We don’t have any test logic in here right now, just the boilerplate that launches the headless Chrome browser, goes to the Algolia website, and sets the viewport to whatever we passed into the function. At the end, we close the browser so it doesn’t stay open in memory.
  3. Then, we export a function that runs the same test on different screen sizes.

To run this test from inside index.js in the test folder, we only need to run require("./login")();. With this one-line-per-test setup, we can even group the tests into batches and trigger those batches with flags on the command line, e.g. node test --dashboard. The sky is the limit with the customization here, so tweak it to the needs of your application.

In my case, I’m going to throw errors whenever the tests fail and not worry about batching anything — I don’t have enough tests for it to be meaningful to focus my effort there. In your application though, it might be helpful for these test functions to return the result of the test instead of just throwing errors when they fail, as that’ll give you the ability to print out more detailed messages based on the tests run from index.js. Right now, I’m aiming to make everything run silently unless there’s a problem that we need to fix.

Let’s dive a bit deeper into the actual test logic, which is arguably simpler than what you might expect:

// step 0. accept cookies so the site works like normal
await page.waitForSelector('a[href="/policies/cookies/"] + button');
await page.click('a[href="/policies/cookies/"] + button');

// step 1. wait for the log in link to appear and click on it
if (width < 960) {
	await page.waitForSelector('[data-mobile-menu-button]');
	await page.click('[data-mobile-menu-button]');
}
await page.waitForSelector('a[href="/users/sign_in"]');
await page.click('a[href="/users/sign_in"]');

// step 2. wait for the email input to appear on the new page, focus on it, and type the email from .env
await page.waitForSelector('input[type=email]');
await page.focus('input[type=email]')
await page.keyboard.type(process.env.email);

// step 3. focus on the password input, and type the password from .env
await page.focus('input[type=password]')
await page.keyboard.type(process.env.password);

// step 4. click the log in button
await page.click("form#new_user button[type=submit]");

// step 5. if we're on mobile, we'll need to click one more button to indicate we'd like to stay on mobile
if ((await page.url()).includes("mobile")) {
	await page.waitForSelector('.continue-button');
	await page.click('.continue-button');
}

// step 6. wait for the application selector to appear and click it
await page.waitForSelector('#application-select');
await page.click("#application-select");

// step 7. click on the application with the name in .env
await page.$$eval(
	'div > span.options',
	(options, applicationName) => {
		const matchedOptions = options.filter(option => option.innerText.split("\\n")[1] == applicationName);
		if (matchedOptions.length) {
			matchedOptions[0].click();
		} else {
			console.log(`Application ${applicationName} does not exist`);
		}
	},
	process.env.applicationName
);

// step 8. wait for the api keys button to appear on the new page, and click it
await page.waitForSelector('#overview-api-keys-link');
await page.click("#overview-api-keys-link");

// step 9. wait for the page to load and get the application id and public api key for this project
await page.waitForSelector('#api-keys-page-heading');
const [applicationID, publicAPIKey] = await page.$$eval(
	'input[readonly]',
	inputs => inputs.slice(0, 2).map(input => input.value)
);

// step 10. log the results of our test
if (applicationID != process.env.applicationID)
	throw `Application ID is not correct. Test produced "${applicationID}", but it should have been "${process.env.applicationID}"`;
if (publicAPIKey != process.env.publicAPIKey)
	throw `Public API key is not correct. Test produced "${publicAPIKey}", but it should have been "${process.env.publicAPIKey}"`;

// step 11. close the browser
await browser.close();

This isn’t a trivial example — if you read the comments above each step of the process, none of this should be too surprising. It’s very literally just mimicking the user’s behavior in a real browser, taking into account the quirks of how Algolia’s homepage handles mobile differently from desktop. All that goes along with this is an .env file where I’ve defined the email and password to log in with, the name of the intended application (which is what the user would use to find the right application), and the publicAPIKey and applicationID which the test should return. If all goes well, this outputs nothing. If something breaks though, it’ll throw a descriptive error so that we can jump in to fix it before the build finishes.

So what have we learned in this series? Well, there are two main types of testing:

  1. Unit testing(as discussed in part 1 of this series) this is perfect for simple, pure functions that don’t interact with the outside world. It’s good practice to isolate even the pure parts of otherwise impure functions so that you can test them individually with simple assertions baked right into the same file as the code that you’re testing.
  2. Browser emulation — for larger workflows, you’ll need your test to actually pretend to be a user, so we boot up the headless version of Chromium called Puppeteer to “puppet” a fake user and walk through the entire process, with the browser not even knowing that it’s interacting with another program.

It should be noted that these methods aren’t mutually exclusive — this particular functionality of displaying the correct API credentials to the developer in the Algolia dashboard is important enough to test with both methods. We’ve written a Puppeteer program to walk through the entire workflow from the user’s perspective, but in addition, it would be helpful to break down the hydration process of that API keys page that we scraped. If we isolated as many pure functions as possible in that page, we could test them with unit tests to ensure that the vast majority of that process never silently breaks. Since it’s pulling data from the database to display in the view, it’s probably not possible to make the whole thing pure, but that’s the benefit of using both types of testing: we don’t have to get 100% coverage with either method. As long as they cover everything together, we can be confident in our application’s durability and long-term sustainability. Happy testing!

About the author
Jaden Baptista

Technical Writer

Recommended Articles

Powered byAlgolia Algolia Recommend

Resilience testing in production: test as you deploy
engineering

Xavier Grand

Engineering Manager

How to improve site search with Algolia A/B testing
e-commerce

Loise Mercier

How to send engaging customer emails with Twilio SendGrid and Algolia Recommend
engineering

Chuck Meyer

Sr. Developer Relations Engineer