Pragmatic Releasing: Less Worry, More Shipping

Here I’ll share a few things I’ve learned managing libraries at Algolia. They are by no means a perfect set of instructions or recommendations, but tips for making your (developer) life easier.

Here are some good signs that you may want to re-work your release workflow:

• You tend to batch features before releasing them
• You wait as long as possible before implementing something although you know well it will eventually need to be done
• You are stressed when releasing
• You’d rather leave bugs in your code than have to release
• You only release when forced to, and in general it puts you in a bad mood

Don’t rely on a single person to release

When I was just about six months into my job at Algolia, a bug inside one of our JavaScript libraries impacted customer search implementations in production. We realized that one of the latest changes committed to the repository was having unexpected side effects, which led to always displaying empty search results. Unfortunately, the one person who knew how to release the library had just left the office and was commuting home. Thankfully, they still managed to point us at a document detailing the release process of the library. We were relieved.

The team’s action plan was to:

• Revert the latest code changes introduced during the day
• Quickly release a patched version
• Calmly investigate the issue we had pushed while clients enjoy a working search

The first step was easy to do given we had used Git as the source control system for the project. We just had to revert some of the latest commits. For the second point though, the procedure included many steps.

Even though having a lot of steps is not a problem in its own right, what was a problem is that a lot of questions came up: 

• Will I be able to authenticate to the npm registry?
• Am I even part of the organization owning the library on npm?
• How do I propagate the change to the CDNs so that clients can have their bug fixed as fast as possible?
• I thought that the CDN update was now automated, why do we need to do that part?

Even if I was asking myself only one of the above questions, I would have started doubting myself and feeling uncomfortable about the entire process. Asking myself all four was…too much.

I learned two things that day:

1. 1. Any developer in the company should be able to release any given project. Put differently: relying on a single person to deploy/release a project can be dangerous.
3. 2. To be able to have anyone deploy a project, they must be comfortable doing so. Comfortable means that no questions are left unanswered in the release process, and that the release process itself is simple enough to be actionable.

To make sure a fair amount of engineers in your company can deploy a project, the best solution I can think of is a single bash script which would guide them through the publishing steps.

Some companies like Etsy take this quite seriously and have their new employees release something to production from day one in the company

Next time you publish a project, here are a couple of questions you could ask yourself to evaluate the quality of your release workflow:

• Could this be released by someone else than myself?
• If yes, will they have to reach out to me to fix the build?

Automation is key

I recommend automating the entire release process as much as possible, which includes taking care of those tiny little things you think are not worth automating.

For example, you should not have to replace the version number in any file manually, the reason being that one day you will forget about replacing one version number, or one day it won’t be you releasing. That day, you will probably lose a lot of time and possibly negatively impact production environments. Furthermore, chances are that the time you will lose that day is more than the time it takes to automate your release process today.

A good practice is to aim for a single command line you can execute to get your software released. Ideally the command should be interactive and guide you in the process by asking some questions like::

• Do you want to release a beta or production version?
• Here are the current changes, which one would you like to push as a new version?
• Here’s the previous changelog and version, given we use SemVer and the selected changes, what should be the next version?
• Do you also want to release the documentation website?

Here’s an example including some of those questions:

Make sure your can release fast

In almost every project I work on, there is some kind of a continuous integration setup.

On every new commit pushed to the repository under source control, all tests are run in a single environment or in multiple ones. This has the advantage of making sure future releases are working correctly on the targeted platforms, but has the drawback of slowing down the time needed for the release to get out.

Because implementing pragmatic releasing is an iterative process, some projects I’m working on are still taking up to 45 minutes to have all tests pass. This mainly happens when the project has many integrations and end-to-end tests including relying on calls to an external API, Algolia in my case.

Having long running tests like these can be a real bottleneck for productivity. I would personally tend to avoid having to add features to those projects, because I know it is going to be time consuming.

Just to give you an idea, here is what the process of adding a single feature to such a repository looks like:

• Push the changes
• Do something else during more or less 45 minutes
• Go back to the project, and eventually realize that a test failed on a given platform
• Push a fix to support the platform and wait again for 45
• Eventually remember that you had pushed a given feature to the repository and ask for reviewers to approve the code
• Redo steps 2 & 3 if any feedback has been given by any reviewer
• Merge the changes
• Release the project with newly integrated changes

In a best-case scenario, it takes about an hour to release even the simplest possible feature. If you are unlucky, though, you could spend a day working on getting a feature out. Now imagine this feature is an actual bug fix impacting production environments. It would be totally unacceptable to have to wait one full day to get the patch out.

Ideally, the time to release a new version should be equivalent to the time to implement the feature and get it reviewed. A few ways to help accomplish this:

• Reducing as much as possible any kind of a long running process directly impacting the speed of releasing new features or bug fixes.
• If there are end-to-end tests relying on external APIs in place, caching API responses, and making tests run on mocked calls.
• If you often have to reject PRs because the format of the commit is incorrect, have your CI platform validate the format for you. If your CI platform is slow, it is worth investigating that issue as well.

Whatever the issue might be, taking some time to speed up to release cycle is key to fast iterations.

Be confident in your code

In the previous paragraph, I shared how much I think speed of releasing is important for a project. One thing that is equally important is the quality of the builds you are shipping.

The moment you get something released, you have to be confident it achieves what it was designed for. In other words, you should test your code to ensure business expectations are met. There should be no way to release a project that has failing tests.

However, I think that it is also perfectly fine to ship a “work in progress” feature as long as it doesn’t impact other features. If you have a chance to break down a big feature into many smaller ones, you’ll be able to iterate faster because the review will be easier.

A couple of ideas to challenge your existing release workflow

Have someone else do your next release

Next time you are about to release your library, ask a colleague who knows nothing about your project to deploy it. Give them the URL of your repository as the only instruction. If they succeed in releasing, it probably means the quality of your release cycle is not bad.

Iterate faster, deploy more

Next time you have an incoming task that is easy to address, get it done right away and force yourself to release the change. If you feel like you would have preferred to open an issue and deal with it later instead, it probably means you can still optimize your release workflow.

The benefits of releasing often

Improve your mental well-being

When you reach the stage where you can release on demand, you address issues differently. Instead of polluting the repository with issues that would distress and distract you, you get things done instead. Plus, given other teammates are able to release the project without your help, you can live without the fear of having to remotely guide a stressed out coworker on your day off. 

Improve your productivity

When you have a robust and simple release script, you can release when you see fit. The mental gap between release intention and the actual release should be negligible.

Be more reactive

In the “Don’t rely on a single person to release” section, I shared a real story about a project that impacted real production environments. Between the initial report and the actual fixed release we lost about an hour of time.  By implementing a release process that follows the principles shared here, the time to release the bug fix is now equal to the time necessary to implement the bug fix. In most cases, this would be a couple of minutes if you are using a source control system like Git: just the time to revert the changes and release again.

Tooling to create better release workflows

Here’s how I like to design the release workflow as of today:

Use conventional commits

Conventional commits dictates a format which every single commit of your repository should follow. By doing so, you’ll be able to:

• Automate the generation of your CHANGELOG file at every new release. Conventional Changelog is a good tool for this.
• Make it easy to see what has been done since last release by dumping out unreleased work.
• Make sure your CHANGELOG never misses a single entry and that the formatting stays consistent.

I would recommend you use conventional commits to avoid the burden of manually having to update the CHANGELOG file.

I would also suggest you have your CI platform test all commits to ensure a badly formatted commit cannot be committed to your master branch. You could use something like commitlint to ensure the format is correct.

Create a release-pr script

Create a script (in bash for example) that will do the following:

• Check that you are currently on master
• Check that the working tree is clean
• Install the dependencies
• Run the tests
• Ask for the new version of the project after having dumped the unreleased changes
• Update all the version numbers in all the files where it appears
• Push a new branch `chore(release): ${VERSION}`
• Wait for a teammate’s approval
• Merge the branch

Here is an example of such a bash releasing script:

Create a publishing script (manual)

Create a script (in bash for example) that will do the following:

• Check that you are currently on master
• Check that the working tree is clean
• Install the dependencies
• Run the tests
• Push the new version to a public listing if required (e.g., npm for the JavaScript example below)
• Tag the current commit with Git with the current version, and push the tag to the remote repository

Here is an example of such a bash publishing script:

Create a publishing script (Continuous Delivery)

Previous step assumes you manually check out the changes after you have merged the release branch, and then run the publishing script. You could also let your CI platform handle this publishing for you every time something gets merged into your master branch. Personally, I like to keep this publishing task manual, because if something goes wrong, you are able to fix it easily.

Final word

Releasing software should be something you enjoy doing: each time you release, you either fix a bug or introduce awesome new features. Spending some time to optimize your release workflow helps delivering better quality faster.

I would love to hear what you do to make your release flow less stressful and more fun: @rayrutjes.