Designing a developer friendly API client in Scala

The Algolia Scala API client is our 14th API client and was released earlier this year. As with every API client we release, we made it as easy to use as possible and want to give you some insight into our building process.

Is it developer friendly?

Have you ever struggled to use an external library or even just find some simple getting started documentation? What about a clear enough API that you don’t actually need to read the documentation?

As developers, we know how painful it can be to use external libraries, so we wanted to make the Algolia experience as easy as possible. I remember when I used Algolia for the first time; I used the ruby gem and loved the experience. It took me only 10 minutes to index and start searching some data, and this included a bundle install!

Every time we develop a new Algolia API client, our main goal is to finish with something that strongly leverages the language’s strengths and, at the same time, is easy to use and feels productive from the very first minute.

RTFM

First, the documentation. It should be easy to read, but it also should be easy to write. As we support 10+ languages, we need the documentation to be clear enough that someone can just jump in, regardless of the language. For now, we use a custom project that generates Markdown files. It’s quite simple—a template, some code in each programming language, some ifs to handle special case. This way, every time we add a feature to our API, we don’t need to completely rewrite the documentation. Furthermore, we use the custom project to generate the README.md on GitHub and the documentation on the website.

Second, the code. As with the documentation, it should be pretty straightforward for a developer to use. We want each API client to use the idioms/syntax of its programming language. But, at the same time, we want to use the same vocabulary across each of our own API clients so a developer can move seamlessly from one to another. For example, we refer to a document as an object, so this term should be used in every API client.

What principles are unique to Scala?

Scala is a very powerful language, and because of that developers are used to powerful libraries that can do a lot in just a few lines. Furthermore, with the non-blocking trend, we are used to asynchronous libraries. These two things were the core design principles we had in mind when we developed our Scala API client.

A few lines of english

One great thing about Scala is its flexibility. For example, you can omit parentheses and dots when calling method, with some rules. For example:

myObject.myMethod(myArg)

can be written as

myObject myMethod myArg

It can also be extended further.

myObject.myMethod(myArg).anotherMethod(anotherArg)

can be written as

myObject myMethod myArg anotherMethod anotherArg

With this simple functionality, and with some clever choice of words, you can have a fluent API that looks like English sentences. As you might have guessed, we used this a lot to provide a really straightforward API:

delete from "myIndex" objectId "0912093"

There are some limitation to this. First, it only works with 1 parameter methods. Second, some keywords are reserved in Scala, so you can’t use them as method names (for example: object).

The first limitation is not really a limitation as you can always find a way to get around it. If you need 2 or more parameter methods, you can find ways to express it differently in English. Another way is to use `object` for drop-in words. Let’s take this code:

clear synonyms of index "toto" and forwardToSlave

If you add the parentheses, it becomes:

clear.synonyms(of).index("toto").and(forwardToSlave)

As you can see, the methods `synonyms` and `and` take an `object` as parameters. The first one `of` is just for show, and the second one is a real parameter that will be used in the API.

The real issue is for 0 parameter methods. For example, to list all the indices of your account:

list.indices

It’s possible to write

list indices

but it’s considered unsafe.

The second limitation can be worked around with the backtick notation:

def object() = ???

does not compile, but

def `object`() = ???

does.

For example, indexing an object:

index into "toto" `object` MyObject("algolia", 1)

Don’t bore me with simple stuff

Another thing that could help developers is taking care of repeating tasks for them. For example, we decided to map as many things to object/classes as possible. The objects you index are `case class`, type safe and much easier to use. To have as much type safety as possible, a lot of parameters were transformed to case objects. This way you have the compiler working for you. For example:

sealed trait Acl

object Acl {
 case object search extends Acl
 case object browse extends Acl
}

That’s not much, but it can save a lot of time in the case of a typo. Our search is typo tolerant, not our settings. 😉

Second things second: Asynchronous

It seems as simple as we should just return `Future[_]`. But there are two things to consider:

First, users might want to specify their own `ExecutionContext`, so we added it as an implicit parameter:

(implicit executor: ExecutionContext)

But when we wanted to implement `browse`, we stumbled upon a limitation. This method allows you to get all the objects of an index. It uses a cursor, as in a SQL database. The first call returns a list of objects and a cursor, then you use this cursor to call the API again so you can get the next objects, and you repeat that until the cursor is empty.

Of course the first call to `browse` in Scala would return a `Future[Iterable[_]]`. But we need to call it multiple times to have all the results. So our final method would have returned a `Iterable[Future[Iterable[_]]`. Not nice.

Furthermore, each `Future` needs the result from the previous one to be able to run (remember the cursor thingy). And it would have been nicer to have, as a result type, something like a `Stream[_]`. It’s achievable with an `Iteratee`, but it would have needed to add a dependency. As we want to keep our API clients as small as possible, we chose not to implement it in this client. For more on `Iteratee`, see here.

What could we improve?

Of course there are always improvements to be made. For example, we would love to simplify the way batches are made (see: https://github.com/algolia/algoliasearch-client-scala/issues/67).
As of today, if you want to have a batch, you need to call the method `batch` that takes a list of operations:

batch(
 index into "indexToBrowse" `object` Test("algolia1", 10, alien = false),
 index into "indexToBrowse" `object` Test("algolia2", 10, alien = false),
 index into "indexToBrowse" `object` Test("algolia3", 10, alien = false),
 index into "indexToBrowse" `object` Test("anything", 10, alien = false)
)

Simple enough, but these pesky ending commas could be removed, and there is a lot of repetition (index into “indexToBrowse” `object`). It would be better to do something like this:

client.inIndex("indexToBrowse") { implicit index =>
 index `object` Test("algolia1", 10, alien = false)
 index `object` Test("algolia2", 10, alien = false),
 index `object` Test("algolia3", 10, alien = false),
 index `object` Test("anything", 10, alien = false)
}

This feature is planned, but we are not sure if this new syntax is needed.

Long story short, I tried to keep in mind the ease of use while developing the Scala API client. Hopefully you have some better insight into this process now. I would love to hear your feedback and any suggestions you have for improving it. Feel free to leave a comment or open a GitHub issue to contact me directly.