This year a bunch of our engineers from our Sydney office attended GopherCon AU at University of Technology, Sydney, in the city centre. There was a lot going on: many interesting talks and side convos. Notably it also turned out to be great to get to spend some time together as a team and throw some ideas around.

The organisers did an amazing job of attracting lots of international speakers, as well as bringing in some locals. Russ Cox (currently leading the Go project), and Rob Pike (co-creator of the Go language) were highlights, Bill Kennedy (Ardan Labs) also ran one of the pre-conf workshops.

It was also interesting to see how deeply the Go language is involved in search technology! Below, we’ve summarized some of the talks we attended to share the latest on Golang.

Testing (Russ Cox)

There’s always a lot to be learned from reading how the Go std lib + tooling handles various engineering challenges. Back in the early days of Go it was the only “big” project that you could lean on for inspiration, but even now it continues to deliver interesting ideas and techniques for making programs simpler to manage and iterate on.

A big theme here was the reminder that “software engineering is managing programming + time”, and how testing is central to making this work at scale. Spending time on improving testing pays dividends: making tests clear and easy to read and debug, but also easy to write is key to ensuring smooth iteration of the system whilst it passes between different engineers and even different teams.

There were lots of small nuggets of information and ideas, but one highlight was a reminder that you can write code to make things easier for yourself. A great example of this was table tests, which have become a staple of Go testing. Often tables can be cumbersome to maintain, and a lot of syntax can obscure the checks being run — so why not break out and use a simple txt file to store test cases and output.

Examples included writing tests for binaries (which has been recently extracted into a separate repo):

• GitHub – rsc/script: Script engine, especially good for tests (Test scripts in Go — Bitfield Consulting )
• And another simple testing tool used to test the Go website: webtest package – golang.org/x/website/internal/webtest – Go Packages (and an example test: https://cs.opensource.google/go/x/website/+/master:cmd/golangorg/testdata/blog.txt).

Russ has since posted a re-recoding of this talk on YouTube: Go Testing By Example (GopherCon Australia 2023)

Constraining Complexity in the Generics Design (Axel Wagner)

Having already stumbled across some of the limitations of the generics implementation in Go, it was great to have a full run through of some of the underlying issues that prevent us from being able to represent some constructs in the current version of Go type parameters. In particular the mixing of primitive types and interfaces was very interesting, and maybe speaks to a bigger issue with using interfaces in the type parameter description in general. However, it was also good to see that some changes are possible here which should allow a path forward to slowly integrating some interface types — we could see mixing in unions in the future!

Axel is one of the few non-Googlers that’s very involved in discussions on the Go project, so it was great for our team to be able to talk to him in person.

Coroutines and Go (Raghav Roy)

There are several new articles and proposals around co-routines (research!rsc: Storing Data in Control Flow , research!rsc: Coroutines for Go ) and so it was great to run through this in a talk. Not sure the speaker knew that Russ Cox (who is currently leading the Go project) would be in the audience when he submitted the abstract — but it was very interesting none-the-less! Some of the new approaches look a bit alien when compared to the simpler Go constructs that we are used to dealing with, so it was good to pick apart the details and go through trade-offs in the proposed implementations.

Building a Beginner Programming Language with Go (Julia Ogris)

Julia identified a gap between visual programming languages meant for beginners like Scratch and professional programming languages like Python. Her daughters had built things in Scratch but had a strong appetite to write ‘real code’ (i.e. text) but faced many hurdles with Python. And so Evy was born.

The talk was a really interesting dive into the implementation and design of Evy. It was interesting how the parser was mostly so simple (implementing the rules of precedence for operators added a lot of complexity!) and this was a really good example of a problem where the solution was to write a programming language, and indeed an entire programming environment!

When we look at programming languages we tend to think of them as impossibly huge feats of engineering, but like with Russ’ talk where he made the case for implementing effectively tiny little languages with trivial parsers to aid writing tests, this talk showed what can be achieved if we challenge that idea.

Dog Food Can Be Delicious (Jeff Levin)

Jeff’s talk walked us through his adventure in trying to add load testing as an acceptance test to the Grafana release process. The Go related content was light but the more interesting aspect was the cultural and social roadblocks (or organizational friction as he referred to it) encountered.

The stories were all very relatable:

• “Have you pulled the main branch? A huge change just went in”
• Begging on Slack at midnight for someone to help build a dashboard in Grafana
• Exasperated conversations with his manager around various frustrations to do a seemingly simple task
• Trying to connect with and motivate colleagues in a remote-first and very distributed organization
• A huge amount of grit to pursue an endeavour that nobody else seemed to care about, even though the potential return on invest was massive
• Reaching out to the K6 team to request some new features. Context: K6 was just a small company recently acquired by Grafana, a rather large, geographically distributed company
• After finally creating the new load testing service, he then handed it off to another team to own and run

There were other stories, but the main takeaway was that pure technical brilliance isn’t enough, there’s a bit of luck and also having the right relationships and finding the right people to convince your thing is worthwhile — sell people on how they win if you succeed.

The Power of Bloom Filters: Building a Cutting Edge Go Search Engine to Explore the World’s Source (Ben Boyter)

Ben talked about scaling his code search engine (searches over 75 billion lines of code) on a single server using some compression tricks originating from bloom filters (his presentation is shared here). He expanded on some of the techniques inspired by BitFunnel to further compress and make search iteration faster. This work showed the index sizes could be orders of magnitude smaller than what would be expected from standard search indexes without a loss of quality.

The talk showed the ever present trade offs required to deliver search technology at scale! By the end of the conference multiple people from the audience had approached him and contributed code improvements which was fantastic to see.