Low friction development environments

While setting up a sample project from an unnamed large vendor the other week I was disappointed by having to read large amounts of documentation and run various bits of script to install dependencies and set up infrastructure. We live in a world that has tools old (Make) and new (Docker) that can be combined to make onboarding engineers low or zero friction.

Wobbly onboarding

At San Digital we keep in mind some of the original intent of the Agile software movement. Documentation is great, but a working tool is even better. Particularly when you have a problem where code and document are quite similar, like setting up and performing actions on a software project.

We have all been there, you arrive at your new job, sit down at the computer and check out the code you will be working on. The directory is littered with various READMEs and dangerous looking shell scripts, you ask one of your colleagues how to build it and they send you a link to a wiki page last updated 3 years ago. Eventually you crack and ask to pair with a tech lead, who begins hammering away in a terminal saying things like “oh I remember now”. Eventually you get given a few nasty looking shell snippets to build, test and deploy the project and you lean on your .zsh_history like a true professional.

Things have improved a little over the years, even as our toolchains have got more complicated. Continuous integration pretty much guarantees that at least something can build the project from scratch, and preferably not on a My Machine. It’s not just build and test however, there are many other housekeeping operations, test data generation, setting up local development infrastructure associated with a modern software project.

I don’t want to have to trawl a wiki and read a load of probably out of date TLDR; to get a project up and running. Nor do I want to waste the time of my colleagues asking exactly which version of frumpy.js needs installing globally to stop the build screaming at me in red.

There’s no need for this, at all. And in typical ‘Draw the rest of the owl’ style I will present a simple but illustrative example - the build system for this website.

Docker

Pretty much everyone uses docker now, unless you have been living under a rock in a space like embedded software development. Package the exact versions of a tool and OS you want, spin it up. Really useful. You should try it if you haven’t already.

The tool used to build this website is a rust static site generator called Zola (I like Rust, it’s beautifully designed from build system to language and has a remarkably talented and well governed community). To dockerise an instance of Zola we use the following Dockerfile, which is simple enough to just leave in the root of our site’s git repository.

FROM bitnami/minideb AS builder
ENV ZOLA_VERSION v0.12.2
RUN mkdir -p /workdir
WORKDIR /workdir
RUN curl -L https://github.com/getzola/zola/releases/download/$ZOLA_VERSION/zola-$ZOLA_VERSION-x86_64-unknown-linux-gnu.tar.gz | tar xz
RUN mv zola /usr/bin

This uses a minimalist debian base, downloads a release of Zola from github, untars it and stuffs it in /usr/bin. A terrible thing to do to your development machine but you can be mean to docker images.

So now we should be able to run a dockerised Zola on all sane operating systems, with no system dependency other than docker.

We can spin it up and use it to build and serve our website with:

> docker build . -t san-zola
> docker run -p 1111:1111 -v `pwd`:/work -w /work -it san-zola zola serve

Now all we need is a nice way of running it, rather than pasting that snippet into slack.

The re-entrant makefile trick

The only other thing we are assuming is installed is some version of make, which is a good bet. Make has some nice affordances, even when you are not really using it to ..make things. It is a terse, but pretty readable declarative syntax that can manage dependencies and integrates well with most shells. You can usually type make, then tab out the options.

We have two things that we need to do for this smooth owl sized projects - statically compile a version of the size. And do the same, but while watching for changes and serving content.

.PHONY: docker
docker: 
	docker build . -t san-zola

docker-%: docker
	docker run -v $(PWD):/work -w /work -it san-zola make $*

.PHONY: build
build:
	zola build

.PHONY: serve
serve:
	zola serve

Our first target ‘docker’ unsurprisingly builds the dockerfile into an image. .PHONY is a way of telling make to not behave like make, without the PHONY it will start checking for a file called ‘docker’ which is probably not what you want when it happens.

Our second target is the re-entrant trick. The docker-% target passes whatever is on the other side of the hypen to as the $* variable. Allowing us to call make inside the docker container with the other target. So when we call make docker-serve this happens:

1. Make sees the docker-% target is matched
2. docker-% has a dependency of docker so make calls the docker target
3. Make calls docker build . -t san_zola, ensuring we always have an up to date docker image
4. Make calls docker run -v $(PWD):/work -w /work -it san-zola make zola serve
5. Zola spins up and does what we want

This is an obviously simple example. But on projects San Digital lead and work on, we will ensure that all required operations for day to day developer activity are made and kept this frictionless.