Published on 2020-09-21.

Adventures in CI land, or how to speed up your CI

Every project has a Continuous Integration (CI) pipeline and every one of them complains its CI is too slow. It is more important than you might think; this can be the root cause of many problems, including lackluster productivity, low morale, high barrier of entry for newcomers, and overall suboptimal quality.

But this need not be. I have compiled here a lengthy list of various ways you can simplify your CI and make it faster, based on my experience on open-source projects and my work experience. I sure wish you will find something in here worth your time.

And finally, I hope you will realize this endeavour is not unlike optimizing a program: it requires some time and dedication but you will get tremendous results. Also, almost incidentally, it will be more secure and easier to audit.

Lastly, remember to measure and profile your changes. If a change has made no improvements, it should be reverted.

This article assumes you are running a POSIX system. Windows developers, this is not the article you are looking for.

Reduce the size of everything

Almost certainly, your CI pipeline has to download ‘something’, be it a base docker image, a virtual machine image, some packages, maybe a few company wide scripts. The thing is, you are downloading those every time it runs, 24/7, every day of the year. Even a small size reduction can yield big speed ups. Remember, the network is usually the bottleneck.

In no particular order:

Be lazy: Don’t do things you don’t need to do

Miscellenaous tricks

A note on security

I am a DevOps Engineer, what can I do?

Most of the above rules can be automated with a script, assuming the definition of a CI pipeline is in a text format (e.g. Gitlab CI). I would suggest starting here, and teaching developers about these simple tips than really make a difference.

I would also suggest considering adding strict firewall rules inside CI pipelines, and making sure the setup/teardown of CI runners is very fast. Additionally, I would do everything to avoid a situation where no CI runner is available, preventing developers from working and deploying.

Finally, I would recommend leading by example with the pipelines for the tools made by DevOps Engineers in your organization.

Closing words

I wish you well on your journey towards a fast, reliable and simple CI pipeline.

I noticed in my numerous projects with different tech stacks that some are friendlier than others towards CI pipelines than others (I am looking at you, Gradle!). If you have the luxury of choosing your technical stack, do consider how it will play out with your pipeline. I believe this is a much more important factor than discussing whether $LANG has semicolons or not because I am convinced it can completely decide the outcome of your project.