Recently I spent some time configuring a pipeline on Gitlab CI for an Elixir project. While mix
is basically all you need, there are several principles I wanted to apply for this configuration that made the setup more complex than expected, especially when trying to also speed up the build.
Of course, everything in this post is opinable and I am really open to suggestions, if you have any hint please add a comment at the end of this post.
The final configured flow will be the following:
Gitlab CI will compile the project with MIX_ENV
set to test
in a first, separated step. As soon as this job is done it will fire 3 other tasks in parallel:
test
to run our project tests.format
to check that our source code is compliant with elixir-format
.credo
to statically analize our source code for refactoring opportunities.If everything goes well the pipeline will be green and a deploy could eventually be triggered.
You might ask yourself why should the pipeline be this complex when a single CI job, configured to run all the above commands in sequence, would have been enough to solve the problem. Well, mainly for three reasons:
The first task to configure is the compilation step (beware that this runs in a runner of type docker
in Gitalb CI, you can’t use the exact same configuration with other runners like the shell
runner).
stages:
- build
- quality
build:
stage: build
image: elixir:1.9.4
variables:
MIX_ENV: "test"
script:
- mix deps.get
- mix compile
artifacts:
paths:
- _build/
- deps/
expire_in: 10 minutes
This job uses a predefined docker image that has Elixir already installed, fetches the dependencies and compiles the application. Before ending this task some artifacts are uploaded on Gitlab CI and stored for 10 minutes, the _build/
folder and the deps/
folder. These two folders are reused in the next steps to avoid a new compilation and speed up the build.
As you can see on the next tasks, there is no need to redeclare the artifacts when you want to use them, the subsequent tasks will use them right away.
This is the setup for test
, format
and credo
.
test:
stage: quality
image: elixir:1.9.4
variables:
MIX_ENV: "test"
POSTGRES_DB: "app_db"
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres"
POSTGRES_ENV_HOSTNAME: "postgres"
POSTGRES_ENV_PORT: "5432"
services:
- name: postgres:latest
script:
- mix deps.get
- mix ecto.create
- mix ecto.migrate
- mix test
format:
stage: quality
image: elixir:1.9.4
variables:
MIX_ENV: "test"
script:
- mix deps.get
- mix format --check-formatted --dry-run
credo:
stage: quality
image: elixir:1.9.4
variables:
MIX_ENV: "test"
script:
- mix deps.get
- mix credo suggest --min-priority 10
You may wonder why the Gitlab CI cache functionality has not been used in this example, as it is the recommended way to store project dependencies. While it works, for some reasons mix
triggers a new compilation in the test
, format
and credo
steps when using the cache, and correctly skips the compilation when using artifacts.
With this configuration each task is run in its own CI job, some jobs are executed in parallel and there is an acceptable level (at least for me) of isolation between jobs, while skipping a recompilation on each build step. In terms of execution time, most of it is spent on the compilation step, which unfortunately always starts from scratch. It would be better if subsequent commits on the same branch could be detected (which is possible with Gitlab CI caching) and if mix
could just recompile the changes instead of retriggering the pipeline from scratch, but that’s an optimization I haven’t tried yet.
I hope that’s useful, if you have any feedback I am all ears.
Comments