Recently I spent some time configuring a pipeline on Gitlab CI for an Elixir project. While
mix is basically all you need, there are several principles I wanted to apply for this configuration that made the setup more complex than expected, especially when trying to also speed up the build.
Of course, everything in this post is opinable and I am really open to suggestions, if you have any hint please add a comment at the end of this post.
The final configured flow will be the following:
Gitlab CI will compile the project with
MIX_ENV set to
test in a first, separated step. As soon as this job is done it will fire 3 other tasks in parallel:
testto run our project tests.
formatto check that our source code is compliant with
credoto statically analize our source code for refactoring opportunities.
If everything goes well the pipeline will be green and a deploy could eventually be triggered.
You might ask yourself why should the pipeline be this complex when a single CI job, configured to run all the above commands in sequence, would have been enough to solve the problem. Well, mainly for three reasons:
The first task to configure is the compilation step (beware that this runs in a runner of type
docker in Gitalb CI, you can’t use the exact same configuration with other runners like the
stages: - build - quality build: stage: build image: elixir:1.9.4 variables: MIX_ENV: "test" script: - mix deps.get - mix compile artifacts: paths: - _build/ - deps/ expire_in: 10 minutes
This job uses a predefined docker image that has Elixir already installed, fetches the dependencies and compiles the application. Before ending this task some artifacts are uploaded on Gitlab CI and stored for 10 minutes, the
_build/ folder and the
deps/ folder. These two folders are reused in the next steps to avoid a new compilation and speed up the build.
As you can see on the next tasks, there is no need to redeclare the artifacts when you want to use them, the subsequent tasks will use them right away.
This is the setup for
test: stage: quality image: elixir:1.9.4 variables: MIX_ENV: "test" POSTGRES_DB: "app_db" POSTGRES_USER: "postgres" POSTGRES_PASSWORD: "postgres" POSTGRES_ENV_HOSTNAME: "postgres" POSTGRES_ENV_PORT: "5432" services: - name: postgres:latest script: - mix deps.get - mix ecto.create - mix ecto.migrate - mix test format: stage: quality image: elixir:1.9.4 variables: MIX_ENV: "test" script: - mix deps.get - mix format --check-formatted --dry-run credo: stage: quality image: elixir:1.9.4 variables: MIX_ENV: "test" script: - mix deps.get - mix credo suggest --min-priority 10
You may wonder why the Gitlab CI cache functionality has not been used in this example, as it is the recommended way to store project dependencies. While it works, for some reasons
mix triggers a new compilation in the
credo steps when using the cache, and correctly skips the compilation when using artifacts.
With this configuration each task is run in its own CI job, some jobs are executed in parallel and there is an acceptable level (at least for me) of isolation between jobs, while skipping a recompilation on each build step. In terms of execution time, most of it is spent on the compilation step, which unfortunately always starts from scratch. It would be better if subsequent commits on the same branch could be detected (which is possible with Gitlab CI caching) and if
mix could just recompile the changes instead of retriggering the pipeline from scratch, but that’s an optimization I haven’t tried yet.
I hope that’s useful, if you have any feedback I am all ears.