Continuous Integration

In software engineering, continuous integration (CI) is the practice of merging all developers’ working copies to a shared mainline regularly. It is often split into three steps:

  1. automate the tests: run command on each commit (or each Pull Request), typically unit tests and integration tests.
  2. automate the build: when dealing with a compiled language, compile the source to generate binaries. I can also build the documentation.
  3. automate the deployment: send the binaries to the repository or generate a website, etc.

A CI pipeline runs commands on some virtual machine automatically.

Benefits of CI

  • One can not forget to run tests and immediate feedback is provided: it runs at each commit or Pull Request. A report is sent to the author of the commit.
  • Protects the master branch: commit or PR can be rejected if the test does not pass.
  • Contributor doesn’t need to know details: only the project maintainers need to know how the system works.
  • Can enforce style: A linter can run to check PEP8 or Black.
  • Can check the code on many systems: virtual machines can run Linux, Windows or MacOS systems.

What do you need?

Many solutions exist to run CI pipelines (Gitlab, Github, Jenkins, TravisCI, Appveyor, Azure Pipelines, CircleCI…). They all:

  • Run a test when a web-hook is triggered (usually at each push or PR).
  • Can act as a build-farm (for binaries or documentation) on a “build matrix” (i.e., run on many environments, OS, etc.).
  • Requires clear declaration of dependencies and a set-up virtual machine (that should be maintained).
  • Reports success/Failure to the CSV.

Example

GitHub has recently developed a high-level solution of CI. Before digging into the process, please make sure that your test file is working locally. You should have something like:

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /home/bcharlier/packaging_tutorial
plugins: cov-2.8.1
collected 3 items

biketrauma/tests/test_biketrauma.py ...                                  [100%]

============================ warnings summary ==============================
/home/bcharlier/.local/lib/python3.7/site-packages/pygal/_compat.py:23
  /home/bcharlier/.local/lib/python3.7/site-packages/pygal/_compat.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import Iterable

-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================= 3 passed, 1 warning in 1.56s ======================

Add a .github/workflows file

Setting up a CI is rather easy. It is sufficient to add a single text file .github/workflows in your project. Github has developed a graphical user interface to do it:

  1. In your GitHub project repository: Go to the Actions menu and then select python package workflow.
  2. Customize the workflows file depending on your needs. Beware: getting a correct configuration file is sometimes tedious with CI system…
  3. You can add a badge showing the result of CI to the end-user directly in your Readme.md.
EXERCISE: Setting up CI tools.
  1. Fork the biketrauma project https://github.com/HMMA238-2020/biketrauma.
  2. Setup a CI with GitHub on biketrauma.

References

Back to top