Continuous Integration
In software engineering, continuous integration (CI) is the practice of merging all developers’ working copies to a shared mainline regularly. It is often split into three steps:
- automate the tests: run command on each commit (or each Pull Request), typically unit tests and integration tests.
- automate the build: when dealing with a compiled language, compile the source to generate binaries. I can also build the documentation.
- automate the deployment: send the binaries to the repository or generate a website, etc…
A CI pipeline runs commands on some virtual machine automatically.
Benefits of CI
- One can not forget to run tests and immediate feedback is provided: it runs at each commit or Pull Request. A report is sent to the author of the commit.
- Protects the master branch: commit or PR can be rejected if the test does not pass.
- Contributor doesn’t need to know details: only the project maintainers need to know how the system works.
- Can enforce style: A linter can run to check PEP8 or Black.
- Can check the code on many systems: virtual machines can run Linux, Windows or MacOS systems.
What do you need?
Many solutions exist to run CI pipelines (Gitlab, Github, Jenkins, TravisCI, Appveyor, Azure Pipelines, CircleCI…). They all:
- Run a test when a web-hook is triggered (usually at each push or PR).
- Can act as a build-farm (for binaries or documentation) on a “build matrix” (i.e., run on many environments, OS, etc.).
- Requires clear declaration of dependencies and a set-up virtual machine (that should be maintained).
- Reports success/Failure to the CSV.
Example
GitHub has recently developed a high-level solution of CI. Before digging into the process, please make sure that your test file is working locally. You should have something like:
$ pytest=========================== test session starts ============================
-- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
platform linux /home/bcharlier/packaging_tutorial
rootdir: -2.8.1
plugins: cov3 items
collected
/tests/test_biketrauma.py ... [100%]
biketrauma
============================ warnings summary ==============================
/home/bcharlier/.local/lib/python3.7/site-packages/pygal/_compat.py:23
/home/bcharlier/.local/lib/python3.7/site-packages/pygal/_compat.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
from collections import Iterable
-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================= 3 passed, 1 warning in 1.56s ======================
Add a .github/workflows
file
Setting up a CI is rather easy. It is sufficient to add a single text file .github/workflows
in your project. Github
has developed a graphical user interface to do it:
- In your GitHub project repository: Go to the Actions menu and then select
python package
workflow. - Customize the
workflows
file depending on your needs. Beware: getting a correct configuration file is sometimes tedious with CI system… - You can add a
badge
showing the result of CI to the end-user directly in yourReadme.md
.