CI Testing Pipeline
The following resources are used by Roundup to implement a Continuous Integration/Test pipeline.
GitHub Repo
We maintain roundup using Mercurial at Sourceforge (see the Code link in the sidebar). Primary developers also have access to the GitHub clone at https://github.com/roundup-tracker/roundup and sync changes to GitHub. Usually larger code changes are synced. Documentation and minor changes that can be sufficiently tested on a developer system are usually not synced. Only the newest commit when a sync happens is submitted to the net step in the pipeline.
CI
TravisCI
The newest commit from a sync to GitHub is built on TravisCI: https://app.travis-ci.com/github/roundup-tracker/roundup. Only two python versions are tested to preserve the free credits we get as an open-source project.
The last python2 release is tested. This is the only python2 test as GitHub's native support for python2 was removed. Also the 3.12 alpha release is also tested.
GitHub Actions
GitHub actions are used to test 5 Python 3.x releases. Python 2 is not tested anymore because the Python workflow no longer supports Python 2. If required we could try spinning up a Python 2 docker environment to test Python 2, but for now we just use TravisCI. In addition we test:
- python 3.6 on ubuntu 20.04 (earliest supported Python version, not available on ubuntu 22.)
- python 3.10, 3.8 and 3.11 on ubuntu 22.04
- python 3.12 pre-release (beta 4 at this time) on ubuntu 22.04
The goal is to test earliest version, two or three intermediate releases and the newest release. Also test the newest version of the pre-release (beta/alpha).
In addition to running the test suite, GitHub actions also run:
- dependabot - looking for outdated dependencies. Note that it is not used for
- reporting on the python3 Alpine docker image as returns a false positive when the Debian (not Alpine) python3 image is updated.
- ossf-scorecard supply chain scanning.
- anchore/grype - docker image security scanning.
- codeql - source code security scanning
CodeCov
The results of code coverage are uploaded to CodeCov. We target 80 or better % coverage of changes for each commit. However a few parts of our codebase (most notably templating) are sparsely covered. On March 19, 2019 code coverage was at 65%. We have consistently been getting better. Both Python 2 and Python 3 testing is reported here.
Coveralls
Python 3 test results are loaded to Coveralls. Since Python 2 is not supported by Coveralls, the reported coverage is lower than CodeCov.