In the past few years autopkgtest established itself as the system for automated testing of Debian packages.
Although there is good documentation available, both in terms of the command itself and the details about writing tests, I often have to look around and copy-paste code to add automated testing to Python libraries. For this reason, I’m dropping a few notes here. They’re surely going to be useful for me, and hopefully for others too!
Adding tests to a packaged Python library
If the package you’re working with already has a debian/tests/ directory, then congratulations! It already ships some autopkgtests, so you can just look around there. Otherwise, go ahead and create the directory. Then, create a file called debian/tests/control with the following contents:
Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py -c 'import module_name' ; done
Replace module_name with the name of the Python module you want to test, and you’re done writing some basic automated testing for your Debian Python package. What the test does is simply trying to import the module, with the added bonus of doing so for all supported python3 versions on the system.
You likely want to test something more than just importing the library. Simply write your test code as a python script (eg: debian/tests/smoke.py) and change debian/tests/control as follows:
Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py -c 'import module_name' ; done Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py debian/tests/smoke.py' ; done
By default, autopkgtest installs all the packages generated by the source package containing the tests. If you want to install only one specific package, or if the tests to be executed need another set of packages, use the Depends: field:
Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py -c 'import module_name' ; done Depends: python3-module_name Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py debian/tests/smoke.py' ; done Depends: python3-module_name, curl
A lot of Debian packages are using autopkgtest. Get inspired by taking a look at some of them with the Debian Code Search query Depends path:debian/tests/control.
Running the tests
There are many ways to execute the tests with autopkgtest on your development machine. The system supports various virtualization servers, depending on the level of isolation you want. LXC, Docker, and QEMU are some of the supported virtualization servers. QEMU is the easiest way I found, and the one providing the best isolation. See isolation-machine in the specifications.
An autopkgtest virtual machine image for QEMU can be created as follows:
sudo autopkgtest-build-qemu unstable /var/tmp/autopkgtest-unstable.img
Once the image is in place, you can execute your tests from within the package source directory with:
autopkgtest -- qemu /var/tmp/autopkgtest-unstable.img
There are multiple knobs to play with, for example in terms of the resources to assign to the VM. For example, you can specify the number of virtual CPUs and the amount of memory:
autopkgtest -- qemu --cpus=8 --ram-size=4096 /var/tmp/autopkgtest-unstable.img
Additionally, you may want to get dropped into a shell in case of tests failure to investigate the situation. That can be done with --shell-fail:
autopkgtest --shell-fail -- qemu --cpus=8 --ram-size=4096 /var/tmp/autopkgtest-unstable.img
See man autopkgtest-virt-qemu(1) for the fine details, and happy testing!