Testing Debian Python Packages with Autopkgtest

Published in

In the past few years autopkgtest established itself as the system for automated testing of Debian packages.

Although there is good documentation available, both in terms of the command itself and the details about writing tests, I often have to look around and copy-paste code to add automated testing to Python libraries. For this reason, I’m dropping a few notes here. They’re surely going to be useful for me, and hopefully for others too!

Adding tests to a packaged Python library

If the package you’re working with already has a debian/tests/ directory, then congratulations! It already ships some autopkgtests, so you can just look around there. Otherwise, go ahead and create the directory. Then, create a file called debian/tests/control with the following contents:

Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py -c 'import module_name' ; done

Replace module_name with the name of the Python module you want to test, and you’re done writing some basic automated testing for your Debian Python package. What the test does is simply trying to import the module, with the added bonus of doing so for all supported python3 versions on the system.

You likely want to test something more than just importing the library. Simply write your test code as a python script (eg: debian/tests/smoke.py) and change debian/tests/control as follows:

Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py -c 'import module_name' ; done

Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py debian/tests/smoke.py' ; done

By default, autopkgtest installs all the packages generated by the source package containing the tests. If you want to install only one specific package, or if the tests to be executed need another set of packages, use the Depends: field:

Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py -c 'import module_name' ; done
Depends: python3-module_name

Test-Command: for py in $(py3versions -s); do echo "[*] testing on $py:"; $py debian/tests/smoke.py' ; done
Depends: python3-module_name, curl

A lot of Debian packages are using autopkgtest. Get inspired by taking a look at some of them with the Debian Code Search query Depends path:debian/tests/control.

Running the tests

There are many ways to execute the tests with autopkgtest on your development machine. The system supports various virtualization servers, depending on the level of isolation you want. LXC, Docker, and QEMU are some of the supported virtualization servers. QEMU is the easiest way I found, and the one providing the best isolation. See isolation-machine in the specifications.

An autopkgtest virtual machine image for QEMU can be created as follows:

sudo autopkgtest-build-qemu unstable /var/tmp/autopkgtest-unstable.img

Once the image is in place, you can execute your tests from within the package source directory with:

autopkgtest -- qemu /var/tmp/autopkgtest-unstable.img

That’s it!

There are multiple knobs to play with, for example in terms of the resources to assign to the VM. For example, you can specify the number of virtual CPUs and the amount of memory:

autopkgtest -- qemu --cpus=8 --ram-size=4096 /var/tmp/autopkgtest-unstable.img

Additionally, you may want to get dropped into a shell in case of tests failure to investigate the situation. That can be done with --shell-fail:

autopkgtest --shell-fail -- qemu --cpus=8 --ram-size=4096 /var/tmp/autopkgtest-unstable.img

See man autopkgtest-virt-qemu(1) for the fine details, and happy testing!