How to Improve Software Quality in Open Source Projects: Part II – Packaging PyNWB for Multiple Operating Systems and Python Versions

Index To the Series
1. Overview of Continuous Integration (CI) and Software Process in PyNWB
2. Packaging PyNWB for Multiple Operating Systems and Python Versions
3. Code Coverage, Documentation and Style

In our previous post, we introduced the concept of continuous integration (CI) and described its benefits for software quality. In this post, we talk about the nuts and bolts of the PyNWB project such as Python 2 and Python 3 support, cross-platform operating system support, source distributions and dependency versioning.

Python 2 and Python 3 Support

While we wait for all dependent projects of the PyNWB ecosystem to be updated to support Python 3.x, we still need to support both Python 2.7 and Python 3.6. Using a tool such as Six as a compatibility layer between Python versions allows us to write concise code that can work efficiently with Python 2.7 and Python 3.6. Six does not test packages against different Python versions though. In order to run the tests manually, we have to create two virtual environments, establish the software process on both environments and run the tests in both environments.

Let’s consider what happens if PyNWB also needs to support Python 3.7: The above process must be repeated. To do so, we would use tox. This is an automation tool and a common standard for creating test environments across different Python versions. It has a simple configuration file called “tox.ini,” which is human readable and can integrate with most CI vendors.

This specific configuration lets tox create two clean environments with different Python versions (2.7 and 3.6), install the library and test dependencies and run the tests in both environments. We can add a new environment with one more entry to the envlist (such as py37).

The tox file used in the PyNWB project is located in the root directory of the repository. Other files located in the repository are requirements.txt and requirements-dev.txt, which help create a reproducible environment. The requirements.txt file lists Python dependencies of the package, and the requirements-dev.txt file lists Python dependencies for the development of the package. Alternatively, setup.py installs the dependencies of the package as part of its installation on a system. The difference between requirements.txt and setup.py is covered in great detail in “setup.py vs requirements.txt.”

As a side note, you need more than the ability to test code in multiple environments to support two different versions of Python. There are many resources dedicated to this topic. For PyNWB, we worked with the project’s core maintainers to select a codebase that is compatible with Python 2 and Python 3. We used the Six library and prevented the inclusion of any Python 3 language features in the codebase. We also worked with the maintainers to accommodate the schedule and requirements of the different elements of the PyNWB ecosystem. Navigating these accommodations is typical when creating a robust and distributed open source community. The “social” tools of clear communication, an articulated code of conduct, and compromise are every bit as important as the technical tools.

Cross-platform Operating System Support

A common challenge in CI is testing software across several different operating systems, as the process requires different CI vendors. (Currently, each vendor allows testing to occur on a single operating system for unpaid open source projects.)

For PyNWB, we use CircleCI for Linux, AppVeyor for Windows and Travis CI for macOS. All of these platforms can run tests for different Python versions using tox. When compared to scripts that are specific to one platform, tox greatly reduces the maintenance overhead associated with running tests with all three vendors, as it abstracts away platform differences. The fact that the syntax for all three vendors is fairly similar also reduces overhead.

For each CI vendor, we employ specific files: a CircleCI file that configures the build on Debian by leveraging Docker images, an AppVeyor file that configures the build on Windows and a Travis CI file that configures the build on macOS. Manylinux is the preferred way to create wheels on Linux based systems for packages that require compilation. With PyNWB since we don’t compile software to build the wheel we just used the Debian images provided by CircleCI. Every time there is a pull request, the build is triggered on all three operating systems, and the results are reported back to GitHub. GitHub notifies services like Read the Docs to build the documentation.

We set the results of the multiple build systems as requirements for merging a pull request, i.e., a pull request must pass tests on all platforms before it can be merged. To activate the integration between GitHub and the various CI services, we set webhooks on certain events in the GitHub repository.

Source and Wheel Distributions

PyPI is the official Python package distribution center for sending packages to pip. For the PyNWB project, we publish packages on PyPI so that they can be installed using the pip tool. We also publish the packages on GitHub. More specifically, we use scikit-ci-addons to publish the development (or pre-release) wheels and tarballs we create for the package. We create these wheels and tarballs as part of the CircleCI build. Directives on publishing the development wheel and source distribution on GitHub are available in the CircleCI file.

Conda has become very popular in the Python community, especially among scientists. We set PyNWB to publish packages to conda-forge using a conda-forge recipe so that we can install the latest release from Conda. To learn more about publishing Python packages, we recommend in depth tutorials for PyPI and Conda.

Dependency Versioning

The requires.io service ensures that a development environment is kept up to date, thereby incorporating essential bug fixes, performance improvements and security patches into the environment. The service discovers dependencies in a repository and checks if there are security updates or other updates for a package.

When all of the requirements are up-to-date, requires.io furnishes the repository with this shiny badge:

For PyNWB, we operate in a very fast-moving environment. Every time there is a change in the PyNWB code (either a Pull Request or an update to the integration branch), our test suite checks the changes against three operating systems (Linux, macOS, and Windows), three versions of Python (2.7, 3.5 and 3.6) and two package managers (pip and Conda). Running the tests against all of these systems, versions and managers makes us confident that we are not breaking code for a wide array of users.