Packaging Python software with pbr

Monday 02 January 2017 Python, pbr Comments

Packaging Python has been a painful experience for long. The history of the various distribution that Python offered along the years is really bumpy, and both the user and developer experience has been pretty bad.

Fortunately, things improved a lot in the recent years, with the reconciliation of setuptools and distribute.

Though in the context of the OpenStack project, a solution on top of setuptools has been already started a while back. Its usage is now spread across a whole range of software and libraries.

This project is called pbr, for Python Build Reasonableness. Don't be afraid by the OpenStack colored themed of the documentation – it is a bad habit of OpenStack folks to not advertise their tooling in an agnostic fashion. The tool has no dependency with the cloud platform, and can be used painlessly with any package.

How it works

pbr takes inspiration from distutils2 (a now abandoned project) and uses a setup.cfg file to describe the packager's intents. This is how a setup.py using pbr looks like:

import setuptools
 
setuptools.setup(setup_requires=['pbr'], pbr=True)


Two lines of code – it's that simple. The actual metadata that the setup requires is stored in setup.cfg:

[metadata]
name = foobar
author = Dave Null
author-email = foobar@example.org
summary = Package doing nifty stuff
license = MIT
description-file =
README.rst
home-page = http://pypi.python.org/pypi/foobar
requires-python = >=2.6
classifier =
Development Status :: 4 - Beta
Environment :: Console
Intended Audience :: Developers
Intended Audience :: Information Technology
License :: OSI Approved :: Apache Software License
Operating System :: OS Independent
Programming Language :: Python
 
[files]
packages =
foobar


This syntax is way easier to write and read than the standard setup.py.

pbr also offers other features such as:

  • automatic dependency installation based on requirements.txt
  • automatic documentation building and generation using Sphinx
  • automatic generation of AUTHORS and ChangeLog files based on git history
  • automatic creation of the list of files to include using git
  • version management based on git tags

All of this comes with little to no effort on your part.

Using flavors

One of the feature that I use a lot, is the definition of flavors. It's not tied particularly to pbr – it's actually provided by setuptools and pip themselves – but pbr setup.cfg file makes it easy to use.

When distributing a software, it's common to have different drivers for it. For example, your project could support both PostgreSQL or MySQL – but nobody is going to use both at the same time. The usual trick to make it work is to add the needed library to the requirements list (e.g. requirements.txt). The upside is that the software will work directly with either RDBMS, but the downside is that this will install both libraries, whereas only one is needed. Using flavors, you can specify different scenarios:

[extras]
postgresql =
psycopg2
mysql =
pymysql


When installing your package, the user can then just pick the right flavor by using pip to install the package:

$ pip install foobar[postgresql]


This will install foobar, all its dependencies listed in requirements.txt, plus whatever dependencies are listed in the [extras] section of setup.cfg matching the flavor. You can also combine several flavors, e.g.:

$ pip install foobar[postgresql,mysql]


would install both flavors.

pbr is well-maintained and in very active development, so if you have any plans to distribute your software, you should seriously consider including pbr in those plans.