Inside Out | Python pipfiles and packaging

I’m happy to announce the first release of bout↗. It parses the pdf bank statements and exports the transactions to QIF↗ format.

I got an opportunity to play around with pipenv for this tool. So let’s go over some tidbits.

Delicious environments

As the name may suggest, pipenv = pip + virtualenv. An amazing tool that provides an alternative to requirements.txt, and promises deterministic builds. I’ll provide a quick command line walkthrough below. There’s a nice screencast and more details are available at pipenv↗.

> sudo pacman -S python-pipenv
> mkdir /tmp/trial
> cd /tmp/trial
> which python
/usr/bin/python
> pipenv --three    # create a python 3 virtualenv
> pipenv shell
(trial)> which python
/tmp/trial/.venv/bin/python

Let’s install a few packages into the fresh virtual environment.

(trial)> pipenv install bout
Adding bout to Pipfile's [packages]…                                                                                                                        
Locking [dev-packages] dependencies…                                                                                                                       
Locking [packages] dependencies…                                                                                                                           
Updated Pipfile.lock (127440)! 

(trial)> cat Pipfile # should have the reference to `bout`
...
[packages]
bout = "*"

...
(trial)> cat Pipfile.lock
# See the locked package versions and other metadata -> deterministic builds

(trial)> bout --help  # run any installed command as you'd in a virtualenv

There’s a way to instruct pipenv to always create the virtualenv in the local directory (instead of a common location provided by the pew package). To do that set export PIPENV_VENV_IN_PROJECT=1 in your shell init (~/.zshenv for me). Now the virtualenv will be setup in .venv subdirectory of a project.

More tricks are at https://pipenv.readthedocs.io/en/latest/advanced/↗.

Packaging a pipenv based project

Hopefully you’re convinced on using pipenv for the next app. Let’s look at the next problem: do I need to specify dependencies in both setup.py and Pipfile. How do we keep both in sync?

The answer is simple and interesting: don’t maintain the runtime dependencies in Pipfile. Pipfile is an app environment, setup.py/install_requires is the runtime dependency. There are a few readings at the end of the post if you’re interested to learn more.

So we’d just install the app in Pipfile since app is the crucial part of the environment, along with other helpful packages (say pytest to test it). E.g. pipenv install -e .. This would emit a Pipfile as shown below. Under the hoods, it will run python setup.py develop, which will gather requirements from install_requires.

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
"e1839a8" = {path = ".", editable = true}

[dev-packages]

[requires]
python_version = "3.6"

Any dev packages can be installed with pipenv install pytest --dev. They’d be listed in dev-packages category.

Recommended readings which clarified these thoughts:

That’s all for today. Namaste!