Pipfile.lock → requirements.txt

TL;DR

Update

As it was pointed out in the comments, in the v2022.4.8 release a new command is added, which does the desired:

1
2

pipenv requirements > requirements.txt
pipenv requirements --dev-only > requirements-dev.txt

Here is the documentation for the command.

Old approach

For requirements.txt:

1
2
3
4

jq -r '.default
        | to_entries[]
        | .key + .value.version' \
    Pipfile.lock > requirements.txt

For requirements-dev.txt:

1
2
3
4

jq -r '.develop
        | to_entries[]
        | .key + .value.version' \
    Pipfile.lock > requirements-dev.txt

Explanation

I intensively use Python at my job. And Pipenv is a must for managing packages and virtual environments. It keeps all dependencies in Pipfile and lock their precise versions in Pipfile.lock. Although it’s a great tool in development, sometimes we still need requirements.txt file.

As an example I will use building a docker container image with python environment inside. In our case first caveat was that we were using pipenv install during container build. This command (surprisingly) doesn’t just install all packages with the versions from Pipfile.lock as we expected, but update dependencies with lock subcommand first and then run sync subcommand to actually install them. Because the lock subcommand updates packages only to versions compatible with the ones specified in Pipfile, It might be not a problem as far as versions in Pipfile are restrictive enough. In the other case, one can get a broken distribution even if an application was working locally.

After discovering this behavior of Pipenv, to avoid such issues we started using pipenv sync directly.

But in the next time our build process was broken by buggy update of Pipenv itself. So, we had to fix Pipenv version during installation in inside Dockerfile.

And afterwards I got an idea that we, actually don’t need Pipenv inside containers at all. It was used only as a wrapper on pip for parsing Pipfile.lock. We just need to provide requirements.txt file and use pip on it.

There is an official way to generate requirements.txt with Pipenv:

1

pipenv lock -r > requirements.txt

It works, however, it has the before mentioned side-effect and there is no pipenv sync -r alternative.

Someone even created pipenv-to-requirements python package for requirements.txt generation. I haven’t tried it, because I found another way.

Sometimes I need to operate with JSON data in terminal and jq is really powerful tool for that, check it if you haven’t seen it. I thought that I can generate requirements.txt with jq, as far as Pipfile.lock is a json file.

So, this is how it looks like:

1
2
3
4

jq -r '.default
        | to_entries[]
        | .key + .value.version' \
    Pipfile.lock > requirements.txt

• -r is needed to get unquoted output
• .defailt gets content of the default section inside Pipfile.lock, change to .develop to generate requirements-dev.txt
• to_entries[] transforms package: info pair to an object {"key": package, "value": info} and pack them to an array
• .key + .value.version builds a string of package name and it’s version

Look inside Pipfile.lock for better understanding.