This is a tool to convert a Pipenv file into requirements.txt (or requirements-dev.txt).
When containerizing a Python application that used pipenv to specify requirements, I found that the images were very large -- often more than 2.0GB -- with a large amount of cached data and long build times, even on reasonable hardware. By switching to use 'requirements.txt' instead of Pipfile (and Pipfile.lock), I was able to drop one image from 2.16GB to 0.68GB (688MB) and cut build times by about the same ratio (roughly 7 minutes down to roughly 2 minutes on average). The image stored on AWS ECR was about 0.200GB (200MB) which was very helpful to us. I was able to explore this path because virtual environments -- which can be immensely helpful when developing in an environment with multiple projects, multiple Python versions, multiple module requirements, etc. -- weren't needed in the image where only ever one version of Python would be installed along with only the modules needed to run the application.
The application being containerized didn't need pipenv or virtual
environments -- it just needed a set of Python modules. So, to
simplify the process, I created an image that would use pipenv
to generate a Pipfile.lock
file which I was able to feed into
pipenv_to_requirements
to generate a requirements.txt
(or requirements-dev.txt
) file which
pip install -r
could read and apply.
This image goes through two steps:
- use
pipenv --lock
to generate aPipfile.lock
file - use
pipenv_to_requirements
to generaterequirements.txt
The pipenv_to_requirements tool works from a Pipfile.lock
file
(not Pipfile
); therefore, the first step is to generate Pipfile.lock
for use in the next step.
Running pipenv --lock
scans for requirements in the [packages]
and
[packages-dev]
sections of the Pipfile
. Because of that, the
pipenv_to_requirements
tool can generate either requirements.txt
(from the [packages]
stanza) or requirements-dev.txt
(from the
[packages-dev]
stanza).
The tool reads the Pipfile from STDIN. Therefore, the content of the
Pipfile
will need to be piped in to docker run
. So, either of these
forms will work:
# piped
cat Pipfile | docker run -i [...]
# redirected
docker run -i [...] < Pipfile
The "piped" form is more readable while the "redirected" form spawns one less process (which makes it slightly more efficient).
Note: because docker run
is reading from STDIN, the -i
flag for
docker run
is required to bind STDIN to the container.
Once we have the Pipfile.lock
file, we can use pipenv_to_requirements
to generate requirements.txt
or requirements-dev.txt
.
A container running this image is able to pass several flags along to
pipenv_to_requirements
:
- -o: generate
requirements.txt
(the default) - -d: generate
requirements-dev.txt
- -q: run quietly (don't display logs)
- -f: generate files with frozen versions
- -l: the file provided was a Pipfile.lock (not a Pipfile)
The flags are placed at the end of the docker run
command after the
name of the image. Technically speaking, a script is setup as an
ENTRYPOINT so that flags are passed as through to the script as by
a CMD option.
The script that manages the process is a Bash script that uses
getopts
so GNU-style flag combination is supported (e.g.,
-df
is the same as -d -f
)
docker build -t pipenv_to_requirements .
docker run -i --rm pipenv_to_requirements < Pipfile
# local image:
docker run -i --rm pipenv_to_requirements -d < Pipfile
# dockerhub image:
docker run -i --rm wesleydeanflexion/pipenv_to_requirements -d < Pipfile
# pipe form with implied output specification
cat Pipfile | docker run -i --rm pipenv_to_requirements > requirements.txt
# redirect form with explicit output specification
docker run -i --rm pipenv_to_requirements -o < Pipfile > requirements.txt
cat Pipfile.lock | docker run -li --rm pipenv_to_requirements > requirements.txt
cat Pipfile | docker run -i --rm pipenv_to_requirements -d > requirements-dev.txt
cat Pipfile | docker run -i --rm pipenv_to_requiremenst -df > requirements-dev.txt