Hello and thank you for this useful buildpack!

We hit dependabot/dependabot-core#6462 in our project, where Dependabot generates a Python version with the tilde specifier (~3.11) in poetry.lock when pyproject.toml contains a plain full version like 3.11.1, which is considered invalid with this buildpack.

I saw the discussion in #6, but would you consider supporting at least the exact version specifier (==3.11.1) as suggested in the Dependabot issue discussion? That's what Dependabot generates if it's used in pyproject.toml.

Hello,
I use the latest version of this buildpack and since #66 it installs poetry-plugin-export, but this plugin requires Poetry 1.6 and so the deployment doesn’t work if one uses an older version.

Enumerating objects: 32, done.
Counting objects: 100% (32/32), done.
Delta compression using up to 8 threads
Compressing objects: 100% (13/13), done.
Writing objects: 100% (17/17), 2.48 KiB | 636.00 KiB/s, done.
Total 17 (delta 9), reused 0 (delta 0), pack-reused 0
-----> Cleaning up...
-----> Building myapp from herokuish
-----> Adding BUILD_ENV to build environment...
       BUILD_ENV added successfully
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/moneymeets/python-poetry-buildpack.git
=====> Detected Framework: Python Poetry
-----> Read Poetry version 1.5.1 from poetry.lock
-----> Generate requirements.txt with Poetry
-----> Install Poetry
       Retrieving Poetry metadata
       
       # Welcome to Poetry!
       
       This will download and install the latest version of Poetry,
       a dependency and package manager for Python.
       
       It will add the `poetry` command to Poetry's bin directory, located at:
       
       /app/.local/bin
       
       You can uninstall at any time by executing this script with the --uninstall option,
       and these changes will be reverted.
       
       Installing Poetry (1.5.1)
       Installing Poetry (1.5.1): Creating environment
       Installing Poetry (1.5.1): Installing Poetry
       Installing Poetry (1.5.1): Creating script
       Installing Poetry (1.5.1): Done
       
       Poetry (1.5.1) is installed now. Great!
       
       To get started you need Poetry's bin directory (/app/.local/bin) in your `PATH`
       environment variable.
       
       Add `export PATH="/app/.local/bin:$PATH"` to your shell configuration file.
       
       Alternatively, you can call Poetry explicitly with `/app/.local/bin/poetry`.
       
       You can test that everything is set up by executing:
       
       `poetry --version`
       
-----> Add Poetry to the PATH
-----> Install poetry-plugin-export
remote: 
remote: Because no versions of poetry-plugin-export match >1.6.0,<2.0.0
remote:  and poetry-plugin-export (1.6.0) depends on poetry (>=1.6.0,<2.0.0), poetry-plugin-export (>=1.6.0,<2.0.0) requires poetry (>=1.6.0,<2.0.0).
remote: So, because poetry-instance depends on both poetry (1.5.1) and poetry-plugin-export (^1.6.0), version solving failed.
       Using version ^1.6.0 for poetry-plugin-export
       
       Updating dependencies
       Resolving dependencies...
remote:  !     Failure during app build
remote:  !     Retagging old image 793abcde as myremote/myapp:latest
remote:  !     App build failed

Would it be possible to install the plugin only if the Poetry version is >1.6?

Hello - thanks for this buildpack, we are successfully using it in production!

My issue: I'd like to have a dependency that's only installed in production, and not in development. In particular, it's recommended to use psycopg2-binary in development and build psycopg2 from source for production.

Based on this issue it seems that the Poetry recommendation for this use case is to install the prod dependency as an extra and then do poetry install/export -E production when you want to install the prod version.

My ideal solution would be to set an environment variable in Heroku that sets the -E in for the export step in this buildpack. If this is something you'd be willing to accept I can take a stab at implementing and open a PR, let me know!

Steps to reproduce

1. Use this buildpack plus heroku/python to deploy a Python project to Heroku.
2. Use Heroku CI.
3. Include poetry commands in the CI checks:

.local/bin/poetry check
.local/bin/poetry lock --no-update --check

4. Upgrade to Python 3.11.1

[tool.poetry.dependencies]
python = "3.11.1"

Expected behavior

Poetry installs and generates a requirements.txt and runtime.txt, Heroku installs Python version 3.11.1 and the requirements.txt packages, Poetry's checks succeed.

Actual behavior

Poetry installs and generates a requirements.txt and runtime.txt, Heroku installs Python version 3.11.1 and the requirements.txt packages, Poetry's checks fail with an error like this:

Python path configuration:
  PYTHONHOME = '/app/.heroku/python'
  PYTHONPATH = '/app'
  program name = '/app/.local/share/pypoetry/venv/bin/python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/app/.local/share/pypoetry/venv/bin/python'
  sys.base_prefix = '/app/.heroku/python'
  sys.base_exec_prefix = '/app/.heroku/python'
  sys.platlibdir = 'lib'
  sys.executable = '/app/.local/share/pypoetry/venv/bin/python'
  sys.prefix = '/app/.heroku/python'
  sys.exec_prefix = '/app/.heroku/python'
  sys.path = [
    '/app',
    '/app/.heroku/python/lib/python310.zip',
    '/app/.heroku/python/lib/python3.10',
    '/app/.heroku/python/lib/python3.10/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007ff6ee928480 (most recent call first):
  <no Python frame>

Hypothesis

Poetry is installed into its venv using the Heroku default Python for heroku-22 which is 3.10.9. Once heroku/python switches to 3.11.1, the 3.10.9 installation is deleted, which breaks the Poetry venv.

Suggested solution

In this buildpack, use pyenv to install an isolated Python that Poetry can use and rely on. Related to the discussion here: heroku/heroku-buildpack-python#796 (comment)

I'm not sure how to do this in a way that plays nicely with Heroku's caching and stuff. I haven't made a buildpack of my own before.

Workaround

Do not upgrade to 3.11.

My poetry files are not at the root of my project.
So, when I try to deploy the project on Heroku, it fails on the detect phase.

As a workaround, I noticed I can create symbolic links, it works but it's a bit ugly to me.
Is there any other way? I noticed a buildpack works with BUILD_DIR var but I did not find where I can specify it.

Thanks.

Hello dear @marns93 ,
would you mind publishing the pack to Buildpacks.io?
Here is a quick guide : https://buildpacks.io/docs/buildpack-author-guide/publish-a-buildpack/

When we use this build pack in one of our Heroku instance, we got the below error and build rejected.

I was happened when we add new package called requests-negotiate-sspi
Let me know if this package support need to be added under buildpack.

I am trying to use this buildpack for my poetry project but it fails when trying to fetch the custom buildpack:

-----> Building on the Heroku-22 stack
-----> Using buildpack: moneymeets/python-poetry-buildpack
 !     error fetching custom buildpack https://buildpack-registry.s3.amazonaws.com/buildpacks/moneymeets/python-poetry-buildpack.tgz
 !     Push failed

I am not familiar with heroku or this buildpack.
Would appreciate if someone could point me to a solution please 😺
I have also reached out to Heroku support

I'm trying to set the POETRY_VERSION to 1.6.1
however, heroku client just resets it to 1.1.13

to replicate run:

heroku config:set POETRY_VERSION=1.6.1

expected response:

POETRY_VERSION: 1.6.1

actual response:

POETRY_VERSION: 1.1.13

Is there a mechanism that prevents updating this env?

Typically you'd want to specify your python version like:

python = '>=3.7,<4'

or perhaps more reasonably:

python = '^3.7.6'

For a new project that intends to support Python 3.7+. However nether of these options work and will fail with an error like:

remote: -----> Export Python version from Poetry to Heroku runtime.txt file
remote: -----> '^3.7.6' not valid, please specify an exact Python version (e.g. 3.8.1) in your pyproject.toml (and thus poetry.lock)

It would be nice if the latest supported version of python, given the specification string, were automatically picked.

Hi Team,
We are using moneymeets/python-poetry-buildpack for one of our project. we got one error during build. Can you please check and let us know where the issue is exist here.

Please let me know if you need any other details.

GET_POETRY_IGNORE_DEPRECATION=1 doesn't help as well

-----> Building on the Heroku-22 stack
-----> Using buildpacks:
1. https://github.com/moneymeets/python-poetry-buildpack.git
2. heroku/python
3. https://github.com/heroku/heroku-buildpack-cli
-----> Python Poetry app detected
-----> Using Poetry version from POETRY_VERSION config var: 1.1.13
-----> Generate requirements.txt with Poetry
-----> Install Poetry
Retrieving Poetry metadata

   This installer is deprecated, and cannot install Poetry 1.2.0a1 or newer.
   Additionally, Poetry installations created by this installer cannot `self update` to 1.2.0a1 or later.
   You should migrate to https://install.python-poetry.org instead. Instructions are available at https://python-poetry.org/docs/#installation.
   
   This installer will now exit. If you understand the above warning and wish to proceed anyway, set GET_POETRY_IGNORE_DEPRECATION=1 in the environment and run this installer again.

-----> Export requirements.txt from Poetry
/tmp/codon/tmp/buildpacks/6e1a99cb61ad645328dd1c64bd71597add4e2b5b/bin/compile: line 66: poetry: command not found
! Push rejected, failed to compile Python Poetry app.
! Push failed

Since poetry currently does not export runtime.txt Heroku installs a different Python version than what is specified in pyproject.toml.

After Poetry upgrade my heroku deployment stopped working (deploy log attached), switching to previous version with heroku config:set POETRY_VERSION=1.0.10 works as expected:

Here is the output with the latest version:

remote: -----> Build succeeded!
remote: -----> Python Poetry app detected
remote: -----> No Poetry version specified in POETRY_VERSION config var. Defaulting to 1.1.0.
remote: -----> Generate requirements.txt with Poetry
remote: -----> Install Poetry
remote:        Retrieving Poetry metadata
remote:        
remote:        # Welcome to Poetry!
remote:        
remote:        This will download and install the latest version of Poetry,
remote:        a dependency and package manager for Python.
remote:        
remote:        It will add the `poetry` command to Poetry's bin directory, located at:
remote:        
remote:        $HOME/.poetry/bin
remote:        
remote:        This path will then be added to your `PATH` environment variable by
remote:        modifying the profile file located at:
remote:        
remote:        $HOME/.profile
remote:        
remote:        You can uninstall at any time by executing this script with the --uninstall option,
remote:        and these changes will be reverted.
remote:        
remote:        Installing version: 1.1.0
remote:          - Downloading poetry-1.1.0-linux.tar.gz (57.00MB)
remote:        
remote:        Poetry (1.1.0) is installed now. Great!
remote:        
remote:        To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
remote:        environment variable. Next time you log in this will be done
remote:        automatically.
remote:        
remote:        To configure your current shell run `source $HOME/.poetry/env`
remote:        
remote: -----> Export requirements.txt from Poetry
remote: /app/.poetry/lib/poetry/_vendor/py2.7/subprocess32.py:149: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.
remote:   "program uses threads.", RuntimeWarning)
remote: 
remote: Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
remote: You should consider updating your Python version to a supported one.
remote: 
remote: Note that you will still be able to manage Python 2.7 projects by using the env command.
remote: See https://python-poetry.org/docs/managing-environments/ for more information.
remote: 
remote: 
remote: IndexError
remote: 
remote: list index out of range
remote: -----> Export Python version from Poetry to Heroku runtime.txt file
remote: -----> Write 3.8.5 into runtime.txt
remote: -----> App not compatible with buildpack: https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/python.tgz
remote:        More info: https://devcenter.heroku.com/articles/buildpacks#detection-failure
remote: 
remote:  !     Push failed
remote: Verifying deploy...
remote: 
remote: !	Push rejected to heroku-example-app.
remote: 

Current Poetry installer gives

This installer is deprecated. Poetry versions installed using this script will not be able to use 'self update' command to upgrade to 1.2.0a1 or later.

warning. Related discussion can be found here

Now that we are featured on Heroku, it would be nice to have a README:

https://elements.heroku.com/buildpacks/moneymeets/python-poetry-buildpack

Here is an official Heroku Python buildpack README for some inspiration:

https://elements.heroku.com/buildpacks/heroku/heroku-buildpack-python

Hi. My heroku build crashes with the following error:

-----> Export Python version from Poetry to Heroku runtime.txt file
-----> ^3.7 not valid, please specify an exact Python version (e.g. 3.8.1) in your pyproject.toml (and thus poetry.lock)

My pyproject.toml contains a valid python version specification, following the recommendations in the Poetry documentation. According to the Poetry docs the ^ syntax should be valid.

[tool.poetry.dependencies]
python = "^3.7"

I am not exactly sure what I am doing wrong? Do I have the wrong Python version specification/different attribute name in pyproject.toml? Any pointers, why it is complaining? Thanks.

What is the right way to use this buildback, while using Heroku Multi Procfile buildpack in the same time?

I tried this way:

My Python app (and Procfile) is in back/ folder of my root repo. Obviously, I soon as I tried to deploy, I get this error:

Any idea? workaround? Thanks.

I tried to deploy my app that uses Poetry, but got an error:

-----> Patch Poetry export file
-----> Export requirements.txt from Poetry
  AttributeError
  'NoneType' object has no attribute 'to_dependency'
  at ~/.poetry/lib/poetry/packages/locker.py:234 in get_project_dependencies
      230│                     # project level dependencies take precedence
      231│                     continue
      232│ 
      233│                 # we make a copy to avoid any side-effects
    → 234│                 requirement = deepcopy(__get_locked_package(requirement).to_dependency())
      235│                 requirement._category = pkg.category
      236│ 
      237│                 if pinned_versions:
      238│                     requirement.set_constraint(
-----> Export Python version from Poetry to Heroku runtime.txt file
-----> Write 3.8.6 into runtime.txt
-----> App not compatible with buildpack: https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/python.tgz
       More info: https://devcenter.heroku.com/articles/buildpacks#detection-failure
 !     Push failed

My poetry.lock file:

[[package]]
category = "main"
description = "Fast and simple WSGI-framework for small web-applications."
name = "bottle"
optional = false
python-versions = "*"
version = "0.12.18"

[[package]]
category = "main"
description = "WSGI HTTP Server for UNIX"
name = "gunicorn"
optional = false
python-versions = ">=3.4"
version = "20.0.4"

[package.dependencies]
setuptools = ">=3.0"

[package.extras]
eventlet = ["eventlet (>=0.9.7)"]
gevent = ["gevent (>=0.13)"]
setproctitle = ["setproctitle"]
tornado = ["tornado (>=0.2)"]

[[package]]
category = "main"
description = "Python client for Redis key-value store"
name = "redis"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
version = "3.5.3"

[package.extras]
hiredis = ["hiredis (>=0.1.3)"]

[metadata]
content-hash = "184994068e190b2720ed44423cc8673e8300ceae582e90301adbe7502e598b42"
lock-version = "1.0"
python-versions = "3.8.6"

[metadata.files]
bottle = [
    {file = "bottle-0.12.18-py3-none-any.whl", hash = "sha256:43157254e88f32c6be16f8d9eb1f1d1472396a4e174ebd2bf62544854ecf37e7"},
    {file = "bottle-0.12.18.tar.gz", hash = "sha256:0819b74b145a7def225c0e83b16a4d5711fde751cd92bae467a69efce720f69e"},
]
gunicorn = [
    {file = "gunicorn-20.0.4-py2.py3-none-any.whl", hash = "sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c"},
    {file = "gunicorn-20.0.4.tar.gz", hash = "sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626"},
]
redis = [
    {file = "redis-3.5.3-py2.py3-none-any.whl", hash = "sha256:432b788c4530cfe16d8d943a09d40ca6c16149727e4afe8c2c9d5580c59d9f24"},
    {file = "redis-3.5.3.tar.gz", hash = "sha256:0e7e0cfca8660dea8b7d5cd8c4f6c5e29e11f31158c0b0ae91a397f00e5a05a2"},
]

Hello,

I have forked your repo and modified it as such as to have an action which deploys the buildpack to buildpacks.io when creating a release.

Would you be interested in merging this? I'd create a PR if yes

What are your thoughts on having this buildpack run poetry lock if it doesn't find an existing poetry.lock file?

I have dependabot running on my GitHub repo. Each time I merge a dependabot PR, it forces a cascade rebase of all other dependency PRs because of lockfile hash conflicts. It'd save me a ton of time (and CI/CD credits) if I could not commit the lockfile and just have it be generated at runtime.

Hello! It appears that Python version extraction is currently breaking if the python-versions = "x.x.x" line in poetry.lock includes a trailing space. This trailing space will be added if the Python version specification in pyproject.toml also includes a trailing space. I found this because the VSCode TOML plugin I am using adds these spaces: oovm/vscode-toml#7.

When that happens, the Heroku build fails with the following message:

-----> 3.9.11  is not valid, please specify an exact Python version (e.g. 3.8.1) in your pyproject.toml (and thus poetry.lock)

To make this more robust, I propose that we strip out spaces in this command: