frostming / pdm-packer Goto Github PK

View Code? Open in Web Editor NEW

42.0 2.0 2.0 234 KB

A PDM plugin that packs your packages into a zipapp

License: MIT License

Python 100.00%

pdm zipapp packaging

pdm-packer's Introduction

pdm-packer

A PDM plugin that packs your packages into a zipapp

Requirements

pdm-packer requires Python >=3.7

Installation

On PDM 1.6.4+, you can install the plugin directly by:

$ pdm plugin add pdm-packer

If you have installed PDM with the recommended tool pipx, add this plugin by:

$ pipx inject pdm pdm-packer

Or if you have installed PDM with pip install --user pdm, install with pip to the user site:

$ python -m pip install --user pdm-packer

Otherwise, install pdm-packer to the same place where PDM is located.

Usage

$ pdm pack [common-options] [pack-options]

Common Options:

-h, --help

show this help message and exit

-v, --verbose

-v for detailed output and -vv for more detailed

-g, --global

Use the global project, supply the project root with -p option

-p PROJECT_PATH, --project PROJECT_PATH

Specify another path as the project root, which changes the base of pyproject.toml and __pypackages__

Pack Options:

-m MAIN, --main MAIN

Specify the console script entry point for the zipapp

-o OUTPUT, --output OUTPUT

Specify the output filename. By default the file name will be inferred from the project name.

-c, --compress

Compress files with the deflate method, no compress by default

--pyc, --compile

Compile source into pyc files

--no-py

Remove the .py files in favor of .pyc files

-i INTERPRETER, --interpreter INTERPRETER

The Python interpreter path, default: the project interpreter

--exe

Create an executable file. If the output file isn't given, the file name will end with .exe(Windows) or no suffix(Posix)

Examples

# Create with default name(<project_name>.pyz) and console_script as the __main__.py
pdm pack
# Create an executable file
pdm pack --exe
# Create with custom __main__.py and filename
pdm pack -o app.pyz -m app:main

Caveats

If the result zipapp contains binaries, it can only be deployed to the platforms with the same abi, any cross-abi usage of that app might expect a failure.
Any console scripts except for what is given to --main will be lost.
The .exe file is different from what is produced by pyinstaller in the way that it doesn't embed a Python interpreter. This means you have to install a Python with exactly the same version on the deployment platform.
If you have code to run in your project, the project itself should be installed into __pypackages__ as well. Make sure you have set a project name in pyproject.toml.

About executable zipapp

By default, zipapp is created with .pyz suffix. On Windows, if you have associted .pyz files with Python program, you can run the app by double-clicking the file in the explorer. But if you create the app with --exe turn on, you can have a .exe file on Windows and an executable file on Unix-like systems, so that the app can be executed without a python command prefixing it and no matter you assoicated the file exensition properly or not.

Changelog

See CHANGELOG.md

pdm-packer's People

Contributors

Stargazers

Watchers

Forkers

pombredanne franzhaas

pdm-packer's Issues

Write bytes into the file stream instead of memory

packing pdm itself apears only to work with python 3.11, but neither 3.9 nor 3.10 on win10

Hi,

I was succesfully able to pack pdm into a zipapp. This works nicely when I use python 3.11 to execute the zipapp.

I encountered 2 problems with the exe target.
1.) I needed to use the -i command to specify the interpreter "py" which is to my understanding the recommended way to find python on windows.
2.) on the target machine python 3.11 needs to be the default selected by "py".

The issue appears to only affect sync, not lock or search.

I would like to know if there is a possibility to specify the python version which should've used, like "py -3.11" when building an exe

This is the way I create the zipapp.:


git clone https://github.com/pdm-project/pdm.git
cd pdm
git checkout 2.3.4
pdm add pdm-packer
pdm pack
pdm pack --exe -i py.exe

Any hints?

Thanks in advance,
Franz

pdm_packer:plugin: cannot import name 'cached_property' from 'pdm.utils' .

After installing packer, i get the following error message.:

Failed to load plugin pdm-packer=pdm_packer:plugin: cannot import name 'cached_property' from 'pdm.utils' .

pdm version is 2.3.2
pdm packer version is 0.4.0

any thoughts?

pdm packer cant run compiler from zipapp distributed pdm

file only works if the python module is actually stored as a file... this is not necessarily the case. When running packer from a zip app.:

How do you pack all files within project and access them?

Probably just some examples I'm missing but how do you pack and access resource files? For example my project consists of:

lib/python/.... (python modules)
run.py (run function as entrypoint)
.env
README.md
etc.

How do I access .env file for example? because now my app is failing due to dotenv.load_dotenv function

Traceback (most recent call last):
  File "/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/mariusg/Documents/Projects/infra-terraform/./app.pyz/__main__.py", line 3, in <module>
  File "/tmp/pdm-pack-rpnrz65t/lib/run.py", line 10, in run
    load_dotenv()
  File "/tmp/pdm-pack-rpnrz65t/lib/dotenv/main.py", line 331, in load_dotenv
  File "/tmp/pdm-pack-rpnrz65t/lib/dotenv/main.py", line 298, in find_dotenv
  File "/tmp/pdm-pack-rpnrz65t/lib/dotenv/main.py", line 255, in _walk_to_root
OSError: Starting path not found

run.py:

#!/usr/bin/env python3

from dotenv import load_dotenv

def run():
    # Load up .env files
    load_dotenv()

Package command:

pdm pack -m run:run -c --pyc --no-py --exe -o app.pyz

when packing pdm the version given at the stdout is not the same as the one given when doing --version

I can see the version of pdm used listed on stdout when creating the zipapp, but when running the zip app with --version, I get a different version.

To Reproduce
git checkout 2.3.3
pdm add pdm-packer
pdm pack
---------8<---------8<---------8<---------8<---------8<---------
...
[2022-12-15T20:38:56.771Z] v Install pdm 2.3.3+d20221215 successful
[2022-12-15T20:38:56.771Z]
[2022-12-15T20:38:56.771Z] All complete!
[2022-12-15T20:38:56.771Z]
[2022-12-15T20:38:56.771Z] Packages are prepared at xx
[2022-12-15T20:38:56.771Z] Creating zipapp...
[2022-12-15T20:38:57.026Z] Zipapp is generated at pdm.pyz
[2022-12-15T20:38:57.586Z] pdm.pyz --version
[2022-12-15T20:38:58.947Z] PDM, version 0.0.0+local
---------8<---------8<---------8<---------8<---------8<---------

I would have expected the version to be 2.3.3+d20221215, or 2.3.3 but not 0.0.0+local

Is this expected? What would I need to do to get the version to be 2.3.3+packedbymyorgnamehere?

Thanks in advance,
Franz

crosspack zipapps

It would be awesome if it would be possible to run

pdm pack --target=cp312-win_amd64 -o win.pyz
pdm pack --target=cp312-manylinux_x86_64 -o linux.pyz

to pack for multiple targets on the same machine...

what do you think?

Support excluding py files in the zipapp

The project files were not packaged in when packaging

This will cause python app.pyz to not run correctly when I only give the pyz package.

I checked the source code and this should happen here: https://github.com/frostming/pdm-packer/blob/main/src/pdm_packer/command.py#L100

Can I hide console window and set an icon for exe?

Like PyInstaller’s -w and -i.

Executable throws exception on numpy dependency

Describe the bug

I'm packaging up a python application that has numpy as a dependency. I package the application with pdm pack --exe -g -p ./ - when I run ./myexecutable myFunction "arg1" I get the following error

ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/Users/tpeterson/Code/ai_software/luna/.venv/bin/python"
  * The NumPy version is: "1.24.2"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

I've tried python 3.8-3.11 and have reinstalled numpy several times. numpy is imported inside of "myFunction" - other python functions I call that don't have numpy as an import succeed.

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Run command '...'
Scroll down to '...'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

System (please complete the following information):

pdm-packer version: latest
Python version: 3.8
OS: MacOS with M2 Max

Additional context
Add any other context about the problem here.

Cannot find psycopg2-binary

Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "./nina_database_manager.pyz/__main__.py", line 2, in <module>
  File "/usr/local/lib/python3.8/site-packages/manager.py", line 9, in <module>
    import psycopg2
  File "<frozen zipimport>", line 259, in load_module
  File "./nina_database_manager.pyz/psycopg2/__init__.py", line 51, in <module>
ModuleNotFoundError: No module named 'psycopg2._psycopg'

Describe the bug
psycopg2-binary is a dependency of the project, but it is not found when I try to execute the .pyz zip file.

To Reproduce
Steps to reproduce the behavior:

Add psycopg2-binary
Import psycopg2 in your app
Pack your app
Run it
See error

Expected behavior
psycopg2 should be imported as usual.

System (please complete the following information):

pdm-packer version: 0.2.1
Python version: 3.8
OS: Linux

Additional context
Other dependencies, such as ldap, seems to work as usual.

Add option to include/exclude pyc files

Latest release is incompatible with latest pdm release

Describe the bug
The latest released version of pdm-packer is not compatible with the latest version of pdm (currently 1.14.1).
This is indicated by an error message when calling pdm as well as the pdm pack not working.
Thankfully, the underlying problem is already fixed on main :-)

To Reproduce

Run pdm plugin add pdm-packer and see the installation succeed

Run pdm plugin list and see this error:

Failed to load plugin pdm-packer=pdm_packer:plugin: cannot import name 'get_architecture' from 'pdm.models.in_process' (/some/venv/pdm/lib/python3.10/site-packages/pdm/models/in_process/__init__.py)

Install pdm-packer from the latest main branch in Git instead, e.g. with pipx: pipx inject pdm "git+https://github.com/frostming/pdm-packer.git@main"
Run pdm plugin list: No error shows up

Expected behavior
The latest version of pdm-packer in PyPI should work with the latest version of pdm.

System

pdm-packer version: 0.3
pdm version: 1.14.1
Python version: 3.8, 3.10
OS: Ubuntu 20.04, macOS 12.3

Local Python environment used as shebang in pyz

Describe the bug

The shebang in my pyz file points to the build-time project venv. This won't exist on target environments.

To reproduce

Create local environment with pdm sync --dev
Build pyz file with pdm pack
Open pyz file in a text editor and inspect first line

Expected behavior

To the best of my knowledge, the universal shebang for Python 3 is:

#!/usr/bin/env python3

Instead I see:

#!/home/user/Documents/myproject/.venv/bin/python

System

$ pipx list --include-injected
...
   package pdm 2.5.2, installed using Python 3.10.4
    - pdm
    Injected Packages:
      - keyring 23.9.3
      - pdm-packer 0.6.0

$ lsb_release --all
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

The problem with git-lfs....

Unless git-lfs is installed (and configured possibly requiring extra paid services from GitHub) a plain git clone of this repo fails.
This is creating problems like @kloczek reported in pdm-project/pdm#1084

And I reckon that your position is pdm-project/pdm#1084 (comment)

Checking in binary files in a git repository is not a good practice. Because Git will store all histories in the .git directory, if the file is a binary the storage is not efficient: it can't store the delta changes like a plain text file.

I understand your point, but here this is eventually creating difficulties for any adopter and aspiring contributor. IMHO this may not be the effect you are looking for? The whole set of wheels and archives is about 3.1MB total and this are binaries but not what I would call large files.

I would suggest to consider using something else:

just copy the test wheels inside the repo... these are not super big and not evolving quickly
or use a separate repo just for the test wheels and use git modules to get it
or put these in a tarball under a release of this repo and fetch these from there

[QUESTION]. how to use it?

I created a test project

pdm init

add run.py

def run():
    print('hello')

pack

pdm pack -m run:run -o app.zip -v
Packing packages...
All packages are synced to date, nothing to do.

  All complete!
Packages are prepared at U:\Temp\pdm-pack-a64tsc6l\lib
Creating zipapp...
Traceback (most recent call last):
  File "C:\Python\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\users\user\.local\bin\pdm.exe\__main__.py", line 7, in <module>
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm\core.py", line 254, in main
    return Core().main(args)
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm\core.py", line 187, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm\core.py", line 182, in main
    f(options.project, options)
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm_packer\command.py", line 120, in handle
    zipapp.create_archive(
  File "C:\Python\lib\zipapp.py", line 111, in create_archive
    raise ZipAppError("Source does not exist")
zipapp.ZipAppError: Source does not exist

Cannot pack if there are no packages? - pdm add colorama
(why not use without packages)
and try again

pdm pack -m run:run -o app.zip -v
Packing packages...
Synchronizing working set with lock file: 1 to add, 0 to update, 0 to remove

unearth: Downloading <Link https://files.pythonhosted.org/packages/77/8b/7550e87b2d308a1b711725dfaddc19c695f8c5fa413c640b2be01662f4e6/colorama-0.4.5-py2.py3-none-any.whl (from None)> to U:\Temp\pdm-build-42ueqwgg\colorama-0.4.5-py2.py3-none-any.whl
  v Install colorama 0.4.5 successful

  All complete!
Packages are prepared at U:\Temp\pdm-pack-1trs0fvn\lib
Creating zipapp...
Zipapp is generated at app.zip

app.zip structure

        colorama
        __main__.py

no run.py file

pdm pack --help
Usage: pdm pack [-h] [-v] [-g] [-p PROJECT_PATH] [-m MAIN] [-o OUTPUT] [-c] [--pyc] [--no-py] [-i INTERPRETER] [--exe]

Pack the packages into a zipapp

only packages without project files? then why need -m run:run?

pdm info && pdm info --env

pdm info && pdm info --env
PDM version:
  2.0.0
Python Interpreter:
  C:\Python\python.exe (3.10)
Project Root:
  z:/testapp
Project Packages:
  z:\testapp\__pypackages__\3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.5",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19044",
  "python_full_version": "3.10.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "win32"
}