pypa / installer Goto Github PK

View Code? Open in Web Editor NEW

111.0 16.0 51.0 716 KB

A low-level library for installing from a Python wheel distribution.

Home Page: https://installer.readthedocs.io/

License: MIT License

Python 100.00%

wheel python

installer's People

Contributors

Stargazers

Watchers

installer's Issues

Release with CLI

Hi, I’d like to use the CLI. Could you please publish a release containing it?

Specifying destdir as empty string leads to no scripts installed

Hi! Recently I have been experimenting a bit with build and installer in the context of CI pipelines.

I noticed that if python -m installer --destdir="" <wheel> at least scripts (but maybe also the rest of the wheel contents) are not installed to / but likely somewhere else.
From a user perspective I would assume that providing empty string would behave the same way as not providing --destdir at all (in which case files are properly installed to the root filesystem).

When looking at e.g. how this is handled in a Makefile which allows overriding of the destination using the DESTDIR environment variable we can observe that the following code snippet follows my above assumption:

PREFIX ?= /usr/local

install:
    install -vDm 644 some/file -t "$(DESTDIR)/$(PREFIX)/share/foo/"

FileExistsError with pylint 1.8.3

If I try and use installer to install the pylint 1.8.3 wheel I just built:

$ rm -rf foo
$ python3 -minstaller --destdir foo pylint-1.8.3-py2.py3-none-any.whl
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/ross/Mess/installer/src/installer/__main__.py", line 84, in <module>
    _main(sys.argv[1:], "python -m installer")
  File "/home/ross/Mess/installer/src/installer/__main__.py", line 80, in _main
    installer.install(source, destination, {})
  File "/home/ross/Mess/installer/src/installer/_core.py", line 109, in install
    record = destination.write_file(
  File "/home/ross/Mess/installer/src/installer/destinations.py", line 203, in write_file
    return self.write_to_fs(
  File "/home/ross/Mess/installer/src/installer/destinations.py", line 167, in write_to_fs
    raise FileExistsError(message)
FileExistsError: File already exists: foo/usr/bin/epylint

(yes, this is an old release)

This is quite likely a pylint bug, especially as I fixed it by upgrading to the latest release, but pip will install the wheel fine so possibly installer should be doing something better to handle wheels like this.

Support for --prefix

Add support for --prefix. Note this is not the same as --destdir which was discussed in https://github.com/pradyunsg/installer/issues/58 and has been added.

Discussion on --prefix in https://github.com/pradyunsg/installer/pull/94#issuecomment-1013876999 and https://github.com/pradyunsg/installer/pull/66. I don't see anyway around needing --prefix in Nixpkgs.

A sysconfig scheme is static and doesn't provide the flexibility we need, that is, being able to install every package in its own prefix.

Alternatively, I could imagine reading a scheme from say a json or toml file when installing.

Something that could work for us:

var = {"installed_base": "$out", "base": "$out", "platbase": "$out", "installed_platbase": "$out"}
# Note there is no `sysconfig.get_default_scheme()`
sysconfig._expand_vars("posix_prefix", var)

Scope of this project and API design

This project is born out of https://discuss.python.org/t/3869/. Let me know here if you participated there and want push access.

As a first step, we should figure out what exactly do we want this project to be. :)

Add support for pycompile (as an opt-out)

Based off https://github.com/pradyunsg/installer/pull/66#discussion_r763810443

Misusage of the `Record` constructor

When implementing #35 I hit the probably common footgun of passing a string to Record(...) instead of a Hash instance.

I added an assert to track down the origin of the issue and was surprised to find out the was a lot of breakage, other than mine.

diff --git a/src/installer/records.py b/src/installer/records.py
index 4fbd415..dc21537 100644
--- a/src/installer/records.py
+++ b/src/installer/records.py
@@ -105,6 +105,7 @@ class RecordEntry(object):
         :param size: file's size in bytes
         """
         super(RecordEntry, self).__init__()
+        assert isinstance(hash_, Hash)

         self.path = path
         self.hash_ = hash_

After fixing my code, I get the following when running the test suite.

platform linux -- Python 3.9.1, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /home/anubis/git/installer
plugins: forked-1.3.0, xdist-2.2.0, cov-2.10.1
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
gw0 [74] / gw1 [74] / gw2 [74] / gw3 [74] / gw4 [74] / gw5 [74] / gw6 [74] / gw7 [74]

.....................F..FF.F.....F........F...F.................F.....F. [ 95%]
F.                                                                       [100%]
=================================== FAILURES ===================================
__________ TestSchemeDictionaryDestination.test_finalize_write_record __________
[gw6] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_destinations.TestSchemeDictionaryDestination object at 0x7f1e33d62c40>
destination = <installer.destinations.SchemeDictionaryDestination object at 0x7f1e33d752b0>

    def test_finalize_write_record(self, destination):
        records = [
            destination.write_file("data", "my_data1.bin", io.BytesIO(b"my data 1")),
            destination.write_file("data", "my_data2.bin", io.BytesIO(b"my data 2")),
            destination.write_file("data", "my_data3.bin", io.BytesIO(b"my data 3")),
            destination.write_file("scripts", "my_script", io.BytesIO(b"my script")),
            destination.write_file(
                "scripts", "my_script2", io.BytesIO(b"#!python\nmy script")
            ),
            destination.write_script(
                "my_entrypoint", "my_module", "my_function", "console"
            ),
        ]
    
>       destination.finalize_installation("purelib", "RECORD", records)

tests/test_destinations.py:107: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/destinations.py:113: in finalize_installation
    record = RecordEntry("RECORD", None, None)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7f1e33d75970>
path = 'RECORD', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
_________________ TestRecordEntry.test_valid_elements[a.py--] __________________
[gw3] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestRecordEntry object at 0x7f4829b0c490>, path = 'a.py'
hash_ = '', size = ''

    @pytest.mark.parametrize(
        "path, hash_, size",
        [
            ("a.py", "", ""),
            ("a.py", "", "3144"),
            ("a.py", "sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\\_pNh2yI", ""),
            ("a.py", "sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\\_pNh2yI", "3144"),
        ],
    )
    def test_valid_elements(self, path, hash_, size):
>       RecordEntry.from_elements(path, hash_, size)

tests/test_records.py:101: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7f4829b0c070>
path = 'a.py', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
_______________ TestRecordEntry.test_valid_elements[a.py--3144] ________________
[gw4] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestRecordEntry object at 0x7fd644679400>, path = 'a.py'
hash_ = '', size = '3144'

    @pytest.mark.parametrize(
        "path, hash_, size",
        [
            ("a.py", "", ""),
            ("a.py", "", "3144"),
            ("a.py", "sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\\_pNh2yI", ""),
            ("a.py", "sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\\_pNh2yI", "3144"),
        ],
    )
    def test_valid_elements(self, path, hash_, size):
>       RecordEntry.from_elements(path, hash_, size)

tests/test_records.py:101: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fd644679700>
path = 'a.py', hash_ = None, size = 3144

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
_ TestRecordEntry.test_populates_attributes_correctly[elements5-test1\n-True] __
[gw5] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestRecordEntry object at 0x7fcb28978130>
elements = ('test6.py', None, None), data = b'test1\n', passes_validation = True

    @pytest.mark.parametrize(("elements", "data", "passes_validation"), SAMPLE_RECORDS)
    def test_populates_attributes_correctly(self, elements, data, passes_validation):
        path, hash_string, size = elements
    
>       record = RecordEntry.from_elements(path, hash_string, size)

tests/test_records.py:107: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fcb289897f0>
path = 'test6.py', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
___________ TestRecordEntry.test_validation[elements5-test1\n-True] ____________
[gw1] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestRecordEntry object at 0x7fc7554a96d0>
elements = ('test6.py', None, None), data = b'test1\n', passes_validation = True

    @pytest.mark.parametrize(("elements", "data", "passes_validation"), SAMPLE_RECORDS)
    def test_validation(self, elements, data, passes_validation):
>       record = RecordEntry.from_elements(*elements)

tests/test_records.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fc7554a95e0>
path = 'test6.py', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
______ TestRecordEntry.test_string_representation[elements5-test1\n-True] ______
[gw7] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestRecordEntry object at 0x7fd3f8c3d670>
elements = ('test6.py', None, None), data = b'test1\n', passes_validation = True

    @pytest.mark.parametrize(("elements", "data", "passes_validation"), SAMPLE_RECORDS)
    def test_string_representation(self, elements, data, passes_validation):
>       record = RecordEntry.from_elements(*elements)

tests/test_records.py:124: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fd3f8c3d6a0>
path = 'test6.py', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
_ TestParseRecordFile.test_accepts_all_kinds_of_iterables[record_simple_list] __
[gw2] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestParseRecordFile object at 0x7fa168a29ac0>
record_input = ['file.py,sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\\_pNh2yI,3144', 'distribution-1.0.dist-info/RECORD,,']

    @pytest.mark.parametrize(
        "record_input",
        ["record_simple_list", "record_simple_iter", "record_simple_file"],
        indirect=True,
    )
    def test_accepts_all_kinds_of_iterables(self, record_input):
        """Should accepts any iterable, e.g. container, iterator, or file object."""
>       records = list(parse_record_file(record_input))

tests/test_records.py:143: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:204: in parse_record_file
    record = RecordEntry.from_elements(elements[0], elements[1], elements[2])
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fa168a29f40>
path = 'distribution-1.0.dist-info/RECORD', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
_ TestParseRecordFile.test_accepts_all_kinds_of_iterables[record_simple_iter] __
[gw7] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestParseRecordFile object at 0x7fd3f8be87c0>
record_input = <list_iterator object at 0x7fd3f8be8fd0>

    @pytest.mark.parametrize(
        "record_input",
        ["record_simple_list", "record_simple_iter", "record_simple_file"],
        indirect=True,
    )
    def test_accepts_all_kinds_of_iterables(self, record_input):
        """Should accepts any iterable, e.g. container, iterator, or file object."""
>       records = list(parse_record_file(record_input))

tests/test_records.py:143: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:204: in parse_record_file
    record = RecordEntry.from_elements(elements[0], elements[1], elements[2])
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fd3f8be8d00>
path = 'distribution-1.0.dist-info/RECORD', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
_ TestParseRecordFile.test_accepts_all_kinds_of_iterables[record_simple_file] __
[gw2] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_records.TestParseRecordFile object at 0x7fa1689dae50>
record_input = <_io.TextIOWrapper name='/tmp/pytest-of-anubis/pytest-84/popen-gw2/test_accepts_all_kinds_of_iter0/RECORD' mode='r' encoding='UTF-8'>

    @pytest.mark.parametrize(
        "record_input",
        ["record_simple_list", "record_simple_iter", "record_simple_file"],
        indirect=True,
    )
    def test_accepts_all_kinds_of_iterables(self, record_input):
        """Should accepts any iterable, e.g. container, iterator, or file object."""
>       records = list(parse_record_file(record_input))

tests/test_records.py:143: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:204: in parse_record_file
    record = RecordEntry.from_elements(elements[0], elements[1], elements[2])
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fa168a29760>
path = 'distribution-1.0.dist-info/RECORD', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError
______________________ TestConstructRecord.test_construct ______________________
[gw5] linux -- Python 3.9.1 /home/anubis/git/installer/.nox/test-3-9/bin/python

self = <test_utils.TestConstructRecord object at 0x7fcb289439a0>

    def test_construct(self):
>       records = [
            RecordEntry.from_elements(*elements) for elements, _, _ in SAMPLE_RECORDS
        ]

tests/test_utils.py:154: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_utils.py:155: in <listcomp>
    RecordEntry.from_elements(*elements) for elements, _, _ in SAMPLE_RECORDS
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:187: in from_elements
    return cls(path=path, hash_=hash_value, size=size_value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'RecordEntry' object has no attribute 'path'") raised in repr()] RecordEntry object at 0x7fcb28943820>
path = 'test6.py', hash_ = None, size = None

    def __init__(self, path, hash_, size):
        # type: (FSPath, Optional[Hash], Optional[int]) -> None
        r"""Construct a ``RecordEntry`` object.
    
        Most consumers should use :py:meth:`RecordEntry.from_elements`, since no
        validation or parsing is performed by this constructor.
    
        :param path: file's path
        :param hash\_: hash of the file's contents
        :param size: file's size in bytes
        """
        super(RecordEntry, self).__init__()
>       assert isinstance(hash_, Hash)
E       AssertionError

.nox/test-3-9/lib/python3.9/site-packages/installer/records.py:108: AssertionError

----------- coverage: platform linux, python 3.9.1-final-0 -----------
Name                                                                                 Stmts   Miss  Cover   Missing
------------------------------------------------------------------------------------------------------------------
.nox/test-3-9/lib/python3.9/site-packages/installer/__init__.py                          1      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/_compat/__init__.py                  0      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/_compat/importlib_resources.py       3      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/_compat/typing.py                    4      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/_scripts/__init__.py                 0      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/destinations.py                     39      1    97%   114
.nox/test-3-9/lib/python3.9/site-packages/installer/records.py                          67      1    99%   142
.nox/test-3-9/lib/python3.9/site-packages/installer/scripts.py                          53      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/sources.py                          19      0   100%
.nox/test-3-9/lib/python3.9/site-packages/installer/utils.py                            53      5    91%   151-155
------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                  239      7    97%
Coverage HTML written to dir /home/anubis/git/installer/.nox/test-3-9/htmlcov

FAIL Required test coverage of 100% not reached. Total coverage: 97.07%
=========================== short test summary info ============================
FAILED tests/test_destinations.py::TestSchemeDictionaryDestination::test_finalize_write_record
FAILED tests/test_records.py::TestRecordEntry::test_valid_elements[a.py--] - ...
FAILED tests/test_records.py::TestRecordEntry::test_valid_elements[a.py--3144]
FAILED tests/test_records.py::TestRecordEntry::test_populates_attributes_correctly[elements5-test1\n-True]
FAILED tests/test_records.py::TestRecordEntry::test_validation[elements5-test1\n-True]
FAILED tests/test_records.py::TestRecordEntry::test_string_representation[elements5-test1\n-True]
FAILED tests/test_records.py::TestParseRecordFile::test_accepts_all_kinds_of_iterables[record_simple_list]
FAILED tests/test_records.py::TestParseRecordFile::test_accepts_all_kinds_of_iterables[record_simple_iter]
FAILED tests/test_records.py::TestParseRecordFile::test_accepts_all_kinds_of_iterables[record_simple_file]
FAILED tests/test_utils.py::TestConstructRecord::test_construct - AssertionError
======================== 10 failed, 64 passed in 0.99s =========================

We should fix this and maybe consider actually adding an assert there to avoid people some trouble.

Make `SchemeDictionaryDestination._write_file` public

Hey, I am implementing a CLI and need to have a destdir option that allows me to relocate the install root to an arbitrary path. This is used when building distro packages for example, because we want to put the files in the package directory instead of on the system. This analogous to the setup.py install --root option.

Would it be possible to make SchemeDictionaryDestination._write_file public so that I can simply overwrite it, instead of having to overwrite write_file and write_script.
I recommend renaming it to write_to_fs, what do you think?

Dropping support for Python 3.6

Is there any good reason to maintain support for Python < 3.7?

Python 3.6 will be EoL this year, and everything before that is definitely EoL. Given that I'm definitely dropping EoL versions in early June (see #42), I want to take the opportunity and modernise this codebase to 3.7+ and roughly follow a 6-months-before-Python-EoL approach.

Expose a utility function to use for sysconfig/distutils paths+scheme handling like pip

xref: pypa/pip#11585

Basically, the idea is to mimic the logic in pip's locations subpackage, and expose that via a single function to end users.

Can't install wheels with non-normalized names

Names are parsed from the distribution filename, but never normalized.

If a distribution has a non-normalized filename, like Quart-0.17.0-py3-none-any.whl, then installation fails:

  File "/usr/bin/pybuild", line 367, in main
    run(func, i, version, c)
  File "/usr/bin/pybuild", line 317, in run
    result = func(context, args)
  File "/usr/share/dh-python/dhpython/build/plugin_pyproject.py", line 102, in build
    self.build_step2(context, args)
  File "/usr/share/dh-python/dhpython/build/plugin_pyproject.py", line 151, in build_step2
    install(
  File "/usr/lib/python3/dist-packages/installer/_core.py", line 77, in install
    root_scheme = _process_WHEEL_file(source)
  File "/usr/lib/python3/dist-packages/installer/_core.py", line 21, in _process_WHEEL_file
    stream = source.read_dist_info("WHEEL")
  File "/usr/lib/python3/dist-packages/installer/sources.py", line 139, in read_dist_info
    return self._zipfile.read(path).decode("utf-8")
  File "/usr/lib/python3.10/zipfile.py", line 1475, in read
    with self.open(name, "r", pwd) as fp:
  File "/usr/lib/python3.10/zipfile.py", line 1514, in open
    zinfo = self.getinfo(name)
  File "/usr/lib/python3.10/zipfile.py", line 1441, in getinfo
    raise KeyError(
KeyError: "There is no item named 'Quart-0.17.0.dist-info/WHEEL' in the archive"

From: https://bugs.debian.org/1008606
Part of the bigger issue in #97.

Cut a release with the entry point handling fix

Apology for the rudeness of opening an issue for a request. But it is just following #63

Update the `simple_launcher` script wrappers

These have been improved since the last update.

Handle wheels whose RECORD file doesn't match

I understand it is the packagers' fault that released a wheel with a mismatching RECORD file. But since pip isn't that strict, people will keep complaining "I can't install xxx but pip works fine." At least installer could provide an option to skip the RECORD validation.

Some examples:

selenium==4.1.0
catboost==1.0.4

The errors are like:

File "C:\Users\user\AppData\Roaming\pdm\venv\lib\site-packages\pdm\installers\installers.py", line 280, in _install_wheel
  File "C:\Users\user\AppData\Roaming\pdm\venv\lib\site-packages\installer\sources.py", line 170, in get_contents
    assert record is not None, "In {}, {} is not mentioned in RECORD".format(
AssertionError: In C:\Users\user\AppData\Local\Temp\pip-unpack-a3gqvydj\catboost-1.0.4-cp39-none-win_amd64.whl, catboost/core.py is not mentioned in RECORD

The executable bit is lost for scripts in data directory

installer seems only to copy streams and won't copy the executable bit when installing

PyQt5_Qt5 5.15.2 fails to install: PyQt5_Qt5-5.15.2.dist-info/RECORD is not mentioned in RECORD

The wheel file for PyQt5_Qt5 version 5.15.2 on PyPi has a RECORD file that lists itself at a a path of PyQt5_Qt5-5.15.2.dist-info\RECORD. It is my understanding that 5d1bab4 should have fixed this issue, but perhaps there are other places where the path is being handled as per PEP376 instead of the Core Metadata specs.

Both installer 0.5.1 and the latest main branch were tested.

Make follow up improvements to `WheelSource.dist_info_dir`

This is a follow up to #137.

Instead of hasattr dynamicism, use an _dist_info_dir: Optional[str] attribute.
Replace assert with rich/proper error objects that can be introspected.

Support for alternative installation schemes

From using this in Linux, it seems like the installation scheme used here is always "posix_prefix." I think it would be helpful to have the option to use "posix_user" too!

Option to set interpreter to use

I'm looking at using installer to install wheels in our distribution. At runtime sys.executable is a special python wrapper that we use, but we don't want to use that on the target as it doesn't exist.

I think adding a simple --interpreter option should be sufficient for this use-case to set the interpreter parameter on SchemeDictionaryDestination, leaving the default as sys.executable.

data_dir heuristics

Hi,

i came here via poetry which has started depending on this lib as of the recently released 1.4

I'm not sure if this is a bug, or if pip is too lenient, but thought worth asking. I have a (malformed?) wheel where the data-dir uses different normalization to the package, causing this library to fail to find it.

pip (afict) uses a heuristic based on the name ending https://github.com/pypa/pip/blob/main/src/pip/_internal/operations/install/wheel.py#L537 (which looks similar to what was introduced here ed47a74 for finding the dist-info folder.

maybe

installer/src/installer/_core.py

Line 43 in ed47a74

if posixpath.commonprefix([data_dir, path]) != data_dir:

could be replaced by a similar heuristic instead of the full name comparison?

i'm not very familiar with python packaging so i may be missing lots of context. if this is a reasonable suggestion however, i'm happy to try to make a patch

Missing version git tags

https://pypi.org/project/installer/ shows that latest version is 0.1.1 but here there is no any verion git tags.

FileExistsError on package reinstall/update

Some of our development workflows typically expect that one can upgrade or reinstall a package installed in site-packages, however when one tries to do this with installer we hit a FileExistsError error.

For example:

>>> host-python-tomli 2.0.1 Installing to host directory
(cd /home/buildroot/buildroot/output/build/host-python-tomli-2.0.1//; PATH="/home/buildroot/buildroot/output/host/bin:/home/buildroot/buildroot/output/host/sbin:/home/buildroot/bin:/home/buildroot/.local/bin:/home/buildroot/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" PYTHONNOUSERSITE=1 PATH="/home/buildroot/buildroot/output/host/bin:/home/buildroot/buildroot/output/host/sbin:/home/buildroot/bin:/home/buildroot/.local/bin:/home/buildroot/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" PKG_CONFIG="/home/buildroot/buildroot/output/host/bin/pkg-config" PKG_CONFIG_SYSROOT_DIR="/" PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_LIBDIR="/home/buildroot/buildroot/output/host/lib/pkgconfig:/home/buildroot/buildroot/output/host/share/pkgconfig" AR="/usr/bin/ar" AS="/usr/bin/as" LD="/usr/bin/ld" NM="/usr/bin/nm" CC="/usr/bin/gcc" GCC="/usr/bin/gcc" CXX="/usr/bin/g++" CPP="/usr/bin/cpp" OBJCOPY="/usr/bin/objcopy" RANLIB="/usr/bin/ranlib" CPPFLAGS="-I/home/buildroot/buildroot/output/host/include" CFLAGS="-O2 -I/home/buildroot/buildroot/output/host/include" CXXFLAGS="-O2 -I/home/buildroot/buildroot/output/host/include" LDFLAGS="-L/home/buildroot/buildroot/output/host/lib -Wl,-rpath,/home/buildroot/buildroot/output/host/lib" INTLTOOL_PERL=/usr/bin/perl  /home/buildroot/buildroot/output/host/bin/python -m installer dist/* )
Traceback (most recent call last):
  File "/home/buildroot/buildroot/output/host/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/buildroot/buildroot/output/host/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/__main__.py", line 85, in <module>
    _main(sys.argv[1:], "python -m installer")
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/__main__.py", line 81, in _main
    installer.install(source, destination, {})
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/_core.py", line 109, in install
    record = destination.write_file(
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 207, in write_file
    return self.write_to_fs(scheme, path_, stream, is_executable)
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 167, in write_to_fs
    raise FileExistsError(message)
FileExistsError: File already exists: /home/buildroot/buildroot/output/host/lib/python3.10/site-packages/tomli/__init__.py

I'm not sure what the correct way to handle this is, I assume the old version should be automatically uninstalled(or there should be a flag to enable this behavior) and then the new version installed if there is an existing installation present?

Entry point scripts don't get byte-compiled

Should entrypoint scripts get byte compiled? Currently if I install eg wheel, it byte-compiles the module but not the wheel entrypoint in bindir.

Support scheme specific console scripts

Currently, the console scripts will run with the site module imported, this is what we want in a general case, but there are some cases where that may not be. As part of the distutils deprecation, we are planning to add a officially supported way for Python distributor to specify custom schemes, like Debian's dist-packages. This would allow distributors to install their packages outside site-packages, to avoid conflicts. It would be really helpful if we could also generate console scripts that will only use specific schemes, preventing pip install/uninstall from shadowing packages that distro scripts expect to be there. Overall, I think this would provide a better UX for users, as it would make their system significantly harder to break.
As part of this mechanism, I would propose adding a way to easily achieve what I am asking here.

Would this be something we could maybe consider supporting? Preferabily in the CLI also.

Implement `installer.sources.WheelFile`

I can't see in the code the WheelFile class which the docs landing page mentions - is that still to come?

Originally posted by @takluyver in https://github.com/pradyunsg/installer/issues/1#issuecomment-827412319

Yes. This is the last big remaining TODO as far as I can tell.

scripts.py utilizes the deprecated read_binary function from importlib.resources.

When building for Fedora I noticed a deprecation warning:

DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.

Installer compiles bytecode for wrong python version

I don't know if this is a valid use case or out of scope for this project but since it's possible to install in another environment one might expect that it is also possible to compile bytecode for the target environment.

In the following example installer lives in a Python 3.9 environment and installs tomli into a Python 3.11 environment. So far so good. However, it compiles bytecode for its own (Python 3.9) environment instead of the target (Python 3.11) environment. (Probably, it's not even possible to compile for the target environment without creating a subprocess to run the target interpreter?)

This issue was originally posted as a side note in python-poetry/poetry#7639. Before implementing our own solution, I just wanted to know if this will probably remain a known limitation of installer or if anyone has an idea if/how it could be supported by installer itself.

$ virtualenv -p 3.11 .venv11
$ virtualenv -p 3.9 .venv9
$ .venv9/bin/python -m pip install installer
$ .venv9/bin/python install_and_compile.py
$ ls .venv11/lib/python3.11/site-packages/tomli/__pycache__
__init__.cpython-39.pyc  _parser.cpython-39.pyc  _re.cpython-39.pyc  _types.cpython-39.pyc

install_and_compile.py

import json
import subprocess

from installer import install
from installer.destinations import SchemeDictionaryDestination
from installer.sources import WheelFile

scheme_dict_oneliner = (
    "import sysconfig; import json; print(json.dumps(sysconfig.get_paths()))"
)

scheme_dict = json.loads(
    subprocess.check_output([".venv11/bin/python", "-c", scheme_dict_oneliner]).decode()
)

destination = SchemeDictionaryDestination(
    scheme_dict,
    interpreter=".venv11/bin/python",
    script_kind="posix",
    bytecode_optimization_levels=(0,)
)

with WheelFile.open("tomli-2.0.1-py3-none-any.whl") as source:
    install(
        source=source,
        destination=destination,
        additional_metadata={
            "INSTALLER": b"amazing-installer 0.1.0",
        },
    )

Always normalize RECORD files to forward-slash when writing

We should unconditionally do path.replace("\\", "/") in RecordEntry.to_row, since we might read a path from a Windows-generated RECORD (or on Windows) and need to normalise it to use / unconditionally.

Record relative paths

Issue description

At present, the base path is written in the RECORDS no matter what scheme it belongs to. This may become a problem for schemes other than root_scheme, the uninstaller can't find the file location corresponding to that record so the file isn't picked for removal. For example, a file under data/scripts will be installed to scripts scheme` but won't be uninstalled correctly.

Minimal reproducible example:

destination = SchemeDictionaryDestination(
    {"purelib": "purelib", "scripts": "scripts"},
    interpreter=sys.executable,
    script_kind="posix",
)

with WheelFile.open("jmespath-0.10.0-py2.py3-none-any.whl") as source:
    install(
        source=source,
        destination=destination,
        # Additional metadata that is generated by the installation tool.
        additional_metadata={
            "INSTALLER": b"amazing-installer 0.1.0",
        },
    )

(The wheel can be got from PyPI)

Actual RECORDS

jp.py,<hash>,<size>
...

Expected RECORDS

../bin/jp.py,<hash>,<size>
...

Possible solution

I don't know what approach you prefer to fix this issue. So I would like to raise the issue first. Here are some approaches(or more to be complemented).

Make SchemeDictionaryDestination aware of the root scheme and produces RecordEntrys with calculated relative path.
Carry scheme with RecordEntry and calculate the relative path in SchemeDictionaryDestination.finalize_installation
Store absolute path in RecordEntry and calculate the relative path in SchemeDictionaryDestination.finalize_installation

Zipinfo Directories

Whilst trying to solve a problem with installing torch using PDM I came across #69. After reading the code I realized that the change provided makes the error message more verbose but doesn't solve the issue for the torch package.

The problem seems to be that some whl files, when looping over self._zipfile.infolist() include the directories as well as the files which is what causes that function to fail because the RECORD file doesn't have information about directories. eg:

Torch (Directories as part of the list):

torch/
torch/_deploy.py
torch/functional.py
torch/onnx/
torch/onnx/symbolic_opset9.py
torch/onnx/symbolic_caffe2.py
torch/onnx/symbolic_helper.py
torch/onnx/symbolic_opset12.py
torch/onnx/symbolic_registry.py

Colorama (Files only):

colorama/__init__.py
colorama/ansi.py
colorama/ansitowin32.py
colorama/initialise.py
colorama/win32.py
colorama/winterm.py
colorama-0.4.4.dist-info/LICENSE.txt
colorama-0.4.4.dist-info/METADATA
colorama-0.4.4.dist-info/WHEEL
colorama-0.4.4.dist-info/top_level.txt
colorama-0.4.4.dist-info/RECORD

Dropping support for EoL Python versions

So... I don't want to be writing Python 2 compatible code anymore. It has been a big part of why I haven't been spending much energy on this project.

My plan right now is that I'll maintain Python 2 compatibility till the end of May, and cut a final Python 2 compatible release then.

If there's been enough feedback and testing to make me comfortable that things are stable by then, then that would be a 1.x.x release. Otherwise, a 0.x.0.betaX will be the first non-Python 2 release.

Regardless, in early June this year, I'm gonna purge Python 2 related compatibility stuff from this codebase, since it's generally been exceedingly painful to work with. I'm happy to accept PRs fixing stuff that's broken on Python 2 till then, but other than that, I'm personally not motivated to do anything for Python 2 now.

Provide sample code for a very basic client

I'm finding it hard to understand how the library would be used in practice. In the longer term, this will be covered by the documentation. But for early adopters, having an example of the intended use would be really helpful.

Could an examples directory be added with a very basic client that takes a wheel file and extracts it? I'd go for:

Wheel filename given on the command line
Options for destinations to unpack to (I'd specifically like the example to not use sysconfig or anything that hides the question of "what locations do I need to support" behind what a given Python installation provides).

Including notes about what isn't supported, compared to something like pip (hash checking? script wrappers?) with a brief note describing how clients would typically implement these, would give a much better feel for the scope of the library, as well.

(My personal concern here is that writing script wrappers is out of scope, but it will be hard for clients to implement without assistance to identify what scripts need writing. I'd like to have sample code that offers a concrete example of how that would play out in reality).

Drop outdated link to importlib_metadata in comment

Is this comment still up-to-date @pradyunsg? The linked target is entirely different from the code below.

Originally posted by @uranusjr in https://github.com/pradyunsg/installer/pull/63#discussion_r655209531

Want ability to separate the "generate/build" step from the "install" step

Some distributions, like Arch Linux, want us to have separate build and install steps. This means that .pyc files and entrypoint scripts would be built in a separate step. For this I propose having a command to do this. We could call it something like "build", "build cache", "generate cache", etc?

It would look like this

python -m installer --build something.whl

Then to install

python -m installer something.whl

Maybe we could also add an option to fail on missing pre-built files, so that we can enforce it inside the distribution workflows.

It would also be great if the command was accessible as an api, so that I could call it directly in python-build, making the final workflow for PEP517 projects something like:

(strictly build)
python -m build

(strictly install)
python -m install --destdir path/to/folder dist/something.whl

Expose some helper functions in WheelSource

When I extend installer to support custom installation logic, I found the following methods or attributes very useful and worth exposing as a public method:

root_scheme. Currently retrieved by an unexported function _process_WHEEL_file, would be good to make it a property of WheelSource
_determine_scheme(path, source, root_scheme). Currently, it is an unexported function, would be good to make it a method of WheelSource: WheelSource.determine_scheme(path). This, IMO, can achieve high cohesion.

RECORD file is not encoded correctly if the path contains commas

Example package: pyqtgraph==0.12.4

The RECORD generated by installer looks like:

...
pyqtgraph/colors/maps/CC-BY license - applies to CET color map data.txt,sha256=c97deeeca4ae375a0334bc7f7af5f707aabfcec959c53c783b9f8771d28fd5b3,14990
pyqtgraph/colors/maps/CC0 legal code - applies to virids, magma, plasma, inferno and cividis.txt,sha256=a2010f343487d3f7618affe54f789f5487602331c0a8d03f49e9a7c547cf0499,7048
...

This results in an error when the uninstaller tries to parse this file.

Proposed Solution

Change the API RecordEntry.to_line() to RecordEntry.to_row() -> tuple[str, str, str]. And construct RECORD using csv.writer

Provide a CLI

As mentioned in #1 and the description, it would be useful to have a CLI. (Creating this so people can get notified)

My use case for this project is to replace the way Linux distribution packages are packaged. As Thomas said in https://github.com/pradyunsg/installer/issues/1#issuecomment-622970860, people currently use error prone command lines like this:

python -m pip install --root $RPM_BUILD_ROOT --no-deps --disable-pip-version-check --progress-bar off --verbose --ignore-installed --no-warn-script-location pyproject-wheeldir/*.whl

Would be great to have a very simple command line that does exactly that: Given a

Python binary with version $pythonver,
$root directory, and
wheel file

it installs the

package and dist info to $root/usr/lib/python$pythonver/site-packages/,
headers to $root/usr/include/,
scripts to $root/usr/bin/, and
data to $root (I assume)

KeyError instead of AssertionError

PDM uses installer internally. When installing torch 1.5.1+cpu, I get this error:

Install torch 1.5.1+cpu failed
Error occurs: 
Traceback (most recent call last):
  File "/usr/lib64/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 190, in install_candidate
    self.manager.install(can)
  File "/usr/local/lib/python3.8/site-packages/pdm/installers/manager.py", line 31, in install
    installer(candidate.build(), self.environment, candidate.direct_url())
  File "/usr/local/lib/python3.8/site-packages/pdm/installers/installers.py", line 74, in install_wheel
    _install_wheel(
  File "/usr/local/lib/python3.8/site-packages/pdm/installers/installers.py", line 173, in _install_wheel
    for record_elements, stream in source.get_contents():
  File "/usr/local/lib/python3.8/site-packages/installer/sources.py", line 165, in get_contents
    record = record_mapping.pop(item.filename)
KeyError: 'caffe2/'

I think you simply forgot the default argument of pop. I'll open a PR.

See pdm-project/pdm#636

Always set owner's read bit, in files written

on sys.platform=="emscripten" where oct(os.umask(0)) == 0o777 by default instead of 0o22
at

installer/src/installer/utils.py

Line 251 in 70ae05b

os.chmod(path, (0o777 & ~_current_umask() | 0o111))

the execute bit is set but the user read one is lost making it impossible to preload libraries (.so). ( maybe make_file_loadable ? that set at least u+rx )

emscripten-core/emscripten#17269

Resolve normalised vs original distribution names

At present, the distribution name comes from the wheel filename, which can be normalised according to the new rule (PEP 503 normalisation, but with _ in place of -) or according to an older rule (any character except A-Za-z0-9. is converted to _).

If we want the original distribution name for anything, we should get that from wheel metadata. If we want the normalised name, we should normalise it ourselves for consistency.

Move project under PyPA?

With the latest announcement on the packaging Discourse encouraging redistributors to use installer, should it perhaps be moved under the PyPA as the "blessed" tool for installing wheels sans pip? There are no contenders in this space and installer is no doubt a high-quality package that the packaging community would benefit from.

0.3.0 release

I'll be cutting this over the wekeend, with everything on main and maybe #68 fixed.

Notably, as per #42, this will be the last release supporting EoL Python versions and soon-to-be-EoL Python 3.6 as well.

Bring over nuances and "special" handling of edge cases from pip's wheel install logic

This is basically about bringing over the nuances in https://github.com/pypa/pip/blob/22.3.1/src/pip/_internal/operations/install/wheel.py over to this project.

Most of this should end up in the core, rather than the source/destination.

Implement foundational validation logic

This would be something like:

installer.validators.validate_all_files_match_record(source: WheelSource) -> None
installer.validators.validate_record(source: WheelSource) -> None

These raise installer.validators.ValidationError if the source is not a valid wheel.

Make follow-up improvements to `validate_record`

There's also a few more changes to make around this (as a follow up to #105):

rename the argument to validate_contents, and make it kw-only.
should validate that everything has a hash/size even if validate_contents is False.
should validate that RECORD signatures are not mentioned in the RECORD.
should validate that the RECORD file is mentioned without hash or size.
documentation needs to present an example and add cross-references to/from WheelSource.validation_error.

0.1.1: pytest based test suite is failing

Looks like in pypi dist tar ball is missing tests/ directory content.

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-installer-0.1.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-installer-0.1.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/tkloczko/rpmbuild/BUILD/installer-0.1.1, configfile: setup.cfg, testpaths: tests/
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, flaky-3.7.0, cases-3.4.6, hypothesis-6.10.0
collected 0 items

============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1233
  /usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1233: PytestConfigWarning: Unknown config option: strict

    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
============================================================================ 1 warning in 0.02s ============================================================================
ERROR: file or directory not found: tests/

Data files copied incorrectly

Hi! I maintan caffeine-ng. When installing via setup.py, everything installs as expected:

python setup.py install --root="$pkgdir" --optimize=1 --skip-build

Installs, amongst other files:

/usr/share/icons/hicolor/16x16/apps/caffeine.png

But when using installer:

 python -m installer --destdir="$pkgdir" dist/*.whl

The files are installed into the wrong path, the example from above ends up in:

/usr/lib/python3.10/site-packages/usr/share/icons/hicolor/16x16/apps/caffeine.png

This is my relevant code: https://codeberg.org/WhyNotHugo/caffeine-ng/src/commit/e351249f3913e3f1354fba768803063597972125/setup.py#L9-L28

I'm not entirely sure what's wrong. Is this usage unsupported by installer? Is it deprecated, or pending implementation? Or have I found a bug?

This is a basic feature in any kind of installer, and without it packagers cannot use this tool.

Originally posted by @FFY00 in https://github.com/pradyunsg/installer/issues/52#issuecomment-835831533

Yeah, use-sysconfig-but-chroot-first is plain necessary for this common use case.

Originally posted by @flying-sheep in https://github.com/pradyunsg/installer/issues/52#issuecomment-838092371