wheelodex / wheel-inspect Goto Github PK

The inspection functions should take a verify_digests: bool = True parameter, and the command line should take a --no-verify-digests (or just --no-verify?) option, for controlling whether the wheel inspection should verify that the digests listed in the RECORD are correct.

Include the ability to ensure file digests are distinct so I can use it for check-wheel-contents tests as well (while still keeping the code part of wheel-inspect, as testing the test helper will depend on the rest of the code too much).

Should the CLI have an option for selecting which metadata JSON format to use?

Current test runs show this error message:

[2022-05-11T06:16:34.578Z] ['error'] There was an error running the uploader: Error uploading to [https://codecov.io:](https://codecov.io/) Error: There was an error fetching the storage URL during POST: 400 - Bad Request - [ErrorDetail(string='Too many uploads to this commit.', code='invalid')]

On https://app.codecov.io/gh/jwodder/wheel-inspect the last uploaded test run was from 15 days ago:
https://codecov.io/gh/jwodder/wheel-inspect/commit/f95f1603710dc443cf885b7b864c9cd6f2df9449/

Since the CLI will output multiple objects when given multiple wheels, it seems only natural that the output should be a JSON array, either by default or when given a certain command-line option. Moreover, there should probably be an option for outputting a stream of objects as JSON Lines.

Alternatively, perhaps inspecting more than one wheel at once with the CLI should be forbidden?

Or perhaps that functionality should be eliminated altogether?

Hi,

Wheel is no longer exported by __init__.py, constituting a breaking change of the API. devpi-builder uses this library and imports Wheel; you can reproduce the failure introduced by v1.6.0 with python3 -m venv tmp_venv; tmp_venv/bin/python -m pip install devpi-builder; tmp_venv/bin/devpi-builder -h:

Traceback (most recent call last):
  File "tmp_venv/bin/devpi-builder", line 6, in <module>
    from devpi_builder.cli import main
  File "~/code/tmp_venv/lib/python3.7/site-packages/devpi_builder/cli.py", line 17, in <module>
    from devpi_builder import requirements, wheeler
  File "~/code/tmp_venv/lib/python3.7/site-packages/devpi_builder/wheeler.py", line 17, in <module>
    from wheel_inspect import Wheel, inspect_wheel
ImportError: cannot import name 'Wheel' from 'wheel_inspect' (~/code/tmp_venv/lib/python3.7/site-packages/wheel_inspect/__init__.py)

• show (or info?) performs the actual inspection
• verify: #18
• schema dumps the schema used by show's output (#8)
• Alternatively, just have a wheel-inspect command with no subcommands, with the "validate" and "schema" behaviors invoked via --validate and --dump-schema options

Context

wheel tags is a new feature from wheel 0.40.0 for wheel maintainers to tag a wheel, independent of the build system. It is extremely handy especially in the post-PEP 517 time where the build backend is no longer dominated by setuptools. Not all build backends allow the developer to easily tweak the wheel tags, and so the easiest way is to build a wheel in the most convenient fashion, and tag it afterwards.

However, wheel-inspect is outdated and not catching up with this new approach. The last update v1.7.1 was released in Apr 2022, whereas wheel tags is part of wheel 0.40.0 released in Mar 2023.

Symptom

Using wheel-inspect v1.7.1 on a wheel tagged via wheel tags, this is what we get

[root@b58bcbb2915e test]# wheel tags --platform-tag "manylinux2014_x86_64" cuquantum_python_cu11-23.6.0-11-cp39-cp39-linux_x86_64.whl
cuquantum_python_cu11-23.6.0-11-cp39-cp39-manylinux2014_x86_64.whl
[root@b58bcbb2915e test]# wheel2json cuquantum_python_cu11-23.6.0-11-cp39-cp39-manylinux2014_x86_64.whl
Traceback (most recent call last):
  File "/opt/conda/bin/wheel2json", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.9/site-packages/wheel_inspect/__main__.py", line 12, in main
    about = inspect_wheel(path)
  File "/opt/conda/lib/python3.9/site-packages/wheel_inspect/inspecting.py", line 191, in inspect_wheel
    return inspect(wf)
  File "/opt/conda/lib/python3.9/site-packages/wheel_inspect/inspecting.py", line 121, in inspect
    about["dist_info"]["wheel"] = obj.get_wheel_info()
  File "/opt/conda/lib/python3.9/site-packages/wheel_inspect/classes.py", line 69, in get_wheel_info
    return parse_wheel_info(txtfp)
  File "/opt/conda/lib/python3.9/site-packages/wheel_inspect/wheel_info.py", line 53, in parse_wheel_info
    wi = infoparser.parse(fp)
  File "/opt/conda/lib/python3.9/site-packages/headerparser/parser.py", line 288, in parse
    return self.parse_stream(scan(iterable, **self._scan_opts))
  File "/opt/conda/lib/python3.9/site-packages/headerparser/parser.py", line 264, in parse_stream
    raise errors.MissingFieldError(hd.name)
headerparser.errors.MissingFieldError: Required header field 'Tag' is not present

where it complains the wheel tag is missing, but I do not think tags have to be present. PyPA has clearly stated that

The wheel built package format includes these tags in its filenames

so the assumption (which wheel-inspect relies on) that the WHEEL file always contains a Tag field is not really held.

Suggested fix

This line can be relaxed to use required=False:
https://github.com/jwodder/wheel-inspect/blob/bd2051e9de2edb88075b40763d430887ef15c26d/src/wheel_inspect/wheel_info.py#L10
If Tag is not found, then wheel-inspect can just use the wheel filename to fetch the tag.

I need the exact text of the license that was delivered in the wheel package. (Even if it is then potentially very often redundant, because it will be very often the same for the popular licenses). Could this be added via cli-command?

Errors to detect:

• project/version in filename/dist-info doesn't match METADATA
  • Be sure to take the filename's underscorification of - in the project name and ! & + in the version into account
• tags in filename don't match those in WHEEL
• build tag in filename doesn't match that in WHEEL
• invalid Python/ABI/platform tag
• tags in filename not sorted?
• version doesn't comply with PEP 440
• METADATA:
  • unknown Metadata-Version?
  • missing required field
  • field has invalid value
  • non-multi-use field used multiple times
  • not at least version 1.1
• WHEEL:
  • unknown Wheel-Version
  • missing required field
  • field has invalid value
  • non-multi-use field used multiple times
• both zip-safe and not-zip-safe
• filename in RECORD contains a backslash?
• filename in RECORD contains a null?
• *.data/ cannot contain both purelib and platlib; which one is allowed depends on whether Root-Is-Purelib
• *.data/ contains a non-directory file? (ignore?)
• *.dist-info file is not UTF-8

The wheel spec states:

A wheel installer should warn if Wheel-Version is greater than the version it supports, and must fail if Wheel-Version has a greater major version than the version it supports.

Should wheel-inspect abide by this?

The inspection functions should take a raising: bool = False parameter, and the command line should take a --raising option, for controlling whether the wheel inspection should raise an exception immediately upon encountering the first wheel validation failure.

• Make the API worth exposing
• Type-annotate everything
• Document everything
• Set up a Read the Docs site
• Export classes for import directly from the wheel_inspect module

This is not always derivable from just the list of modules; consider packages containing only an __init__.py.

• It would make all the (increasing complicated) error handling much, much easier.

• There would need to be a Python extension wrapping the Rust library.

This would mean storing the valid entries plus a list of errors generated by the invalid entries.

Unless an option is given to treat them as dist-info dirs?

If using PEP517 directly (or for other reasons), it's possible to create a directory with the wheel metadata without doing a full build.

Please support parsing this information?

Hi jwodder,

This library looks pretty interesting, however I am having trouble with passing a Bytes Object instead of a path to inspect_wheel.

I would like to read a python wheel I uploaded, however based on what I am seeing in the code so far, it seems most things are dependent on a path.

This means all types of namespace packages listed at https://packaging.python.org/guides/packaging-namespace-packages/.

https://github.com/takluyver/wheeldex may be of use here.

Wheel/dist-info objects should have a filetrees (name WIP) attribute that is a dynamic mapping. Accessing this mapping with the keys "purelib", "platlib", "dist-info", "headers", "scripts", or "data" should return a PosixPath-like object for navigating through the respective subset of the files in the wheel. However — and this is key — it should not be possible to use such a path instance to access or view files outside the given filetree; this includes not being able to access the *.dist-info or *.data directories from the purelib tree (assuming root is purelib; otherwise, platlib).

• The filetrees mapping should accept any name not containing a slash; unknown names are treated as subdirectories of *.data. If a given directory does not exist (or if the corresponding entry in *.data is a file instead of a directory), None is returned.
• filetrees should accept a "root" (or None?) key to access purelib for purelib wheels and platlib for platlib wheels.
• These path objects should be accepted by the API wherever wheel entry filepaths are accepted.
• Path objects should have a wheel_path(?) attribute for getting the actual, full path of the underlying file inside the wheel.

Refuse to proceed if given a zipfile where any of the following are true:

• Zipfile has both path and path/
• Entries in zip are encrypted (Can this be detected?)
• File in zip twice (How does Python handle this?)
• Directory has data (Don't care?)
• Filename in zipfile contains a backslash?
• Filename in zipfile contains a null
• Filename in zipfile is absolute
• Empty filename field ("if input came from standard input")
• Filename in archive not UTF-8 (How does Python handle this?)
• Path in zipfile not normalized

This would allow generating the JSON Schema for the model directly.