agroden / python-minimizer Goto Github PK

Make this module installable via setup.py and consider submitting to PyPI for pip installation

Hi, it looks like minimizer needs updating for Python 3. From what I can see, COMMENT and NL have been removed from token.h in at least Python 3.6

When I run
(venv)$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

from minimizer import minimize
Traceback (most recent call last):
File "", line 1, in
File "[redacted]/minimizer.py", line 10, in
from token import (
ImportError: cannot import name 'COMMENT'

If I modify minimizer.py as follows, the import succeeds, but crashes soon after as COMMENT and NL symbols are not defined.

from token import (
#COMMENT, <-- this import fails
DEDENT,
ENDMARKER,
INDENT,
NEWLINE,
#NL, <-- this import fails
NAME,
NUMBER,
OP,
STRING,
tok_name
)

Hopefully this an easy fix. I'd like to use this in a project I'm working on.

Best regards

Add a recursive flag which indicates the inpath and outpath command line arguments represent folders

Is the output of the minimizer truly the most minimized it could be?

I tried installing 2.0.1 via pip3, I get an error and end up with 2.0.0 installed.

Output:

Collecting python-minimizer
    Using cached python-minimizer-2.0.1.tar.gz (6.5 kB)
    Installing build dependencies ... done
    Getting requirements to build wheel ... done
      Preparing wheel metadata ... error
      ERROR: Command errored out with exit status 1:
       command: [redacted]/vcs/git/test-minimizer/venv/bin/python3 [redacted]/vcs/git/test-minimizer/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpwxc1txni
           cwd: /tmp/pip-install-_o3vpo7e/python-minimizer_dcbca966d3ea4ee1a2f2df2681149475
      Complete output (16 lines):
      Traceback (most recent call last):
        File "[redacted]/vcs/git/test-minimizer/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in <module>
          main()
        File "[redacted]/vcs/git/test-minimizer/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "[redacted]/vcs/git/test-minimizer/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 133, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-sjlvgw0n/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 44, in prepare_metadata_for_build_wheel
          builder = WheelBuilder(poetry)
        File "/tmp/pip-build-env-sjlvgw0n/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 57, in __init__
          super(WheelBuilder, self).__init__(poetry, executable=executable)
        File "/tmp/pip-build-env-sjlvgw0n/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/builder.py", line 85, in __init__
          self._module = Module(
        File "/tmp/pip-build-env-sjlvgw0n/overlay/lib/python3.8/site-packages/poetry/core/masonry/utils/module.py", line 63, in __init__
          raise ModuleOrPackageNotFound(
      poetry.core.masonry.utils.module.ModuleOrPackageNotFound: No file/folder found for package python-minimizer
      ----------------------------------------
  WARNING: Discarding https://files.pythonhosted.org/packages/18/c0/883d5e7a44b2ddd7b98a2f2e140faaaf4f384d61b8bf91c5bb1ff8e5f2f5/python-minimizer-2.0.1.tar.gz#sha256=74aa842865cb821bfe3b2bcaf3c85a917c3234f98621fcdb7814fe2b5b42c639 (from https://pypi.org/simple/python-minimizer/). Command errored out with exit status 1: [redacted]/vcs/git/test-minimizer/venv/bin/python3 [redacted]/vcs/git/test-minimizer/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpwxc1txni Check the logs for full command output.
    Using cached python_minimizer-2.0.0-py3-none-any.whl (6.8 kB)
  Installing collected packages: python-minimizer
  Successfully installed python-minimizer-2.0.0

And here is the output of pip freeze:

$ pip freeze
pkg-resources==0.0.0
python-minimizer==2.0.0

Sorry to be a pest & thanks in advance

Generate a pipeline to make this more official with CI, testing, and a release pipeline to give greater operational assurance to users.

Other code minimizers offer this option, maybe it should be be a feature

Consider allowing a user to preserve the white space from the original document

I've gotten some comments that recursive minimization is renaming sub-directories in odd ways.

Looks like this may be Python 3.6 only. I tried it with Python 3.8, and the imports worked.

There is another error, however.
File [redacted]/venv/lib/python3.8/site-packages/minimizer.py", line 214, in minimize
grps = group_tokens(sbuf)
File "[redacted]/venv/lib/python3.8/site-packages/minimizer.py", line 139, in group_tokens
if verbose > 1:
NameError: name 'verbose' is not defined

Looks like global verbose is not being seen.

Originally posted by @another2020githubuser in #9 (comment)

When two OP type tokens are next to each other and this leads to odd looking (yet totally valid) output; consider fixing this

example: ... tok[0] == NAME) or not rmwspace:
becomes: ... tok[0]==NAME)or not rmwspace: