lyz-code / autoimport Goto Github PK

View Code? Open in Web Editor NEW

87.0 87.0 20.0 4.2 MB

Autoimport automatically fixes wrong import statements.

Home Page: https://lyz-code.github.io/autoimport

License: GNU General Public License v3.0

Makefile 6.18% Python 93.82%

autoimport's People

Contributors

Stargazers

Watchers

autoimport's Issues

Add support for `python3 -m autoimport` invocation

Description

Python CLI tools typically give the ability to be invoked via python3 -m <NAME>. This is useful when multiple python versions are installed, and one want so pick up a specific version. isort, flake8, black and others all support this functionality. autoimport does not:

~$ python3 -m autoimport
/usr/local/bin/python3: No module named autoimport.__main__; 'autoimport' is a package and cannot be directly executed

Possible Solution

This can be easily achieved by adding a __main__.py file in the same directory as __init__.py. Within __main__.py it should be sufficient to just do (not tested):

from .entrypoints.cli import cli
cli()

Disable the blank line inserted at the top of the imports

Description

I wasn't sure if this is a bug or a feature, but whenever I run autoimport a blank line is inserted above the import statements.

before running autoimport:

after running autoimport:

Possible Solution

If this is a feature, it would be great to have a config option to turn it off. If it's a bug, should be as simple as removing whatever is inserting it, I'd be happy to open a PR depending on what needs to be done.

Additional context

I'm running this from the command line via a vim command: :!autoimport %

Deleted dependency detected

I'm a Cyber Security researcher and developer of PackjGuard [1] to address open-source software supply chain attacks.

Issue

During my research, I found that this repo is vulnerable to attack due to deleted dependency from the public PyPI registry.

Details

Specifically, file https://github.com/lyz-code/autoimport/blob/b421368d64027a0d253ae247da8ce08659f5277c/pyproject.toml lists projroot as one of the dependencies. However, it has been deleted from public PyPI. As such, an external bad actor can claim that name and register a malicious package, which will be then installed with pip install command, resulting in arbitrary remote code execution.

Impact

Not only your apps/services using https://github.com/lyz-code/autoimport repo code are vulnerable to this attack, but the users of your open-source Github repo could also fall victim.

You could read more about such attacks here: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610

Remediation

Please manually register a placeholder projroot package on PyPI immediately or remove projroot dependency from https://github.com/lyz-code/autoimport/blob/b421368d64027a0d253ae247da8ce08659f5277c/pyproject.toml to fix this vulnerability.

To automatically fix such issues in future, please install PackjGuard Github app [1].

Thanks!

PackjGuard is a Github app that monitors your repos 24x7, detects vulnerable/malicious/risky open-source dependencies, and creates pull requests for auto remediation: https://github.com/marketplace/packjguard

vscode integration

Hi! Thank you so much for creating this! I was wondering if there are any plans to create a vscode integration.

Do not reformat code lines that end with `# fmt: skip`

Description

There is currently no way to tell both black and autoimport to ignore formatting a line.

This can be an issue when using black along with autoimport but wanting both formatters to ignore a debugger line.

The # noqa: autoimport is not recognised by black, so the current implementations does not suffice.

Black uses # fmt: skip as an instruction to not format a line.

Possible Solution

We can utilise this same marker to ensure neither autoimport or black format a given line.

Add imports of the local package

Description

It would be nice if autoimport was clever enough to import at least the objects that are part of the program/library being developed.

Possible Solution

Add an entry in _find_package similar to _find_package_in_typing that deduces the package that is developed and find any missing objects there.

We could search on the installed packages:

import pkg_resources
installed_packages = pkg_resources.working_set
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
     for i in installed_packages])
print(installed_packages_list)

For any that matches any of the contents of the current working directory.

Then we can recursively use dir() to find the objects of the package and subpackages

Additional context

Related Issue

Stdout of unmodified script should be returned even if autoimport fails to resolve libraries

Description

I think autoimport should output the entire script even if it cannot resolve missing libraries. From what I can see, it doesn't output anything despite #131.

Steps to reproduce

Use the following example:

#! /usr/bin/env python

plt.style.use('ggplot')

Run cat example.py | autoimport -
No response.

Current behavior

It doesn't return the unmodified script.

Desired behavior

I believe it should return the unmodified script.

Environment

╰─ → $ python -c "import autoimport.version; print(autoimport.version.version_info())"

           autoimport version: 1.2.2
               python version: 3.9.12 (main, Jun  1 2022, 11:38:51)  [GCC 7.5.0]
                     platform: Linux-5.18.3-051803-generic-x86_64-with-glibc2.35

Improve the loading of the objects of the package under development

Description

If you are developing a package and install it with pip install -e ., whenever you make a change that breaks the package, autoimport is not going to be able to load the objects as it is not able lo import them.

Desired behavior

We want to be able to import the package objects even if the package is broken

Mac arm64 issues

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/yoland/Library/Python/3.10/lib/python/site-packages/autoimport/__main__.py", line 3, in <module>
    from autoimport.entrypoints.cli import cli
  File "/Users/yoland/Library/Python/3.10/lib/python/site-packages/autoimport/entrypoints/cli.py", line 10, in <module>
    from maison.config import ProjectConfig
  File "/Users/yoland/Library/Python/3.10/lib/python/site-packages/maison/__init__.py", line 2, in <module>
    from .config import ProjectConfig
  File "/Users/yoland/Library/Python/3.10/lib/python/site-packages/maison/config.py", line 12, in <module>
    from maison.schema import ConfigSchema
  File "/Users/yoland/Library/Python/3.10/lib/python/site-packages/maison/schema.py", line 2, in <module>
    from pydantic import BaseModel
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pydantic/__init__.cpython-310-darwin.so, 0x0002): tried: '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pydantic/__init__.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pydantic/__init__.cpython-310-darwin.so' (no such file), '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pydantic/__init__.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

Add `--ignore-init-modules` to configurable options

Description

I'd like that autoimport can figure out its opts, akin to pytests addopts

With that, some more documentation examples would be welcome

Possible Solution

Additional context

PS: I am aware of the maintenance mode. Just wanted to add the idea to the wall.

Related Issue

Option to disable "move-to-top" feature

Description

I work on a project which has many function-local imports because top imports would cause import cycles. But I can't use autoimport because it insists on moving them to the top (and also produces invalid syntax in this case, but that's another story).

Possible Solution

Make "move-import-to-top" feature optional, so it can be disabled, and still keep the "remove-unused-imports" feature.

Provide helpful message instead of crashing, when a symbol cannot be resolved.

Description

Currently, when autoimport FILENAME.py does not find a suitable
completion, the program just crashes.

Instead in such as case, that symbol should be skipped and informative
messages should be given.

Desirable behavior

Consider this exmaple file:

## -- demo.py
def f(x):
    return sin(x) + cos(x)

Instead, it would be desirable to obtain an output message along the lines of

>>> autoimport demo.py
demo.py:2: No such module: 'cos'
demo.py:2: No such module: 'sin'

and otherwise have autoimport continue its work.

Actual behavior

For such a file, autoimport demo.py crashes:

>>> autoimport demo.py 2>&1 | head -5
Traceback (most recent call last):
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 736, in __str__
    return self._str
AttributeError: _str

Interaction with explicit imports

The current behavior of autoimport allows to import objects from
modules by explicitly specifying them, e.g.

## -- demo2.py
def f(x):
    return sin(x) + cos(x)
    from numpy import sin, cos

which allows for more convenient workflows than contantly jumping back
to the import statements by using autoimport's "move imports to
top" functionality.

>>> autoimport demo2.py && cat demo2.py
from numpy import sin, cos

def f(x):
    return sin(x) + cos(x)

However, it fails if any of the missing imports are not given
explicitly, e.g.

## -- demo3.py
def f(x):
    return sin(x) + cos(x)
    from numpy import sin

>>> autoimport demo3.py 2>&1 | tail -5
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 738, in __str__
    self._str = self._format_parsed_parts(self._drv, self._root,
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 718, in _format_parsed_parts
    return drv + root + cls._flavour.join(parts[1:])
RecursionError: maximum recursion depth exceeded while calling a Python object

Full example traceback

## -- demo.py
def f(x):
    return sin(x) + cos(x)

>>> autoimport demo.py
Traceback (most recent call last):
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 736, in __str__
    return self._str
AttributeError: _str

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/USERNAME/anaconda3/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 19, in cli
    fixed_code = services.fix_files(files)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/services.py", line 27, in fix_files
    fixed_source = fix_code(source)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/services.py", line 67, in fix_code
    return SourceCode(original_source_code).fix()
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/model.py", line 59, in fix
    self._fix_flake_import_errors()
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/model.py", line 248, in _fix_flake_import_errors
    self._add_package(object_name)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/model.py", line 260, in _add_package
    import_string = self._find_package(object_name)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/model.py", line 287, in _find_package
    package = getattr(self, check)(name)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/autoimport/model.py", line 303, in _find_package_in_our_project
    project_package = os.path.basename(here()).replace("-", "_")
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/pyprojroot/pyprojroot.py", line 44, in here
    project_path = py_project_root(Path(".").cwd(), project_files)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/pyprojroot/pyprojroot.py", line 18, in py_project_root
    return py_project_root(path.parent, project_files)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/pyprojroot/pyprojroot.py", line 18, in py_project_root
    return py_project_root(path.parent, project_files)
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/pyprojroot/pyprojroot.py", line 18, in py_project_root
    return py_project_root(path.parent, project_files)
  [Previous line repeated 971 more times]
  File "/home/USERNAME/anaconda3/lib/python3.9/site-packages/pyprojroot/pyprojroot.py", line 15, in py_project_root
    found = list(path.glob(file))
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 1167, in glob
    for p in selector.select_from(self):
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 533, in _select_from
    if (is_dir if self.dironly else exists)(path):
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 1414, in exists
    self.stat()
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 1222, in stat
    return self._accessor.stat(self)
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 743, in __fspath__
    return str(self)
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 738, in __str__
    self._str = self._format_parsed_parts(self._drv, self._root,
  File "/home/USERNAME/anaconda3/lib/python3.9/pathlib.py", line 718, in _format_parsed_parts
    return drv + root + cls._flavour.join(parts[1:])
RecursionError: maximum recursion depth exceeded while calling a Python object

Version info

>>> python -c "import autoimport.version; print(autoimport.version.version_info())"
           autoimport version: 0.7.1
               python version: 3.9.5 (default, May 18 2021, 19:34:48)  [GCC 7.3.0]
                     platform: Linux-4.12.14-lp151.28.91-default-x86_64-with-glibc2.26

autoimport was installed by:

>>> pip3 install git+https://github.com/lyz-code/autoimport.git@master
Collecting git+https://github.com/lyz-code/autoimport.git@master
  Cloning https://github.com/lyz-code/autoimport.git (to revision master) to ./pip-req-build-q12jz4nv
  Running command git clone -q https://github.com/lyz-code/autoimport.git /tmp/pip-req-build-q12jz4nv
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Requirement already satisfied: Click in /home/USERNAME/anaconda3/lib/python3.9/site-packages (from autoimport==0.7.1) (8.0.1)
Requirement already satisfied: autoflake in /home/USERNAME/anaconda3/lib/python3.9/site-packages (from autoimport==0.7.1) (1.4)
Requirement already satisfied: sh in /home/USERNAME/anaconda3/lib/python3.9/site-packages (from autoimport==0.7.1) (1.14.2)
Requirement already satisfied: pyprojroot in /home/USERNAME/anaconda3/lib/python3.9/site-packages (from autoimport==0.7.1) (0.2.0)
Requirement already satisfied: pyflakes>=1.1.0 in /home/USERNAME/anaconda3/lib/python3.9/site-packages (from autoflake->autoimport==0.7.1) (2.2.0)

Python `SyntaxError` when removing last import from parenthetical import ... from statement

Description

When autoimport removes the last import from a multi-line parenthetical import statement, the result gives a SyntaxError.

Steps to reproduce

Start with the following file:

from foo import (
    bar,
    baz,
)

Run autoimport
The file now looks like this:

from foo import (
)

Run python, get a SyntaxError.

Desired behavior

Autoimport should produce valid python code. The from foo import (\n) should be removed.

Environment

>>> print(autoimport.version.version_info())
           autoimport version: 1.0.0
               python version: 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:40:17)  [GCC 9.4.0]
                     platform: Linux-5.4.0-91-generic-x86_64-with-glibc2.27

Preserve line endings

Description

Running autoimport on a file with unix line ending (LR) in Windows convert it into Windows line endings (CRLF).

Steps to reproduce

Create a file with unix line endings in Windows
Run autoimport file.py

Current behavior

Line ending converted to OS line endings.

Desired behavior

Line endings should be preserved.

Broken syntax with multiline, function-local imports

Description

Steps to reproduce

Input file:

from foo import bar


class C(bar.C):
    def method(self, param):
        from foo.some_very_long_name_of_a_module.another_submodule.wait_for_it import (
            yet_another_long_name
        )

        yet_another_long_name(param)

Run autoimport on that file.

Current behavior

Output file:

from foo import bar


        from foo.some_very_long_name_of_a_module.another_submodule.wait_for_it import (
            yet_another_long_name
)


class C(bar.C):
    def method(self, param):

        yet_another_long_name(param)

Desired behavior

Syntax is not broken. Import moved correctly (or not moved, see #249)

Environment

autoimport 1.3.3

By the way, neither make version nor python -c "import autoimport.version; print(autoimport.version.version_info())" do work with pipx, I used pipx list, which is a bit rigid.

Pre-generate module symbol list

Description

One of the most useful use case for auto import is when using PyQt/PySide classes. This poses several problems

ambiguity, the same class exists in both PyQt and PySide and also qtpy
performance, these modules contains a lot of symbols and thus importing it at runtime will add to the execution time

Possible Solution

Pre-generate the symbol list in the modules. This should be manually run by the user. Since python modules can have side effect, importing any/all modules in the PYTHONPATH could be dangerous. Module priority could be defined by using configuration file in the project directory.

Pull Request maintenance only :(

Hi all, right now I'm not using this project and don't have the time to maintain it, I plan to start using it again in the future so I'll probably restore the maintenance mode. This means:

I will no longer guarantee that the project is installable with future versions of python not that it works with the latest version of the dependencies (I've disabled the Install and Update CI pipelines).
I will not monitor the issues so don't expect any reply from myself.
I'll still monitor and answer the pull requests for any contribution.

If you want to see the project thrive please evaluate becoming a maintainer and help me answering issues, pull requests and develop the program.

This announcement is not meant to prevent you from opening issues! Even though I'm not maintaining the project right now, it's always good to do requests. Maybe others in the community can help you :).

If you need to get my attention you can reach me here

Get ValueError on unused import & noqa comment

Description

Error happens on unused import ending wirh #noqa: autoimport.

from os import getcwd # noqa: autoimport

Steps to reproduce

Run autoimport test.py

Current behavior

Got ValueError: list.remove(x): x not in list in terminal.

Desired behavior

Just ignore the noqa line.

Environment

python -c "import autoimport.version; print(autoimport.version.version_info())"
autoimport version: 0.7.4
python version: 3.8.10 (default, Sep 28 2021, 16:10:42)  [GCC 9.3.0]
platform: Linux-5.8.0-53-generic-x86_64-with-glibc2.29

`typing.Literal` can cause false-positive import

Hello,

Unfortunately I have a bug to report. I'm getting a false-positive import with certain typing.Literal usage.

before running autoimport:

from typing import Literal as L

var: L["* time"] = "* time"

after running autoimport:

from typing import Literal as L
import time

var: L["* time"] = "* time"

Importing the library time is not correct.

Context:

This bug came up when using the nptyping library, which encourages liberal use of Literal.

Environment:

$ python -c "import autoimport.version; print(autoimport.version.version_info())"
           autoimport version: 1.2.2
               python version: 3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:39:04) [GCC 10.3.0]
                     platform: Linux-5.13.0-35-generic-x86_64-with-glibc2.31

[Bug] handling top-level `if TYPE_CHECKING: ...` block

Description

autoimport does not properly handle top-level use of if TYPE_CHECKING: ...

Steps to reproduce

Case 1:

Run autoimport on the following file:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    foo = "bar"

The output looks like this:

if TYPE_CHECKING:
    foo = "bar"

Why did the from typing ... import disappear?

Case 2:

Run autoimport on the following file:

if TYPE_CHECKING:
    foo = "bar"

The output looks like this:

from typing import TYPE_CHECKING

Why did the if TYPE_CHECKING: ... block disappear?

Case 3 (this is expected behavior):

Run autoimport on the following file:

from typing import TYPE_CHECKING

The output is empty:

Current behavior

Repeatedly running autoimport on the file from Case 1 causes the file's contents to completely disappear.

Desired behavior

autoimport should not remove the if TYPE_CHECKING: ... block or the related import from typing ....

Environment

 $ python -c "import autoimport.version; print(autoimport.version.version_info())"
------------------------------------------------------------------
     autoimport: 1.3.1
     Python: 3.9.12
     Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.31
------------------------------------------------------------------

Support a config file with example imports

Description

It would be useful to support local conventions (such as import datetime as dt) and names from local modules (from mymodule.mypackage.somewhere import SomeName).

Possible Solution

Doing this requires a config; a most simple config for this would be a py file with example imports.

Example implementation: #170

Additional context

For further consideration: an extra tool for gathering all imports from files, to build such config, to cleanup, tune, and use it.

Related Issue

Can be seen as an alternative to #39

Add imports of frequent used objects

Description

Make autoimport manage the commonly used imports.

Possible Solution

Add an entry in _find_package similar to _find_package_in_typing that has a dictionary of object and import string.

For example:

from _pytest.logging import LogCaptureFixture

def test_baz(caplog: LogCaptureFixture):
    func_under_test()
    for record in caplog.records:
        assert record.levelname != "CRITICAL"
    assert "wally" not in caplog.text

We could have the dictionary {'LogCaptureFixture': 'from _pytest.logging import LogCaptureFixture'}.

Additional context

Related Issue

File modification time updates even if autoimport does not change file

Feature Request

If I run autoimport on a file and the file is not changed, the modification time of the file should not be changed either.

For example:

$ echo "print(123)" > foo.py
$ cat foo.py
print(123)
$ date -r foo.py
Thu Jan 20 17:37:42 CST 2022
$ autoimport foo.py
$ cat foo.py
print(123)
$ # ^ foo.py was not changed
$ date -r foo.py
Thu Jan 20 17:37:51 CST 2022
$ # ^ file modification time has changed!

autoimport did not modify foo.py, but the modification time was changed.

Possible Solution

Perhaps autoimport could check whether the file would be modified, and only touch the file if a nonzero diff would occur.

Additional context

I use makefiles and other similar tools which keep track of file modification time. If autoimport does not change the file's contents, modification of the file modification time triggers a false-positive make/build cycle.

Duplicate imports, when the name is used multiple times

Description

When an unimported symbol occurs multiple times, duplicate import statements may be produced.

Steps to reproduce

Create a file demo.py

def f(x):
    return numpy.sin(x) + numpy.cos(x) + numpy.tan(x)

Execute autoimport demo.py.

Current behavior

The file will contain a duplicate import of numpy.

import numpy
import numpy
import numpy

def f(x):
    return numpy.sin(x) + numpy.cos(x) + numpy.tan(x)

Desired behavior

The module should be imported only once.

import numpy

def f(x):
    return numpy.sin(x) + numpy.cos(x) + numpy.tan(x)

Environment

>>> python -c "import autoimport.version; print(autoimport.version.version_info())"
           autoimport version: 0.7.0
               python version: 3.9.5 (default, May 18 2021, 19:34:48)  [GCC 7.3.0]
                     platform: Linux-4.12.14-lp151.28.91-default-x86_64-with-glibc2.26

Search object in all available packages

Description

Make autoimport search for the object in all the available packages.

Possible Solution

Add an entry in _find_package similar to _find_package_in_typing that deduces the package that is developed and find any missing objects there.

We could search all the packages and use dir() at least on the first level. But we'll need to import all, I'm not sure the performance of that.

Additional context

Related Issue

Similar to #37 and #38

Bug: Interaction between `TYPE_CHECKING` and star import

Description

I've stumbled upon a bug that comes up when using the TYPE_CHECKING guard together with a star import.

Steps to reproduce

Here's a minimal reproducible example:

# repro.py
from typing import TYPE_CHECKING

from module_1 import *

if TYPE_CHECKING:
    import module_2

Click here to see the traceback

$ autoimport repro.py
Traceback (most recent call last):
  File "/home/homestar/tmp/.direnv/python-3.11.3/bin/autoimport", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/autoimport/entrypoints/cli.py", line 94, in cli
    fixed_code = services.fix_files(flattened_files, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/autoimport/services.py", line 29, in fix_files
    fixed_source = fix_code(source, config)
                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/autoimport/services.py", line 73, in fix_code
    return SourceCode(original_source_code, config=config).fix()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/autoimport/model.py", line 67, in fix
    self._fix_flake_import_errors()
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/autoimport/model.py", line 308, in _fix_flake_import_errors
    self._remove_unused_imports(import_name)
  File "/home/homestar/tmp/.direnv/python-3.11.3/lib/python3.11/site-packages/autoimport/model.py", line 455, in _remove_unused_imports
    if re.match(
       ^^^^^^^^^
  File "/home/homestar/.pyenv/versions/3.11.3/lib/python3.11/re/__init__.py", line 166, in match
    return _compile(pattern, flags).match(string)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/.pyenv/versions/3.11.3/lib/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/.pyenv/versions/3.11.3/lib/python3.11/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/.pyenv/versions/3.11.3/lib/python3.11/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/.pyenv/versions/3.11.3/lib/python3.11/re/_parser.py", line 455, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homestar/.pyenv/versions/3.11.3/lib/python3.11/re/_parser.py", line 685, in _parse
    raise source.error("multiple repeat",
re.error: multiple repeat at position 37

It seems to be an issue with one of Autoimport's regex patterns. ~~The error only occurs if both the star import and the if TYPE_CHECKING block are present.~~ (Edit: see my comment below)

Environment

------------------------------------------------------------------
     autoimport: 1.3.3
     Python: 3.11.3
     Platform: Linux-5.15.0-75-generic-x86_64-with-glibc2.31
------------------------------------------------------------------

Extra blank lines after imports are removed

Description

autoimport removes extra blank lines after imports, even when it’s required for PEP 8 compliance (e.g. when imports are directly followed by a class or a def). And also I like doing lines_after_imports = 2 in isort for consistency anyway.

While you may say I can just run another auto-formatter afterwards to fix the issue, this actually makes it impossible to use autoimport as a pre-commit hook, as on the next run autoimport will remove the blank line again, and pre-commit requires all hooks to succeed.

Steps to reproduce

Create a file with the following contents:

 import random


 def foo():
     print(random.random())

Run autoimport foo.py

Current behavior

The file gets rewritten to this:

    import random

    def foo():
        print(random.random())

Desired behavior

The file should remain untouched.

Environment

           autoimport version: 1.2.2
               python version: 3.10.4 (main, Apr  4 2022, 14:54:08) [GCC 10.2.1 20210110]
                     platform: Linux-5.10.0-16-amd64-x86_64-with-glibc2.31

`from x import y as z` causes crash

Description

See title

Steps to reproduce

echo from subprocess import run as run > test.py
autoimport test.py

Current behavior

Traceback (most recent call last):
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 19, in cli
    fixed_code = services.fix_files(files)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/autoimport/services.py", line 27, in fix_files
    fixed_source = fix_code(source)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/autoimport/services.py", line 67, in fix_code
    return SourceCode(original_source_code).fix()
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/autoimport/model.py", line 59, in fix
    self._fix_flake_import_errors()
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/autoimport/model.py", line 254, in _fix_flake_import_errors
    self._remove_unused_imports(import_name)
  File "/home/sam/.cache/pypoetry/virtualenvs/charmonium.determ-hash-sYmCxUo--py3.9/lib/python3.9/site-packages/autoimport/model.py", line 405, in _remove_unused_imports
    imports.remove(object_name)
ValueError: list.remove(x): x not in list

I did some basic debugging. The pattern on L397 does not expect as z part of the statement. On L404, imports = ['x as z']. L405 attempts to remove z from this list.

Desired behavior

No crash

Environment

autoimport 0.7.5

crashes with `ValueError` when the import to remove appears twice

Description

autoimport sometimes throws ValueError when the import to remove appears twice

Steps to reproduce

Create the following python file:

# tmp.py
import my_module.m
from my_module import _tm

Invoke autoimport:

 $ autoimport tmp.py
Traceback (most recent call last):
  File "/home/rig1/miniconda3/envs/utils/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/autoimport/entrypoints/cli.py", line 84, in cli
    fixed_code = services.fix_files(flattened_files, config)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/autoimport/services.py", line 29, in fix_files
    fixed_source = fix_code(source, config)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/autoimport/services.py", line 73, in fix_code
    return SourceCode(original_source_code, config=config).fix()
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/autoimport/model.py", line 68, in fix
    self._fix_flake_import_errors()
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/autoimport/model.py", line 288, in _fix_flake_import_errors
    self._remove_unused_imports(import_name)
  File "/home/rig1/miniconda3/envs/utils/lib/python3.10/site-packages/autoimport/model.py", line 454, in _remove_unused_imports
    imports.remove(object_name)
ValueError: list.remove(x): x not in list

Strangely, this bug seems dependent on the exact names of the imports. For example, if I do from my_module import _t instead of from my_module import _tm, no ValueError occurs.

Environment

 $ python -c "import autoimport.version; print(autoimport.version.version_info())"
------------------------------------------------------------------
     autoimport: 1.2.3
     Python: 3.10.5
     Platform: Linux-5.15.0-48-generic-x86_64-with-glibc2.35
------------------------------------------------------------------

Crashes with file that only has imports

Description

While testing with a file that only has imports, I noticed that autoimport crashes with the exception IndexError: list index out of range.

Steps to reproduce

％ autoimport - <<<'import os'
Traceback (most recent call last):
  File "/tmp/aaaa/venv/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 19, in cli
    fixed_code = services.fix_files(files)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/services.py", line 27, in fix_files
    fixed_source = fix_code(source)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/services.py", line 67, in fix_code
    return SourceCode(original_source_code).fix()
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 47, in __init__
    self._split_code(source_code)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 78, in _split_code
    self._extract_typing_statements(source_code_lines)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 155, in _extract_typing_statements
    r"^if TYPE_CHECKING:$", source_lines[typing_start_line]
IndexError: list index out of range

Current behavior

Stack trace:

％ autoimport - <<<'import os'
Traceback (most recent call last):
  File "/tmp/aaaa/venv/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 19, in cli
    fixed_code = services.fix_files(files)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/services.py", line 27, in fix_files
    fixed_source = fix_code(source)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/services.py", line 67, in fix_code
    return SourceCode(original_source_code).fix()
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 47, in __init__
    self._split_code(source_code)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 78, in _split_code
    self._extract_typing_statements(source_code_lines)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 155, in _extract_typing_statements
    r"^if TYPE_CHECKING:$", source_lines[typing_start_line]
IndexError: list index out of range

Desired behavior

Should print an empty string. There are other cases too, for example:

％ cat sample.py
import os  # noqa: F401
import sys
％ autoimport sample.py
Traceback (most recent call last):
  File "/tmp/aaaa/venv/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 19, in cli
    fixed_code = services.fix_files(files)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/services.py", line 27, in fix_files
    fixed_source = fix_code(source)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/services.py", line 67, in fix_code
    return SourceCode(original_source_code).fix()
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 47, in __init__
    self._split_code(source_code)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 78, in _split_code
    self._extract_typing_statements(source_code_lines)
  File "/private/tmp/aaaa/venv/lib/python3.9/site-packages/autoimport/model.py", line 155, in _extract_typing_statements
    r"^if TYPE_CHECKING:$", source_lines[typing_start_line]
IndexError: list index out of range

In that case the output should've been import os.

Environment

％ autoimport --version
           autoimport version: 0.7.3
               python version: 3.9.6 (default, Jun 29 2021, 05:25:02)  [Clang 12.0.5 (clang-1205.0.22.9)]
                     platform: macOS-11.5.2-x86_64-i386-64bit
％ python -c "import autoimport.version; print(autoimport.version.version_info())"
           autoimport version: 0.7.3
               python version: 3.9.6 (default, Jun 29 2021, 05:25:02)  [Clang 12.0.5 (clang-1205.0.22.9)]
                     platform: macOS-11.5.2-x86_64-i386-64bit

Import objects by mimicking other modules

The idea of have a list of "common statements" in the pyproject file seems ok for small project, but it's unmaintainable for large scale ones.

For instance, I have a project with several subpackages defining different classes (sqlalchemy models) and I would like to be able to autoimport them without needing to declare them explicitly.
The point is I have plenty other modules where I already imported those classes and I would like just

So, I guess it could be possible to walk sibling modules (or beyond) and try to find there the any missing object is being imported, in a sake of "automatic" common statements.

If this idea fits in the project I could try to implement it.

crashes when processing some imports that have `# comments`

Description

autoimport crashes when there are multiple imports from the same module and one of the imports is marked with a # noqa comment.

Current behavior

Here is a file that causes a crash:

# tmp.py
from typing import Any  # noqa
from typing import TypeVar

Here is the output from autoimport:

$ autoimport tmp.py
Traceback (most recent call last):
  File "/home/jasha10/miniconda3/envs/pysc/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 21, in cli
    fixed_code = services.fix_files(files, config)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/services.py", line 29, in fix_files
    fixed_source = fix_code(source, config)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/services.py", line 69, in fix_code
    return SourceCode(original_source_code, config=config).fix()
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/model.py", line 62, in fix
    self._fix_flake_import_errors()
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/model.py", line 257, in _fix_flake_import_errors
    self._remove_unused_imports(import_name)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/model.py", line 411, in _remove_unused_imports
    imports.remove(object_name)
ValueError: list.remove(x): x not in list

Desired behavior

Ideally, the noqa line(s) should not be touched, and the other line(s) should be processed as usual.

Environment

$ autoimport --version
           autoimport version: 0.8.0
               python version: 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:23:11)  [GCC 9.4.0]
                     platform: Linux-5.4.0-87-generic-x86_64-with-glibc2.27

Support relative imports of the package under development

Description

When developing a package I prefer to use relative imports over absolute ones to avoid loop dependencies. Right now autoimport only uses absolute imports, so I need to remove the absolute reference and convert it to relative.

Possible Solution

It's not easy to solve, as we can't take into account the relative position of the file we're fixing regarding the rest of the project, as some of the tools that use autoimport (ALE for vim integration), copies the file to a temporal directory and applies the fixer there.

So we would need to:

Deduce the location of the file regarding the package. Either by analyzing the objects of the file we're fixing and searching them in the package, or by analyzing existent relative import statements.
Get the absolute path to the object that we need to import
Get the relative path from the file location to the desired object

Additional context

Related Issue

<deleted>

Flag to skip for `init.py` files

Since __init__.py files tend to have "unused" imports, autoimport will happily remove them.

This is fine if the script is run from somewhere where we can easily filter out the __init__.py (like in a Makefile), but becomes unwieldy when run through "fix-on-save" in your editor.

autoflake has solved the same problem with a --ignore-init-module-imports flag, perhaps we could add the same to autoimport?

Edit: should've opened this as a feature request, sry.

Not able to run `pre-commit` hooks

Description

following instructions here https://lyz-code.github.io/autoimport/contributing/#pull-requests
all steps work fine until getting to the git commit stage. this fails with the following error:

[WARNING] The 'rev' field of repo 'https://github.com/ambv/black' appears to be a mutable reference (moving tag / branch).  Mutable references are never updated after first install and are not supported.  See https://pre-commit.com/#using-the-latest-version-for-a-repository for more details.  Hint: `pre-commit autoupdate` often fixes this.
[WARNING] The 'rev' field of repo 'https://github.com/life4/flakehell/' appears to be a mutable reference (moving tag / branch).  Mutable references are never updated after first install and are not supported.  See https://pre-commit.com/#using-the-latest-version-for-a-repository for more details.  Hint: `pre-commit autoupdate` often fixes this.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/ambv/black.
An unexpected error has occurred: CalledProcessError: command: ('/Library/Developer/CommandLineTools/usr/libexec/git-core/git', 'checkout', 'master')
return code: 1
expected return code: 0
stdout: (none)
stderr:
    error: pathspec 'master' did not match any file(s) known to git

Check the log at /Users/.../.cache/pre-commit/pre-commit.log

i believe this is due to the fact that black repo has no master branch (been renamed to main). however i think there are larger issues here, note the pre-commit warnings at the top of the trace.

also, one of the pre-commit hooks is flakehell which has been archived. there is a fork but perhaps it would be best to use a different tool here?

Steps to reproduce

follow the steps in the contributing guide and then try to commit some changes

Current behavior

the error happens

Desired behavior

i'm able to commit changes

Environment

           autoimport version: 0.7.3
               python version: 3.7.4 (default, Sep 15 2021, 11:54:44)  [Clang 12.0.0 (clang-1200.0.32.29)]
                     platform: Darwin-19.6.0-x86_64-i386-64bit

Feature request: ability to run autoimport on a directory

Description

It would be nice if calling autoimport <directory> (on the command line) would process the python files under that directory.

Possible Solution

I find myself typing this at the command line sometimes:
$ for f in `ls *.py`; do autoimport $f; done

Additional context

This would be consistent with other tools:

mypy <directory>
flake8 <directory>
black <directory>
isort <directory>
# etc...

Bug: unused multiline import statement trigges re.error

Description

Given a piece of code containing at least one unused multiline import statement, autoimport exits abnormally.

Steps to reproduce

Given the following code:

from sympy import *

def compute_expression(expression: str) -> float:
    expression = expression.replace('^', '**')
    assignments, calculation = expression.split(';')[:-1], expression.split(';')[-1]
    symbols_table = {}
    for assignment in assignments:
        variable, value = assignment.split('=')
        symbols_table[variable.strip()] = float(value.strip())
    for variable, value in symbols_table.items():
        calculation = calculation.replace(variable, str(value))
    result = eval(calculation)
    return result

import unittest
from sympy import *

class TestComputeExpression(unittest.TestCase):
    def test_single_operation(self):
        expression = "x = 8; y = 4; z = 2; w = 3; x + y"
        result = compute_expression(expression)
        self.assertEqual(result, 12.0)

    def test_multiple_operations(self):
        expression = "x = 8; y = 4; z = 2; w = 3; x / (y + z) * w - z ^ w"
        result = compute_expression(expression)
        self.assertEqual(result, -1.0)

    def test_parentheses(self):
        expression = "x = 8; y = 4; z = 2; w = 3; (x + y) * z"
        result = compute_expression(expression)
        self.assertEqual(result, 24.0)

    def test_exponentiation(self):
        expression = "x = 8; y = 4; z = 2; w = 3; x ^ y"
        result = compute_expression(expression)
        self.assertEqual(result, 4096.0)

    def test_variable_assignment(self):
        expression = "x = 8; y = 4; z = 2; w = 3; x + y; x = 10; x + y"
        result = compute_expression(expression)
        self.assertEqual(result, 14.0)

And we use autoimport.fix_code(code_str) to fix it, autoimport exits with the following traceback:

Traceback (most recent call last):
File "D:\XXX\fix_code_test.py", line 54, in
print(fix_code(code))
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\services.py", line 73, in fix_code
return SourceCode(original_source_code, config=config).fix()
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\model.py", line 67, in fix
self._fix_flake_import_errors()
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\model.py", line 308, in _fix_flake_import_errors
self._remove_unused_imports(import_name)
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\model.py", line 455, in _remove_unused_imports
if re.match(
File "D:\anaconda3\envs\my_env\lib\re.py", line 190, in match
return _compile(pattern, flags).match(string)
File "D:\anaconda3\envs\my_env\lib\re.py", line 303, in _compile
p = sre_compile.compile(pattern, flags)
File "D:\anaconda3\envs\my_env\lib\sre_compile.py", line 788, in compile
p = sre_parse.parse(p, flags)
File "D:\anaconda3\envs\my_env\lib\sre_parse.py", line 955, in parse
p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
File "D:\anaconda3\envs\my_env\lib\sre_parse.py", line 444, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
File "D:\anaconda3\envs\my_env\lib\sre_parse.py", line 672, in _parse
raise source.error("multiple repeat",
re.error: multiple repeat at position 31

Plausible cause for this issue:

It seems that the _remove_unused_imports function in model.py (lines 441 to 505) failed to consider the scenario where object_name is not a regular identifier but can be a "*" , which can leads to an abnormal regular expression in lines 456 and 457, which then leads to a multiple repeat exception when using re.match to match lines in the code.

crashes on empty file

Hello,

Autoimport crashes on an empty file.
Here is a minimal example:

$ touch empty.py
$ autoimport empty.py
Traceback (most recent call last):
  File "/home/jasha10/miniconda3/envs/pysc/bin/autoimport", line 8, in <module>
    sys.exit(cli())
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/entrypoints/cli.py", line 19, in cli
    fixed_code = services.fix_files(files)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/services.py", line 27, in fix_files
    fixed_source = fix_code(source)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/services.py", line 67, in fix_code
    return SourceCode(original_source_code).fix()
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/model.py", line 47, in __init__
    self._split_code(source_code)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/model.py", line 78, in _split_code
    self._extract_typing_statements(source_code_lines)
  File "/home/jasha10/miniconda3/envs/pysc/lib/python3.9/site-packages/autoimport/model.py", line 154, in _extract_typing_statements
    if re.match(r"^if TYPE_CHECKING:$", source_lines[typing_start_line]):
IndexError: list index out of range

Autoimport version 0.7.2
CPython version 3.9.6

Import lines with a semicolon separator are improperly formatted

Description

Import lines that also contain a semicolon separator will be incorrectly formatted.

Steps to reproduce

Execute autoimport on this file:

class Person:
    def __init__(self, name, age, gender):
        self.name = name
        self.age = age
        self.gender = gender

    def say_hi(self):
        import pdb;pdb.set_trace()
        return f"Hello, my name is {self.name}"

Will result in this output:

import pdb;pdb.set_trace()
        return f"Hello, my name is {self.name}"

class Person:
    def __init__(self, name, age, gender):
        self.name = name
        self.age = age
        self.gender = gender

    def say_hi(self):

This is because it thinks this is a multiline import statement (due to the () character

Desired behavior

The expected output should be:

import pdb

class Person:
    def __init__(self, name, age, gender):
        self.name = name
        self.age = age
        self.gender = gender

    def say_hi(self):
        pdb.set_trace()
        return f"Hello, my name is {self.name}"

Environment

david ~ 1 $ autoimport --version
           autoimport version: 1.0.0
               python version: 3.8.7 (default, Jan  3 2021, 20:50:30)  [GCC 10.2.0]
                     platform: Linux-5.10.89-1-MANJARO-x86_64-with-glibc2.2.5

Integrate autoimport with vim

Feature Request

Define how to use autoimport automatically with vim.

Probably by creating an ale fixer.

Autoimport breaks positional elements (module docstring, shebang, file variables, ...)

Description

When used on a python script, autoimport breaks the shebang and the file-variables line used by Emacs (which must come immediately after the shebang).

Steps to reproduce

Create a script file test.py with contents

#!/usr/bin/env python3
# -*- coding: latin-1 -*-
"""docstring"""
print(os.path.exists("."))

and run autoimport test.py.

Current behavior

The file will now contain

import os

#!/usr/bin/env python3
# -*- coding: latin-1 -*-
"""docstring"""
print(os.path.exists("."))

Not only does this break the shebang, module docstring and file variables, it is also potentially dangerous, because import is an existing executable associated with ImageMagick.

Desired behavior

There are many possible ways changing the position of the imports might break a python file, some of which might depend on the tooling used by the developer.

Imports should therefore be positioned

After all leading comment lines,
At the position where imports are currently located.

Environment

       autoimport version: 0.5.0
           python version: 3.8.5 (default, Sep  4 2020, 07:30:14)  [GCC 7.3.0]
                 platform: Linux-4.12.14-lp151.28.52-default-x86_64-with-glibc2.10

Extend `common_statements`

Description

I'd like to request the ability to extend the common_statements dict.

Possible Solution

One idea is to allow user configuration (via e.g. pyproject.toml or .autoimport.ini or CLI arguments), so that the user can add their own common statements

Additional context

As an example, I find myself typing from numpy import typing as npt quite a bit. It would be nice to automate this with autoimport.

Thanks!

Bug: Import of type "from module import (func1, func2)" can trigger ValueError

Description

Given an import that contains unused function, autoimport exits abnormally.

Steps to reproduce

Given the following code,

from sympy import *
from sympy.parsing.sympy_parser import (parse_expr, standard_transformations, implicit_multiplication_application)
from math import factorial

def fact(n):
    return factorial(n)

def evaluate_expression(expression):
    # replace "!" with the function call to "fact"
    expression = expression.replace('!', ').fact(')
    # add an opening parenthesis before each occurrence of "fact"
    expression = expression.replace('fact', '(fact')
    # replace "√" with the function call to "sqrt"
    expression = expression.replace('√', 'sqrt(') + ')'
    # add an closing parenthesis after each occurrence of "sqrt"
    expression = expression.replace('sqrt', 'sqrt')
    # replace "log10" with the function call to "log"
    expression = expression.replace('log10', 'log(') + ')'
    # add an closing parenthesis after each occurrence of "log"
    expression = expression.replace('log', 'log')

    # parse the expression to sympy and evaluate
    expr = parse_expr(expression, local_dict=namespace, transformations=transformations)

    return expr.evalf()
import unittest

class TestEvaluateExpression(unittest.TestCase):
    def test_simple_expression(self):
        expression = "1+2"
        result = evaluate_expression(expression)
        self.assertEqual(result, 3)

    def test_nested_parentheses(self):
        expression = "(((1+2)*3)^2)"
        result = evaluate_expression(expression)
        self.assertEqual(result, 81)

    def test_exponentiation(self):
        expression = "2^3"
        result = evaluate_expression(expression)
        self.assertEqual(result, 8)

    def test_modulus(self):
        expression = "10%3"
        result = evaluate_expression(expression)
        self.assertEqual(result, 1)

    def test_factorial(self):
        expression = "5!"
        result = evaluate_expression(expression)
        self.assertEqual(result, 120)

    def test_square_root(self):
        expression = "√16"
        result = evaluate_expression(expression)
        self.assertEqual(result, 4)

    def test_logarithm(self):
        expression = "log10(100)"
        result = evaluate_expression(expression)
        self.assertEqual(result, 2)

    def test_complex_expression(self):
        expression = "(((1+2)*3)^2%5)!√4log10(100)"
        result = evaluate_expression(expression)
        self.assertEqual(result, 24)

And we use fix_code to fix it, the autoimport exits with the following traceback:

Traceback (most recent call last):
File "D:\xxx\fix_code_test.py", line 74, in
print(fix_code(code))
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\services.py", line 73, in fix_code
return SourceCode(original_source_code, config=config).fix()
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\model.py", line 67, in fix
self._fix_flake_import_errors()
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\model.py", line 308, in _fix_flake_import_errors
self._remove_unused_imports(import_name)
File "D:\anaconda3\envs\my_env\lib\site-packages\autoimport\model.py", line 478, in _remove_unused_imports
imports.remove(object_name)
ValueError: list.remove(x): x not in list

Plausible cause for this issue:

The _remove_unused_imports function in model.py (lines 441-505) failed to consider the scenario where multiple function names are enclosed in parentheses. Accordingly, given an import like "from module import (func1, func2)", line 476 in _remove_unused_imports will simply separate "(func1, func2)" into "(func1" and "func2)" without removing the parentheses, which trigges an exception of "ValueError: list.remove(x): x not in list" when executing line 478 in _remove_unused_imports.

Allow fixing unicode files with non-ASCII characters on Windows

Description

Cannot run autoimport on a UTF-8 file with cyrillic characters on Windows.

Steps to reproduce

> autoimport <any_file_with_non_ascii>.py

Current behavior

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\Projects\MyProject\.venv\Scripts\autoimport.exe\__main__.py", line 7, in <module>
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\autoimport\entrypoints\cli.py", line 94, in cli
    fixed_code = services.fix_files(flattened_files, config)
  File "C:\Users\user\Projects\MyProject\.venv\lib\site-packages\autoimport\services.py", line 28, in fix_files
    source = file_wrapper.read()
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\encodings\cp1251.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 29668: character maps to <undefined>

Desired behavior

Just works and fixes the file.

Environment

Windows 11
Python 3.9
autoimport==1.3.3

I made a workaround for myself by adding encoding='utf-8' at

autoimport/src/autoimport/entrypoints/cli.py

Line 24 in 485d430

files.append(click.File("r+").convert(py_file, None, None))

            files.append(click.File("r+", encoding='utf-8').convert(py_file, None, None))

and at

autoimport/src/autoimport/entrypoints/cli.py

Line 52 in 485d430

return [click.File("r+").convert(value, param, ctx)]

            return click.File("r+", encoding='utf-8').convert(value, param, ctx)

But it would be better to have a special encoding parameter at the command line arguments and/or in the pyproject.toml configuration file.

Move away from the `xdg` package

Description

https://pypi.org/project/xdg/:

xdg has been renamed to xdg-base-dirs due to an import collision with PyXDG. Therefore the xdg package is deprecated. Install xdg-base-dirs instead.

tl;dr: you'd need to change this

autoimport/pyproject.toml

Line 34 in f0ac03d

"xdg>=5.1.1",

(and that

autoimport/pyproject.toml

Line 382 in f0ac03d

"xdg",

)

and then this

autoimport/src/autoimport/entrypoints/cli.py

Line 9 in f0ac03d

import xdg

to xdg_base_dirs (and the one reference ofc 😅)

Maintenance mode. Help needed!

Hi all, right now I only find time to maintain autoimport this means:

Expect to have an answer to your PR/issue in 2/3 weekdays.
If you open an issue for a new feature or bug fix that I don't feel it's critical, it won't be implemented until you or someone else does a pull request.

If you want to see the project thrive please evaluate becoming a maintainer and help me answering issues, pull requests and develop the program.

This announcement is not meant to prevent you from opening issues! Even though we're in maintenance mode it's always good to do requests, maybe other autoimport users want to implement it :)

lyz-code / autoimport Goto Github PK

autoimport's People

Contributors

Stargazers

Watchers

Forkers

autoimport's Issues

Description

Possible Solution

Description

Possible Solution

Additional context

Issue

Details

Impact

Remediation

Description

Possible Solution

Description

Possible Solution

Additional context

Related Issue

Description

Steps to reproduce

Current behavior

Desired behavior

Environment

Description

Desired behavior

Description

Possible Solution

Additional context

Related Issue

Description

Possible Solution

Description

Desirable behavior

Actual behavior

Interaction with explicit imports

Full example traceback

Version info

Description

Steps to reproduce

Desired behavior

Environment

Description

Steps to reproduce

Current behavior

Desired behavior

Description

Steps to reproduce

Input file:

Current behavior

Output file:

Desired behavior

Environment

Description

Possible Solution

Description

Steps to reproduce

Current behavior

Desired behavior

Environment

Context:

Environment:

Description

Steps to reproduce

Case 1:

Case 2:

Case 3 (this is expected behavior):

Current behavior

Desired behavior

Environment

Description

Possible Solution

Additional context

Related Issue

Description

Possible Solution

Additional context