Comments (2)
bram-tv
commented on June 8, 2024
Looking at the is_skipped code in settings.py, it ends with:
Line 646 in 7de1829
...
# git_ls_files are good files you should parse. If you're not in the allow list, skip.
if (
git_folder
and not file_path.is_dir()
and str(file_path.resolve()) not in self.git_ls_files[git_folder]
):
return True
return FalseWhen the file_path is a directory it will return False which will cause it to traverse the directory.
A possible fix: when file_path is a directory then check if there git_ls_files contains a file inside that directory.
Something like:
diff --git a/isort/settings.py b/isort/settings.py
index a3658fff..67acdbf5 100644
--- a/isort/settings.py
+++ b/isort/settings.py
@@ -641,11 +641,14 @@ class Config(_Config):
# git_ls_files are good files you should parse. If you're not in the allow list, skip.
- if (
- git_folder
- and not file_path.is_dir()
- and str(file_path.resolve()) not in self.git_ls_files[git_folder]
- ):
+ if git_folder:
+ if file_path.is_dir():
+ dir = file_path.resolve()
+ if not any(file.startswith(f"{dir}/") for file in self.git_ls_files[git_folder]):
+ return True
+
+ elif str(file_path.resolve()) not in self.git_ls_files[git_folder]:
+ return True
return True
return FalseWith the example repository listed in the description the output is:
...
sub3 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
sub2 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
sub1 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled....
...
which is good.
However it's not yet perfect...
When the repository is changed into:
tree -a -I '.git'
.
├── bar.py
├── dir1
│ ├── sub1
│ │ ├── dir1_sub1_file1.py
│ │ └── dir1_sub1_file2.py
│ ├── sub2
│ │ ├── dir1_sub2_file1.py
│ │ └── dir1_sub2_file2.py
│ └── sub3
│ ├── dir1_sub3_file1.py
│ └── dir1_sub3_file2.py
├── foo.py
└── .gitignore
4 directories, 9 files
where the .gitignore file now contains:
dir1/sub1/
dir1/sub2/
dir1/sub3/
then the output is:
...
dir1 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
...
which is a bit confusing: in the .gitignore file it's specified to ignore dir1/sub1/, dir1/sub2 and dir1/sub3 but the isort output makes it appear as if the entire directory was ignored..
In a way it was because there were no other files in dir1/ but still the message doesn't "feel" right..
from isort.
bram-tv
commented on June 8, 2024
Also: note that this solution is still suboptimal is some cases: it can still cause too many files/directories to be traversed.
Assume the layout:
$ tree -a -I '.git'
.
├── bar.py
├── dir1
│ ├── sub1
│ │ ├── dir1_sub1_file1.py
│ │ ├── dir1_sub1_file2.py
│ │ ├── subsub1
│ │ │ └── dir1_sub1_subsub1_file_1.py
│ │ ├── subsub2
│ │ │ └── dir1_sub1_subsub2_file_1.py
│ │ ├── subsub3
│ │ │ └── dir1_sub1_subsub3_file_1.py
│ │ ├── subsub4
│ │ │ └── dir1_sub1_subsub4_file_1.py
│ │ ├── subsub5
│ │ │ └── dir1_sub1_subsub5_file_1.py
│ │ ├── subsub6
│ │ │ └── dir1_sub1_subsub6_file_1.py
│ │ ├── subsub7
│ │ │ └── dir1_sub1_subsub7_file_1.py
│ │ ├── subsub8
│ │ │ └── dir1_sub1_subsub8_file_1.py
│ │ └── subsub9
│ │ └── dir1_sub1_subsub9_file_1.py
│ ├── sub2
│ │ ├── dir1_sub2_file1.py
│ │ └── dir1_sub2_file2.py
│ └── sub3
│ ├── dir1_sub3_file1.py
│ └── dir1_sub3_file2.py
├── foo.py
└── .gitignore
13 directories, 18 files
And assume the file dir1/sub1/subsub5/dir1_sub1_subsub5_file_1.py was committed (despite being ignored, i.e. it was added using git add -f)
And .gitignore contains:
dir1/sub1/
dir1/sub2/
dir1/sub3/
The git ls-files output for reference:
$ git ls-files
.gitignore
bar.py
dir1/sub1/subsub5/dir1_sub1_subsub5_file_1.py
foo.py
$ git ls-files --others --exclude-standard
$
then isort output is:
...
sub3 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
sub2 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub1 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub7 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub2 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub4 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub9 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub3 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub6 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
subsub8 was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
dir1_sub1_file2.py was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
dir1_sub1_file1.py was skipped as it's listed in 'skip' setting, matches a glob in 'skip_glob' setting, or is in a .gitignore file with --skip-gitignore enabled.
...
so it still did some unneeded directory traversal.
I'm not sure if the structure of the code allows this to be (easily) fixed tho..
We know the files that may need to be checked (i.e. the files in self.git_ls_files) but that's not how the code loops..
from isort.
Related Issues (20)
- New Yarg dependency adds `site-packages/tests` directory HOT 4
- pip install "isort[colors]" no longer installs colorama HOT 3
- Incorrect identification of first and third party modules
- It seems `--jobs` and `--diff` cannot be used together
- try isort page is b0rk
- from xyz import (abc, # isort:skip yo, hey) HOT 1
- Can we add a json schema to complete pyproject.toml's [tool.isort]?
- isort 5.11.2 fails to install in pre-commit HOT 4
- Import in one line if the import fits within the line-length HOT 4
- files configuration option missing from config file(s) HOT 2
- Non-idempotency with commented multiline imports
- [self tests] test_black_pyi_file fails with black 24.2.0
- [self tests] test_isort_should_warn_on_empty_custom_config_issue_1433 fails against Pytest 8
- Sorting order is broken and not following standard sort order HOT 13
- Third-party incorrectly identify as First-party depending on the name of the root directory
- isort.core.process incorrectly returns False if only changes to lines before imports were made, with no changes to raw import section
- Ran into `ERROR: Unrecoverable exception thrown when parsing <filepath>`
- Feature Request: Improved Handling of `# type: ignore` Comments to Preserve Mypy Compatibility
- Different order on same file depending on where you execute isort v5.13.2 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from isort.