Comments (6)
Original comment by Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):
it sounds like this is because an __init__.py
file contains non ascii string, and attempt to read its content by lgc.modutils fails in such case. This needs deeper investigation for a proper fix.
from astroid.
Original comment by Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):
I can't reproduce it using latest astroid / pylint. Also, pylint knows how to handle files in unicode format. Maybe you used an older version of astroid?
from astroid.
Original comment by Duncan Booth (BitBucket: duncan_booth):
I came across this problem running pylint 1.3.1, astroid 1.2.1 (so the current versions) when running pylint over a package that depends on Unidecode 0.04.16.
The Unidecode/__init__.py
file contains non-ascii characters.
This is not an issue when I run pylint from the bash shell with the environment variable LANG=en_US.UTF-8
, however when run as a sub-process inside the editor I use, the editor sets LANG=C
and that causes astroid to fail when attempting to read the non-ascii __init__.py
file.
#!sh
(wyatt)duncan@ubuntu:~$ cat test.py
from unidecode import unidecode
(wyatt)duncan@ubuntu:~$ LANG=en_US.UTF-8 pylint test.py
No config file found, using default configuration
************* Module test
C: 1, 0: Missing module docstring (missing-docstring)
W: 1, 0: Unused import unidecode (unused-import)
Report
======
2 statements analysed.
Messages by category
--------------------
+-----------+-------+---------+-----------+
|type |number |previous |difference |
+===========+=======+=========+===========+
|convention |1 |1 |= |
+-----------+-------+---------+-----------+
|refactor |0 |0 |= |
+-----------+-------+---------+-----------+
|warning |1 |1 |= |
+-----------+-------+---------+-----------+
|error |0 |0 |= |
+-----------+-------+---------+-----------+
Messages
--------
+------------------+------------+
|message id |occurrences |
+==================+============+
|unused-import |1 |
+------------------+------------+
|missing-docstring |1 |
+------------------+------------+
Global evaluation
-----------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)
Duplication
-----------
+-------------------------+------+---------+-----------+
| |now |previous |difference |
+=========================+======+=========+===========+
|nb duplicated lines |0 |0 |= |
+-------------------------+------+---------+-----------+
|percent duplicated lines |0.000 |0.000 |= |
+-------------------------+------+---------+-----------+
External dependencies
---------------------
::
unidecode (test)
Raw metrics
-----------
+----------+-------+-------+---------+-----------+
|type |number |% |previous |difference |
+==========+=======+=======+=========+===========+
|code |2 |100.00 |2 |= |
+----------+-------+-------+---------+-----------+
|docstring |0 |0.00 |0 |= |
+----------+-------+-------+---------+-----------+
|comment |0 |0.00 |0 |= |
+----------+-------+-------+---------+-----------+
|empty |0 |0.00 |0 |= |
+----------+-------+-------+---------+-----------+
Statistics by type
------------------
+---------+-------+-----------+-----------+------------+---------+
|type |number |old number |difference |%documented |%badname |
+=========+=======+===========+===========+============+=========+
|module |1 |1 |= |0.00 |0.00 |
+---------+-------+-----------+-----------+------------+---------+
|class |0 |0 |= |0 |0 |
+---------+-------+-----------+-----------+------------+---------+
|method |0 |0 |= |0 |0 |
+---------+-------+-----------+-----------+------------+---------+
|function |0 |0 |= |0 |0 |
+---------+-------+-----------+-----------+------------+---------+
(wyatt)duncan@ubuntu:~$ LANG= pylint test.py
No config file found, using default configuration
************* Module test
C: 1, 0: Missing module docstring (missing-docstring)
Traceback (most recent call last):
File "/home/duncan/.virtualenvs/wyatt/bin/pylint", line 9, in <module>
load_entry_point('pylint==1.3.1', 'console_scripts', 'pylint')()
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/__init__.py", line 21, in run_pylint
Run(sys.argv[1:])
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/lint.py", line 991, in __init__
linter.check(args)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/lint.py", line 585, in check
self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/lint.py", line 662, in check_astroid_module
walker.walk(astroid)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/utils.py", line 804, in walk
self.walk(child)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/utils.py", line 801, in walk
cb(astroid)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/checkers/imports.py", line 270, in visit_from
self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name))
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/pylint/checkers/imports.py", line 304, in _add_imported_module
importedmodname = get_module_part(importedmodname)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/astroid/modutils.py", line 330, in get_module_part
context_file=context_file)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/astroid/modutils.py", line 275, in file_from_modpath
return _file_from_modpath(modpath, path, context)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/astroid/modutils.py", line 482, in _file_from_modpath
mtype, mp_filename = _module_file(modpath, path)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/site-packages/astroid/modutils.py", line 603, in _module_file
data = stream.read(4096)
File "/home/duncan/.virtualenvs/wyatt/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 449: ordinal not in range(128)
(wyatt)duncan@ubuntu:~$
from astroid.
Original comment by Vadim Markovtsev (BitBucket: vmarkovtsev, GitHub: @vmarkovtsev?):
I have this issue, too. It breaks my CI distributed builds in Jenkins.
Developers, do you know anything about default codecs? Let me explain you...
Usually, the default codec is utf-8 - in terminal, etc. This is why you are not able to reproduce the problem "using latest astroid / pylint".
But, consider the scenario when pylint is run through a non-interactive SSH session. Not OpenSSH, but things like Paramiko or JSCh. Why this is important, you would ask? Because, OpenSSH is smart enough to pass over environment variables like LC_* and LANG which define the locale and default encoding which python3 uses, but this is not the case with other dumb SSH clients.
When Python3 does not recognize a sane environment, it defaults to (tada!) ascii encoding. And (oh shit) even the following code will fail:
#!python
with open("file_with_utf8_chars.txt", "r") as f:
mystr = f.read()
File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 50864: ordinal not in range(128)
Yes, this is bad, bad Python3 habit. And yes, many people complain about it, see e.g. this link (ctrl-f ssh).
How to reproduce it? Set the default encoding to ascii (LANG= as suggested by Duncan).
How to fix it? Read the docs and open('file', encoding='utf-8')
from astroid.
Original comment by C N Kiser (BitBucket: ckiser):
I am still having this problem - because of Jenkins. On the server I can run pylint from the virtualenv my Jenkins server creates. Works fine if I am on the command line and fails when I try to execute the same command from Jenkins. Since I am building from a bash script, I can add export LANG=en_US.UTF-8
to the script and it looks like that fixes the problem. But it would be great if this worked out of the box if possible.
from astroid.
Original comment by Florian Bruhin (BitBucket: The-Compiler, GitHub: @The-Compiler?):
How to fix it? Read the docs and open('file', encoding='utf-8')
That's not really a good fix, IMHO. Just using UTF-8 is just a guess, like open without an encoding (i.e. the system's default encoding) is as well.
The real solution (if pylint doesn't yet) would be to use tokenize.open on Python3, which detects the PEP263 encoding markers and opens the file correctly, even if it ends up having an encoding: latin1
marker ;)
That being said, I have source files with UTF-8 in them, and I can run pylint on them just fine with LANG=C
(pylint 1.4.3, astroid 1.3.6, common 0.63.2).
from astroid.
Related Issues (20)
- 2.15.7: sdist is still missing `tox.ini` HOT 3
- Regression in Astroid version 2.15.7 in handling subscriptable type parameters HOT 2
- `sphinx-autoapi` after astroid `v3.0.0` HOT 1
- A dataclass of type ``enum.Enum`` does not contain the enum ``__members__`` object HOT 4
- astroid depends on 'crypt', which is removed in Python 3.13 HOT 4
- Pyreverse does not show return type for property methods HOT 1
- Backstory behind license being LGPL? HOT 1
- Explore using `optimize` argument of `ast.parse()` in Python 3.13
- Test failures on Python 3.13 with astroid 3.0.2 HOT 4
- PEP 695: infer generic bases
- Editable installs don't expose `__version__`, break pylint HOT 4
- Exception 'RecursionError: maximum recursion depth exceeded' in astroid decorators with astroid 3.1.0 HOT 1
- test_pydantic_field failes HOT 2
- `test_no_user_warning` assumes pip is available HOT 1
- sphinx warnings `reference target not found` HOT 4
- TryStar.ExceptHandler incorrectly infers types of caught exceptions
- `six.moves` brain is not effective when `six.moves` was already imported in interpreter running astroid HOT 1
- astroid crashes with RecursionError in a pytest session if schemathesis is installed HOT 5
- Test failures against NumPy 2.0.0rc2 HOT 1
- false not-an-iterable for __truediv__ with @overload HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from astroid.