Breaks JSON package in Python 3.7 on Arch linux about parselmouth HOT 9 CLOSED

Fixed in version 0.3.3, that I've finally managed to release. Sorry for the delay; it was a horrible job to get rid of those setlocale calls, but it should be working now?

Thank you very much for reporting!

from parselmouth.

Didn't work?? Sigh, I hate locales, by now.

The other temporary workaround hack (that works on my machine; but in this case, that's no guarantee, it seems) is calling import locale and locale.setlocale(locale.LC_ALL, "C.UTF-8") after the first import of Parselmouth. As long as you keep the "C" part, the internal Praat functionality shouldn't break.

from parselmouth.

I was a bit unclear, sorry ! I didn't try to use subprocess yet, I just changed where I was setting up the LC_ALL and LANG variables (which were set in the Dockerfile, although I'm loading my private env vars from a .env file) but I think it was a kinda stupid move anyway (I was short on time haha, didn't think that through).

However, the locale workaround seemed to do did the trick for me ! Thanks a lot 😃

from parselmouth.

Hi @MichaelGoodale, thanks for reporting!

This should not be happening; especially because I am not using or even importing the json standard library in Parselmouth.

However, luckily, I managed to reproduce it (for a file with non-ascii characters), so I'll start investigating which parts of Parselmouth are causing this. I'll get back to you once I know more ;-)

from parselmouth.

Updates:

• It's not something specific to the json module; calling open("main_survey", "r").read() also fails
• The same problem also occurs on Python 3.6, so it's not 3.7-specific
• I've traced down the problem to the initialization of Praat. Without that initialization, importing Parselmouth does not change anything.

from parselmouth.

@MichaelGoodale A temporary workaround should be adding the encoding='utf-8' keyword argument to your call to open, by the way.

from parselmouth.

Hi,

Edit: didn't see you already traced the bug. Good luck with fixing it ! :)

Previous message:

I got a similar problem using tempfile.NamedTemporaryFile. When Parselmouth is imported in my project, calling os.popen(some subprocess).read() with the file opened throws the same error. Setting the encoding to utf-8 doesn't solve the issue ...

The code:

with tempfile.NamedTemporaryFile(mode='w+t', encoding='utf-8') as temp:
    temp.write(args.text)
    temp.seek(0)
    process = os.popen('java -jar utils/languagetool/languagetool-commandline.jar --encoding utf-8 --language fr --json %s' % temp.name)
    analysis = process.read()
    process.close()

Various workarounds don't seem to solve the issue (adding .encode(...), .decode(...), using another method ...).

from parselmouth.

@Thooto Thanks for adding this to the issue; I'm working on it for the moment, but it's a bit more complicated than I'd hoped for, involving locales and locale-independent reading/writing of doubles :-(

It's the call to process.read() on the result of os.popen(...) that seems to be the problem - I can reproduce this with os.popen('ls').read(). One temporary workaround, until this gets hopefully fixed soon, is to use the subprocess library where you can either set the encoding or use .decode on the returned string.

One thing to be aware of, is that if Parselmouth breaks this code because of Praat setting the locales, your code can be broken when used in other applications as well.

from parselmouth.

Thanks for your quick answer ! I'll try to use the subprocess module instead.

I'm actually using a Flask app running in a docker container, where I had both LC_ALL and LANG "env" set to C.UTF-8 as required by Flask. I'll try loading them later on with the other env vars.

Edit: didn't change anything 🤣 (moving these vars from Dockerfile to .env file)

from parselmouth.