Comments (7)
Okay that makes sense, It's probably something we broke. I'll start looking into this, thanks!
from trurl.
@abbbi I opened #276 which i think should fix the issues you're seeing when doing the builds. Let me know if this is doesn't work for you.
from trurl.
it appears the tests fail if LC_ALL=C is set in the build environment :-)
root@cefix:/trurl# export LC_ALL=C
root@cefix:/trurl# make test >/dev/null 2>&1; echo $?
2
root@cefix:/trurl# export -n LC_ALL
root@cefix:/trurl# make test >/dev/null 2>&1; echo $?
0
locale set in LC_* must be UTF8 (LC_ALL=C.UTF8 will work)
from trurl.
This is interesting, thanks for digging in. Just be clear, these tests past under the same paramaters in previous versions of trurl? I would need to look at what libcurl / libidn is doing, but I would imagine we may need to convert to utf8 before trying to call the lookup function. Here is the line where libcurl actually makes the call, and according the docs it's expecting a UTF-8 encoded string: https://www.gnu.org/software/libidn/libidn2/manual/html_node/Library-Functions.html#Core-Functions
Perform IDNA2008 lookup string conversion on domain name src , as described in section 5 of RFC 5891. Note that the input string must be encoded in UTF-8 and be in Unicode NFC form.
it looks like idn2 has some functions for converting from the current locale to utf8 which may just need to be called before libcurl passes the data into the lookup function above. Another hacky "solution" is just have trurl check if its using utf8 and if not, disable the punycode features.
its strange this hadn't popped up in the past, so it very well may be something silly happening on the trurl side of things.
from trurl.
have not tested or seen the issue with older releases. For building Debian packages we usually use chroots and tooling like cowbuilder or debspawn. I have changed my build environment to use cowbuilder, which seems to set defaults to LC_ALL=C, thus its the first time the issue hit me.
from trurl.
Okay, it looks like after some digging this has been an issue for a while (probably since punycode was introduced to trurl).
it appears the tests fail if LC_ALL=C is set in the build environment :-)
It turns out it doesn't matter what it is in the build environment, it happens at run time. the area where this issue takes place is here where it sets the locale to whatever the global locale is:
Line 1548 in 1074fef
Earlier I had said it seems like it could be an issue with libcurl, but it looks like they are already handling locale stuff fine - I got confused because there were a lot of functions that do pretty similar things in libidn2. There are two general paths I think we can take:
- Modify the tests to either skip them tests that require UTF8 if the global locale doesn't support UTF8 or override the LC_ALL environment variables. I'm leaning towards the former because these tests could be ran on a machine where
*.UTF8
may not exist, so just overriding the locale may result in more failures. - If we are not in a UTF8 locale then don't report
punycode
as a feature in the output for--version
.
from trurl.
It turns out it doesn't matter what it is in the build environment, it happens at run time. the area where this issue takes place is here where it sets the locale to whatever the global locale is
yes, sorry, my last post might be a bit misleading. Build + test are run in the same environment during automatic
debian package built, so yes, this is an runtime error, not an build issue.
from trurl.
Related Issues (20)
- trurl's checksrc.pl is outdated
- Add more options for configuring curl_url_set() HOT 6
- `trurl --trim scheme`? HOT 3
- Query params values in JSON output have NUL replaced by . HOT 4
- Shell Mode Feature HOT 1
- Test fails on windows because of different null device name HOT 4
- Please support zero-sized fragment and query HOT 9
- unable to selectively remove query parameters with asterisk in them HOT 5
- Invalid characters in scheme give wrong error HOT 2
- test fails if built against libcurl 8.3.0 HOT 4
- new version release with recent fixes? HOT 1
- About the release scheme HOT 6
- Components with control characters don't appear in `--json` output, and non-urlencoded `--get` fails HOT 3
- [FR] allow JSON input in the same form as JSON parts output HOT 3
- incorrect decode of %3d as = character in query string leads to out of bound reads
- incorrect decode of %00 in query string leads to out of bound reads when printing the param key HOT 2
- incorrect handling of strings with null characters in memdupdec causes out of bounds write to adjacent memory region HOT 1
- Allow bundling options and arguments
- Read from stdin by default HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trurl.