Convert unicode URLs to punycode about furl HOT 6 CLOSED

furl v0.5.1 is hot of the presses and supports Unicode hosts via Python's IDNA
support.

>>> import furl
>>> furl.__version__
'0.5'
>>> f = furl.furl('http://xn--4cab6c.se/')
>>> f.host
'åäö.se'
>>> f.set(host=u'ドメイン.テスト').url
'http://xn--eckwd4c7c.xn--zckzah/'

Update with

pip install --upgrade furl

I deferred IDNA2008 support. We'll cross that bridge when it becomes an issue.

Thank you for bringing this issue to my attention Jacob. Don't hesitate to let
me know if there's anything else I can do for you.

from furl.

Wonderful suggestion. furl should absolutely support non-ASCII hosts via IDNA
encoding.

I'll add this shortly. Or, pull requests welcome.

That said, IDNA support is a tad delicate.

1. Python's native idna implementation isn't up to spec: it supports IDNA2003
   but not the latest IDNA2008. See http://bugs.python.org/issue17305.

2. The idna library (https://pypi.python.org/pypi/idna) supports the latest
   IDNA2008, but furl should require as few dependencies as possible. At best,
   the idna library becomes an optional dependency.

3. As I've encountered independently, the idna library's idna.encode() barfs
   on perfectly valid URLs if the URL resembles an IDNA encoded URL. For
   example, idna.encode() barfs on the perfectly valid URL u'xx--åäö.se':

    >>> import idna
    >>> idna.encode(u'xx--åäö.se')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/local/lib/python2.7/dist-packages/idna/core.py", line 355, in encode
        result.append(alabel(label))
      File "/usr/local/lib/python2.7/dist-packages/idna/core.py", line 276, in alabel
        check_label(label)
      File "/usr/local/lib/python2.7/dist-packages/idna/core.py", line 239, in check_label
        check_hyphen_ok(label)
      File "/usr/local/lib/python2.7/dist-packages/idna/core.py", line 134, in check_hyphen_ok
        raise IDNAError('Label has disallowed hyphens in 3rd and 4th position')
    idna.core.IDNAError: Label has disallowed hyphens in 3rd and 4th position

   If furl uses the idna library, it needs to account for this behavior.

from furl.

Interesting, didn't know about # 3. Perhaps this is something that should be fixed in the idna library in itself though?

from furl.

Excellent, seems to be working well for my use cases at least 😀

from furl.

Wunderbar.

Let me know if you encounter any problems or have any other comments,
suggestions, or recommendations. Feedback is how good products become great.

from furl.

Will do! Thanks for the implementation!

from furl.