Matching returns wrong url about xurls HOT 6 CLOSED

Thanks for filing this issue. This is a bit of an edge case, because characters like | and , are valid and even common in URLs, but they're also very common in plaintext.

The strategy I ended up with is to accept them as "middle" characters in an URL, but not as the last character. For example:

$ echo "http://foo| bar" | xurls
http://foo
$ echo "http://foo|bar" | xurls
http://foo|bar
$ echo "http://foo, bar" | xurls
http://foo
$ echo "http://foo,bar" | xurls
http://foo,bar

I could make the URL matching more conservative, and to always cut off commas and vertical bars. However, that would break perfectly valid URLs like https://en.wikipedia.org/wiki/Colma,_California, for example. As I'm writing this, I wonder if GitHub will auto-link that properly :)

You can also find real examples using vertical bars, like https://fonts.googleapis.com/css?family=Lato:400,700,400italic,700italic%7CRoboto+Slab:400,700%7CInconsolata:400,700. In both cases, note how modern browsers don't escape the character.

I think the current mechanism is an OK middle ground. If you can provide a real example, perhaps there's some tweak we could make. With the http://google.com|google.com example you gave above, I don't think there's anything we can do without breaking perfectly valid URLs.

from xurls.

I forgot to say - if you would have plaintext that you know makes heavy use of certain characters like |, what you could do is use strings.Split first, then pass each part through xurls later.

from xurls.

Ping @jlory - any thoughts?

from xurls.

So the story behind my findings with the "|" is I'm using the Slack API to parse some messages and I found out that they rewrite URL links in messages using | sometimes: https://api.slack.com/docs/message-formatting#linking_to_urls

In your example the pipe character is escaped and replaced by the proper value, as I'm reading this: https://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid it's still unclear if we should exclude it, I've never seen a URL / URI with | in it.

As for now I'm actually doing a string split with | and discarding the rest.

from xurls.

Hmm, you're right - that's a good example of | being used to separate URLs. I haven't myself found a use case for vertical bars to be part of a URL, so I'll make this change.

If anyone runs into regressions because of it, they can file an issue, and we can reconsider reverting the commit at that point.

from xurls.

Ah, this was never intended to work like this. I simply added a | in the wrong place - between [ and ], effectively adding it to a character set by mistake.

from xurls.