Comments (6)
This is an interesting feature request. I think it shouldn't be on by default; the default behavior to not show overlapping results is I think the most common need, and, like you say, the fastest.
We can't simply try all possible substrings of a found match. For example, that would mean that foobar.com
would also give bar.com
, oobar.com
, and so on. So we need something a bit smarter.
Perhaps we can limit this feature to at most one nested match within a URL's "path" element (including ?queries
etc). That might work, but I wonder how common of a need that is.
Finally, I'm not sure how to expose this in the API or in the command-line tool. Any ideas?
from xurls.
I fully agree this should not be the default.
You are right about the "overlapping" problem to consider.
By "default" now, the requested behaviour is using the space etc as "delimiter".
It may be possible to define a option with an argument defining an additional "new" separator.
-s "/" could mean "use / also as a separator"
$ echo "http://www.fakedomain.com/account/legitdomain.com/folder" | bin/xurls -r -s "/"
http://www.fakedomain.com/account/legitdomain.com/folder
legitdomain.com/folder
$
I fully agree too this requirement is not very common.. Was just an idea as I came across this specific case.. ;)
Thank's
from xurls.
This has nothing to do with separators, though. Your suggestion above would mean we'd support overlapping matches, which we don't now. If anything, adding /
as an optional separator would just break all paths.
from xurls.
Yes, suggestion is optional overlapping match.
Now:
$ echo "foo/domainB.com/bar" | bin/xurls -r
domainB.com/bar
$
$ echo "domainA.com/foo/domainB.com/bar" | bin/xurls -r
domainA.com/foo/domainB.com/bar
$
In this example, domainB.com is identified only when there is no "overlap".
Suggestion about /
is like "replace all / with space/ and check again if something is found:
"domainA.com /foo/domainB.com/bar"
"domainA.com/foo /domainB.com/bar"
"domainA.com/foo/domainB.com /bar"
then "reassemble" removing the extra space (to have the first full path with dir)
from xurls.
What you're suggesting there would be a recursive search; at best, that would be quadratic complexity, and quite a lot of added code for a niche use case.
I've decided to not implement this for now. It should be fairly easy to implement this outside xurls. If you find that the API is nice and the code is generally useful, we can look at adding that later, but for now I don't think it's worth the effort.
from xurls.
That's OK. Thank's
from xurls.
Related Issues (20)
- parsing issue with json file HOT 2
- missing postgresql in schemes HOT 1
- Dangling dots, mid-string, are seen as domains HOT 5
- generate: concurrent map write
- Managing duplicate URLs? HOT 4
- Email support HOT 6
- go: error loading module requirements HOT 3
- cmd/xurls: -fix eats input when URLs get longer
- avoid Relaxed from matching trailing TLDs without a word break HOT 2
- tag request HOT 1
- Static Urls HOT 1
- Issue with Email Addresses HOT 7
- [Bug] - Identifying tel:654654 as URL HOT 1
- go get -u ? HOT 1
- Relaxed mode is too relaxed HOT 3
- xurls does not recognize valid IRIs HOT 1
- authority component parsing does not align with RFC 3987 HOT 3
- make a deterministic variant of "go generate" and have CI check it's up to date HOT 1
- character ranges can optionally be limited to ascii HOT 2
- add a mode to only get relative urls? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xurls.