Comments (4)
OJ
commented on May 11, 2024
1
Thanks for chiming in @justinsteven. I did want gobuster to be as "unixy" as possible (ie. it just does one thing), knowing that there are indeed a bunch of other ways to do exactly this. parallel is one of them, and there are others.
I appreciate the thorough runthrough with samples! They are super helpful.
I'm going to close this issue as "sorry, but no" :)
PS. @justinsteven I'm fine with those terms ;P
from gobuster.
OJ
commented on May 11, 2024
This has been discussed, lightly, in #11. I'm actually not 100% behind the idea of adding this for a few reasons, one of which you've stumbled on in your PR which is to do with dealing with stdin at the same time.
The problems appear thanks to concurrency. Personally, I do not want to see the output of a bunch of sites interleaved, which means segregating based on host name. If we do that, we open up a whole new world of hurt.
The original intent was to keep the design (and implementation) of this tool relatively simple, without turning it into a dir/dns nmap.
I'll think on this a little more. Cheers!
from gobuster.
gehaxelt
commented on May 11, 2024
Hey,
oh, I didn't see that his had already been discussed. Sorry!
[...] one of which you've stumbled on in your PR which is to do with dealing with stdin at the same time.
Well, I'd say if you want to use stdin, then you can still use the standard single processing mode, can't you?
[...] which means segregating based on host name. If we do that, we open up a whole new world of hurt. [...] The original intent was to keep the design (and implementation) of this tool relatively simple, without turning it into a dir/dns nmap.
I wouldn't say that it got much more complicated and backward compatibility still exists, but I get your point(s).
Take your time and decide whatever your heart says is the correct decision :)
Personally, I think this feature might improve my workflow (going through one single file instead of hundreds of files), so I'll eventually merge that into my fork's master.
Thanks for the quick response.
from gobuster.
justinsteven
commented on May 11, 2024
Another alternative: Gnu Parallel. As a bonus, it can gobust* in parallel.
15:51:02[justin@diablo ~]% parallel ~/opt/gobuster/gobuster -u {} -w ~/opt/SecLists/Discovery/Web_Content/apache.txt ::: ring0.lol justinsteven.com
When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
This helps funding further development; and it won't cost you a cent.
Or you can get GNU Parallel without this requirement by paying 10000 EUR.
To silence this citation notice run 'parallel --bibtex' once or use '--no-notice'.
Gobuster v1.1 OJ Reeves (@TheColonial)
=====================================================
[+] Mode : dir
[+] Url/Domain : http://ring0.lol/
[+] Threads : 10
[+] Wordlist : /home/justin/opt/SecLists/Discovery/Web_Content/apache.txt
[+] Status codes : 204,301,302,307,200
=====================================================
/cgi-bin (Status: 301)
/access_log (Status: 301)
/.htaccess (Status: 301)
/cgi (Status: 301)
/.htpasswd (Status: 301)
/dummy (Status: 301)
/.meta (Status: 301)
/cgi-pub (Status: 301)
/cgi-script (Status: 301)
/.web (Status: 301)
/error (Status: 301)
/error_log (Status: 301)
/htdocs (Status: 301)
/httpd (Status: 301)
/phf (Status: 301)
/icons (Status: 301)
/manual (Status: 301)
/httpd.pid (Status: 301)
/logs (Status: 301)
/index.html (Status: 301)
/printenv (Status: 301)
/status (Status: 301)
/server-status (Status: 301)
/server-info (Status: 301)
/~nobody (Status: 301)
/tmp (Status: 301)
/~bin (Status: 301)
/~root (Status: 301)
/~ftp (Status: 301)
/test-cgi (Status: 301)
=====================================================
Gobuster v1.1 OJ Reeves (@TheColonial)
=====================================================
[+] Mode : dir
[+] Url/Domain : http://justinsteven.com/
[+] Threads : 10
[+] Wordlist : /home/justin/opt/SecLists/Discovery/Web_Content/apache.txt
[+] Status codes : 200,204,301,302,307
=====================================================
/.htaccess (Status: 301)
/.htpasswd (Status: 301)
/access_log (Status: 301)
/dummy (Status: 301)
/cgi (Status: 301)
/.web (Status: 301)
/.meta (Status: 301)
/cgi-pub (Status: 301)
/cgi-script (Status: 301)
/cgi-bin (Status: 301)
/error (Status: 301)
/logs (Status: 301)
/htdocs (Status: 301)
/httpd.pid (Status: 301)
/icons (Status: 301)
/manual (Status: 301)
/index.html (Status: 301)
/error_log (Status: 301)
/httpd (Status: 301)
/phf (Status: 301)
/printenv (Status: 301)
/status (Status: 301)
/server-info (Status: 301)
/~nobody (Status: 301)
/~bin (Status: 301)
/~root (Status: 301)
/tmp (Status: 301)
/test-cgi (Status: 301)
/server-status (Status: 301)
/~ftp (Status: 301)
=====================================================
Want to keep an eye on how the gobusting* is going? Redirect the output to files that include the name of the site being gobusted*, being sure to escape the > so that Parallel catches it.
15:55:43[justin@diablo /tmp/tmp.VxovyEzHnR]% parallel ~/opt/gobuster/gobuster -u {} -w ~/opt/SecLists/Discovery/Web_Content/raft-large-directories.txt \> gobuster_{}.log ::: ring0.lol justinsteven.com
When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
This helps funding further development; and it won't cost you a cent.
Or you can get GNU Parallel without this requirement by paying 10000 EUR.
To silence this citation notice run 'parallel --bibtex' once or use '--no-notice'.
And while it's running:
15:56:10[justin@diablo /tmp/tmp.VxovyEzHnR]% tail -n5 *
==> gobuster_justinsteven.com.log <==
/_temp (Status: 301)
/import (Status: 301)
/_derived (Status: 301)
/webstat (Status: 301)
/generator (Status: 301)
==> gobuster_ring0.lol.log <==
/portal (Status: 301)
/_derived (Status: 301)
/import (Status: 301)
/generator (Status: 301)
/webstat (Status: 301)
*The terms gobust, gobusting and gobusted are pending branding approval from @OJ
Edit: You can feed your list of sites to Parallel's STDIN, too
15:58:33[justin@diablo /tmp/tmp.VxovyEzHnR]% cat <<EOF > sitelist.txt
heredoc> ring0.lol
heredoc> justinsteven.com
heredoc> EOF
16:03:35[justin@diablo /tmp/tmp.VxovyEzHnR]% cat sitelist.txt | parallel ~/opt/gobuster/gobuster -u {} -w ~/opt/SecLists/Discovery/Web_Content/apache.txt
When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
This helps funding further development; and it won't cost you a cent.
Or you can get GNU Parallel without this requirement by paying 10000 EUR.
To silence this citation notice run 'parallel --bibtex' once or use '--no-notice'.
Gobuster v1.1 OJ Reeves (@TheColonial)
=====================================================
[+] Mode : dir
[+] Url/Domain : http://justinsteven.com/
[+] Threads : 10
[+] Wordlist : /home/justin/opt/SecLists/Discovery/Web_Content/apache.txt
[+] Status codes : 302,307,200,204,301
=====================================================
/cgi (Status: 301)
/.htaccess (Status: 301)
/access_log (Status: 301)
/.web (Status: 301)
/.meta (Status: 301)
/cgi-pub (Status: 301)
/dummy (Status: 301)
/.htpasswd (Status: 301)
/cgi-bin (Status: 301)
/cgi-script (Status: 301)
/error (Status: 301)
/error_log (Status: 301)
/index.html (Status: 301)
/httpd (Status: 301)
/manual (Status: 301)
/icons (Status: 301)
/htdocs (Status: 301)
/logs (Status: 301)
/httpd.pid (Status: 301)
/phf (Status: 301)
/printenv (Status: 301)
/server-info (Status: 301)
/test-cgi (Status: 301)
/server-status (Status: 301)
/tmp (Status: 301)
/~ftp (Status: 301)
/status (Status: 301)
/~bin (Status: 301)
/~root (Status: 301)
/~nobody (Status: 301)
=====================================================
Gobuster v1.1 OJ Reeves (@TheColonial)
=====================================================
[+] Mode : dir
[+] Url/Domain : http://ring0.lol/
[+] Threads : 10
[+] Wordlist : /home/justin/opt/SecLists/Discovery/Web_Content/apache.txt
[+] Status codes : 302,307,200,204,301
=====================================================
/dummy (Status: 301)
/cgi-bin (Status: 301)
/access_log (Status: 301)
/cgi-script (Status: 301)
/.htaccess (Status: 301)
/.web (Status: 301)
/cgi-pub (Status: 301)
/.meta (Status: 301)
/.htpasswd (Status: 301)
/cgi (Status: 301)
/error (Status: 301)
/error_log (Status: 301)
/httpd (Status: 301)
/htdocs (Status: 301)
/icons (Status: 301)
/manual (Status: 301)
/logs (Status: 301)
/index.html (Status: 301)
/httpd.pid (Status: 301)
/phf (Status: 301)
/printenv (Status: 301)
/server-info (Status: 301)
/server-status (Status: 301)
/~nobody (Status: 301)
/status (Status: 301)
/~bin (Status: 301)
/~root (Status: 301)
/test-cgi (Status: 301)
/~ftp (Status: 301)
/tmp (Status: 301)
=====================================================
from gobuster.
Related Issues (20)
- Error: unknown flag: --wildcard HOT 2
- Show the path when timeout HOT 2
- MTLS not working HOT 6
- vhost mode append domain! HOT 1
- vhost not working correctly HOT 1
- invalid control character in URL HOT 1
- Is it possible to take input URLs via stdin?
- [dns] wordlist line count is always higher by one HOT 7
- Bug Report/Feature Request for vhost/fuzz mode HOT 1
- [fuzz] [fe80::1%wlp2s0]:53: server misbehaving HOT 1
- [New functionnality] - Insensitive option HOT 1
- Is it possible to scan domain like https://pubaccount10086.blob.core.windows.net? HOT 1
- Not able to perform directory enumerations HOT 5
- Recursive directory bruteforcing HOT 1
- Add `--exclude-dir` on dir mode HOT 1
- Pattern flag adds to the number of requests
- Feature Request: Ability to input multiple wordlist files HOT 1
- gobuster -u http://fakebank.com -w wordlist.txt dir HOT 3
- Vhost not working as expected HOT 1
- Status 404? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gobuster.