I'm seeing this behave strangely in different contexts, and so I might just remove it until I figure out what is going on. OSX seems to be the strangest.

So, I looked to see if this was already addressed but didn't seem to be. Hope this was not an oversight on my end...

I am currently writing a script that, based off of nmap output will send the standard 80/443 port designation as well as any proxied http/https designations (i.e. 8080 or 8443) to gobuster. However, even when using gobuster -u syntax I receive this error:

Gobuster v1.2                OJ Reeves (@TheColonial)
=====================================================
[-] Unable to connect: http://10.10.10.10:443/

I was curious if there was a way to force gobuster to take the http:// or https:// off and still have it query the URI by just navigating 10.10.10.10:443/dirtest/ or 10.10.10.10:10000/dirtest/, etc....

I realize this might cause issues with certain web technologies, where it requires the http:// or https:// designation - so I'm starting to wonder if maybe it's my script that's causing this...

Currently, if a redirect loop is hit while using -r then gobuster spins a few times, then returns the following (at least if it's on the initial/base URL):

[-] Unable to connect: https://example.com/somepath/

Looking at my logs, the server is redirecting like so:

301 https://example.com/somepath/
301 https://example.com/somepath
301 https://example.com/somepath/
..etc..

Obviously, if we don't follow redirects with -r this issue won't happen.

This would be interesting to detect on, because in this instance, a non-existent folder will redirect to a 404, whereas an existing folder will enter this redirect loop.

Any plans to include the option to save output?

Add an option that will change the display to include the full length of the returned result (in bytes). This could be done via the Content-Length header or via parsing the page directly.

Hi, thanks for your work.!

It would be really nice to have an option for recursive scan when looking for folders so I don't have to do it manually.

So when you have something like :
http://foo/foophones/category
http://foo/foophones/images

You could also find out the following with the new function
http://foo/foophones/images/products
http://foo/foophones/images/avatars

Thanks.

Was just wondering, if I knocked together a quick little Dockerfile and maybe a run script or two, would you be open to merging it?

Originally I tried pushing it through tor, it stopped with no error message after ~1200 requests.
I tried against my own server not over tor, and it stopped at ~1100.

The wordlist I'm using had ~382,000 items in it, so I'm not sure what's going on..
is there a preferred troubleshooting step you'd like me to follow here?

I was thinking how neat it would be if you could have a third brute mode (mixed?) that combined the dns and directory brute. Just a thought but something like you pass a main top level domain, dns wordlist, and a web dir/file list. Then as gobuster discovers subdomains it kicks off a web dir brute, maybe after a quick check for port 80/443. I could have a go at working up a pull request for this if you like...

This is getting into a bit of an edge case territory, so it may not be best solved here.. but I was using one of the SecLists dictionaries and it prefixed the paths with /, which resulted in a // being generated by gobuster, which resulted in redirect behaviour on the website I was testing.

There is currently functionality to append a forward slash, I would see this as being the anti-case for that option.

-f	Append a forward-slash to each directory request (dir mode only)

It could probably be made more generic by allowing to append/chomp an arbitrary string (that defaults to /)

This may be easier (or at least nicer) to integrate into the CLI with #59

I would then see it being something similar to:

--append-slash
--append
--append /

--chomp-slash
--chomp
--chomp /

Edit: Re-reading this.. the current functionality is 'append', whereas I was thinking of 'prepend'. Maybe the current functionality of adding a slash could be controlled with a --prefix or --prepend and the current -f could be controlled with a --suffix or --append type flag.

Please, this program is awesome, but it miss a completion percentage! Please <3

When running the tool in "dir" mode with "-r" option to follow redirects, status codes of discovered pages are always logged as "OK" (200).

Example:

1. Initial request results in redirect (302)
2. The target of redirection is "Not Found" (404)
   3, The target is logged by gobuster as "OK" (200)

Hey there!
@OJ what you think about splitting gobuster's single main.go file to subpackages? Won't it make project maintaining, mean adding new features, writing tests and merging things easier? Is there any reason to keep gobuster as one big file?
Thank you for an amazing tool, cheers.

Hello!
I find this tool pretty good, so well done!
There are two features that if you could add, it would help a lot.

• Having a progress bar/percentage status would be good. For very big wordlists, the user doesnt have a way to know how far the scanning is from the end.
• When using verbose mode, the tool should save in a dictionary for example, all requests with relevant status codes, and print them in the end. So one doesnt have to scroll over the screen to find relevant status.

Also, tell me what you think of these.
Thank you.

Hi,
First of all congratulations for building such an awesome tool. I mean it's damn fast and does the work nicely.
I though it would be nice for gobuster to have a JSON output flag which would allow us to output the results in JSON format. This would allow for integration of gobuster in automated scripts.
The current output format is not well enough for integration in automated scripts.

Thanks

I noticed that you were using a boolean for the termination in state:

https://github.com/OJ/gobuster/blob/1.4-dev/libgobuster/state.go#L69

Have you considered using contexts instead?

https://golang.org/pkg/context/

An example is in the implementation I did based on your library.

The thread which receives the "Done" message:
https://github.com/kkirsche/gbust/blob/master/libgbust/worker.go#L18

The main context creation. The cancel is a context.CancelFunc which allows you to tell everything using it to "finish" what it's doing:
https://github.com/kkirsche/gbust/blob/master/libgbust/attacker.go#L74

When running: sudo go get && go build from /opt/gobuster/ I get the following

go get: no install location for directory /opt/gobuster outside GOPATH
For more details see: 'go help gopath'

I usually run gobuster on a set of urls, but gobuster does not natively support this. My solution is to script a loop around it:

cat list-of-targets.txt | while read line; do gobuster .... -u "$line" ; done

That's a bit ugly imho.

Passing the file directly to gobuster seems a much more efficient and clean way to do it. Maybe something like nmap's -iL list-of-targets.txt?

I'll have a look at the code and try to implement something.

Lots of people want to know why gobuster doesn't support recursion. I need to add my rationale to the README so that I don't have to type it out every time someone asks.

Add in logic to send a couple random subdomain resolution requests and if they are the same IP ignore responses with that IP for future requests. I think fierce does something similar...

It'd be handy to add a "grep" like feature that let's the users filter the results based on pages that have certain content. For example, if you're interested in pages that return a 200 status code that also have the word login in them.

What is the plausibility of you adding a status hotkey to return where gobuster is currently at in the wordlist and also a pause/resume? Or a switch to start at a specific position in a wordlist?

When doing DNS brute forcing we don't check to see if the base domain exists first. This means that if someone fat-fingers the base domain, they end up waiting forever only to find nothing will be found.

Instead, an initial request should be made to make sure that the base domain exists, and bail out if it doesn't (much like what we do with URLs).

When webserver block gobuster from brute forcing files, gobuster doesn't have the proper I/O exception handling and will just hang forever.

even if you ctr+c to interrupt it won't respond

Hello! Great tool!

I've come across a couple of situations where the stdin detection breaks.

The first situation is: gobuster works using < filename style IO redirection:

▶ gobuster -u "https://www.google.com/" -q < wordlist.txt
/robots.txt (Status: 200)

But not if the input comes from a pipe:

▶ cat wordlist.txt | gobuster -u "https://www.google.com/" -q
1 error occurred:

* [!] WordList (-w): Must be specified

The second situation is kind of the reverse. When in a bash while loop fed using IO redirection, and a wordlist is specified with -w, gobuster erroneously attempts to read the wordlist from the file being fed to the while loop and fails:

▶ while read url; do gobuster -u "$url" -q -w wordlist.txt; done < urls.txt
1 error occurred:

* [!] Wordlist (-w) specified with pipe from stdin. Can't have both!

This doesn't happen when the while loop is being fed with a pipe (because the size of the stdin device is zero, which is checked for here):

▶ cat urls.txt | while read url; do gobuster -u "$url" -q -w wordlist.txt; done
/robots.txt (Status: 200)

I think this is a difficult problem to tackle because of the various different permutations in which the tool can be run. One option is to avoid doing stdin detection at all, and allow people to specify -w - to force reading the wordlist from stdin; it's not something that is without precedent, but it does add a little something extra for the user.

Another option is to always use a wordlist if one is specified with -w, and only attempt to read from stdin if none is specified. This option has the downside that the command will hang if the user forgets the -w option but provides no words on stdin.

Is it compiled or what?

Its likely with how you're parsing what's legitimately a real page, what is a 404 and what happens when a site confuses the two.

www.whoismrrobot.com, for example, doesn't have a 404 - it redirects anything that would be a 404 back to the main page, so you get this as a result:

=====================================================
Gobuster v0.8 (DIR support by OJ Reeves @TheColonial)
              (DNS support by Peleus     @0x42424242)
=====================================================
[+] Mode         : dir
[+] Url/Domain   : http://www.whoismrrobot.com/
[+] Threads      : 10
[+] Wordlist     : list.txt
[+] Status codes : 200,204,301,302,307
=====================================================
/A (200)
/about-us (200)
/aboutus (200)
/acceso (200)
/academics (200)
/abstract (200)
/aboutUs (200)
/about_us (200)
/AboutUs (200)
/About (200)
/accesswatch (200)
/accessories (200)
/action (200)
/accounts (200)
/acciones (200)
/activities (200)
/active (200)
/accessibility (200)
/ad (200)
/adclick (200)
/adlog (200)
/admcgi (200)
/admin-bak (200)
/adm (200)
/admin-old (200)
/admin-console (200)
/admin.back (200)
/add (200)```

Hi,

it would be great to have a command switch to specify which interface gobuster uses for scanning.

At the moment, gobuster indicates that it can't connect to sites if they don't have valid certs. This needs to be sorted.

Hi,

Just had a try of your toy but first compile action gave me an error.

"./main.go:286: undefined: bufio.NewScanner"

Platform: Kali 64 bit.
go compiler installed by using "apt-get install golang"

Any ideas of how that can be resolved?

The NewScanner directive is located only with that line, 286, so I suppose you would know better what's going on at that point.

Thanks.

root@karma:~/pt/gobuster# go run main.go 
main.go:23:2: cannot find package "github.com/OJ/gobuster/libgobuster" in any of:
	/usr/lib/go-1.9/src/github.com/OJ/gobuster/libgobuster (from $GOROOT)
	/root/go/src/github.com/OJ/gobuster/libgobuster (from $GOPATH)

Tried reinstalling GO (sudo apt-purge remove) but can't make it work please help

It would be nice to have the (soon to be official) supported package manager integrated.

• https://github.com/golang/dep

Definitely makes things easier when working on new features, and gives better support around managing dependency versions/branches/etc as required.

dep init
dep ensure

While building:

./main.go:535: multiple-value uuid.NewV4() in single-value context
./main.go:560: multiple-value uuid.NewV4() in single-value context

Code refs:

• https://github.com/OJ/gobuster/blob/master/main.go#L535
• https://github.com/OJ/gobuster/blob/master/main.go#L560

Library causing the issue:

• https://github.com/satori/go.uuid

Looks like the usage has changed to supply an error value in the return as well, eg:

u, err := uuid.NewV4()

This seems to be the change that caused it:

• satori/go.uuid@0ef6afb

If the dependencies were pinned to known working versions (#61), errors like this shouldn't come up.

Currently HTTP Response size of Redirects is not displayed, even with the -l flag in Directory Mode. I'm not sure how common it is in the real world, but I've come across several CTF Machines that unintentionally expose PHP Scripts by forgetting to call exit() after HTTP Redirect.

This is the only reason I'd prefer to use DirBuster over the command line alternatives (GoBuster/Dirb). I'd love to recommend GoBuster as its CLI Driven, Faster, and more reliable however I really dislike the lack of response since on the 301/302 requests.

root in github.com/OJ/gobuster on  master [!] via 🐹 v1.6
•% ➜ ls
libgobuster  LICENSE  main.go  README.md  THANKS

root in github.com/OJ/gobuster on  master [!] via 🐹 v1.6
•% ➜ go get && go build
# github.com/OJ/gobuster/libgobuster
libgobuster/dir.go:108: undefined: uuid.Must
libgobuster/dns.go:13: undefined: uuid.Must

root in github.com/OJ/gobuster on  master [!] via 🐹 v1.6
•% ➜ go install
# github.com/OJ/gobuster/libgobuster
libgobuster/dir.go:108: undefined: uuid.Must
libgobuster/dns.go:13: undefined: uuid.Must

root in github.com/OJ/gobuster on  master [!] via 🐹 v1.6
•% ➜ echo $GOPATH
/root/go

root in github.com/OJ/gobuster on  master [!] via 🐹 v1.6
•% ➜ pwd
/root/go/src/github.com/OJ/gobuster

root in github.com/OJ/gobuster on  master [!] via 🐹 v1.6
•% ➜ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/go"
GORACE=""
GOROOT="/usr/lib/go-1.6"
GOTOOLDIR="/usr/lib/go-1.6/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

I think all the settings are in place but why this won't work, any help ?

Instead of cluttering up #41 with unrelated stuff.. was just wondering, what you're hoping/wanting to land for 1.4?

All of the current stuff in #41 / #51 / #54 seem to be landing in the 1.4-dev branch currently (mostly tagging to track everything back to this issue)

When I request a resource using curl the server responds with a 302 redirection. Requesting the same resource with gobuster gives a 200 response.

###################

With the new DNS feature we currently blindly brute force the DNS names in the word list to see what resolves. This is a pointless exercise when:

• Wildcard records are in use.
• Zone transfers are possible.

We should cater for both of these prior to running the brute force, and exit early if needs be. This is effectively what fierce does.

While it might add a slight bit more size to the binary, I think the flexibility/ease of implementation we can get from using a more full featured CLI would be pretty sweet.

https://github.com/spf13/cobra is used by a huge number of go projects, and is really nice to work with from my little experience.

I think this could work into the stuff already landing (referenced from #55) with the refactor, and make a nice little rework for 1.4.

Would you be open to it @OJ ?

Hello is there a way to add more than one dictionary? I tried -w dic.txt,dic2.txt also with -w dic.txt -w dict2.txt with no luck
Also recursive in most cases sucks.. but is not to have the option in case is needed...

Currently when a wildcard response is found it looks something like this:

[-] Wildcard response found: https://example.com/47ebafdb-032d-4a47-bf41-26b1a4314326 => 302

It would be useful if the destination URL was shown as well, maybe something like:

[-] Wildcard response found: https://example.com/47ebafdb-032d-4a47-bf41-26b1a4314326 => 302 => https://example.com/foo

This comes from:

Currently GoGet appears to just be returning the status code and content size, so changes would need to be made there, and to MakeRequest

Hey @OJ,

Working through some automated OSINT parsing and noticed one drawback to gobuster being a lack of IP resolution for the discovered sub domains.

Super obvious example:
Found: mail.domain.com - 42.1.1.2

Cheers

VERY fantastical.

Is it possible to have a list of target addresses (such as nmap's -iL)

I'll buy you steak next time your in London -_-

Don't kill me...

would there be any value in adding a canary?
random string or user supplied...

If the target resolves every name provided in the list those that have the same IP as the canary domain automatically get stripped out , suppressed,hidden whatnot

I haven't confirmed this, but the feel I got from looking at the code is that currently gobuster downloads the entire page for each checked URL.

If that is the case, there are a few methods I was thinking of that may help to speed this up:

• Using the range header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
• Supporting HEAD/OPTIONS requests (#44)

On the topic of using the range header, it could be that some tests are done at the beginning to see if it is supported (similar to the 'wildcard response found' check) to ensure the server supports ranges.

If there is support for it, gobuster could send the range header to limit the request to a reasonably small size (maybe customizable from flags with a sensible default)

If it's not supported by the server, maybe gobuster could forcibly close the connection after reading X bytes as a fallback.

If there isn't already a timeout set for a connection that 'never returns', that could be implemented in a similar way, again customisable.

There may be other optimisations that could be done in a similar vain to this, but for now, these are the ones coming to mind.

Hi,

I noticed that a few times webservers are configured to return 200s or 301s instead of 404s. That means that gobuster will happily "find" the whole words.txt.

Idea: Count the "valid" hits in a row and if it exceeds 5% or 10% of the whole words.txt, then abort the scan (and/or add an "force" switch).

Kind regards,
gehaxelt

http://stackoverflow.com/questions/18149601/go-install-always-fails-no-install-directory-outside-gopath
mkdir bin
export GOBIN=$GOPATH/bin

Got this issue a few times today. On OSX with 16GB RAM. Calling gobuster from 10 python processes (also tried with 10 threads) with -t 5 and a 10 word list. Is it possible that there's a memory leak? My code is using only subprocess module functions that wait for the program to end before continuing.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1210e58]

goroutine 1 [running]:
main.SetupDir(0xc4200d0140, 0x1010452)
	/Users/me/.go/src/gobuster-master/main.go:563 +0x138
main.Process(0xc4200d0140)
	/Users/me/.go/src/gobuster-master/main.go:433 +0x79
main.main()
	/Users/me/.go/src/gobuster-master/main.go:824 +0x3e

Update: Memory usage shows no signs of climbing out of control before this error happens
5 minutes after last update Update: I think it's just that MakeRequest is returning nil because http.NewRequest is failing.