plazmaz / sublist3r Goto Github PK

View Code? Open in Web Editor NEW

This project forked from aboul3la/sublist3r

116.0 4.0 23.0 1.09 MB

Fast subdomains enumeration tool for penetration testers

License: GNU General Public License v2.0

Python 99.75% Dockerfile 0.25%

sublist3r's Introduction

This is not the official repo for sublist3r, this version contains extra features (notably subdomain takeover checks).

The original version: https://github.com/aboul3la/Sublist3r

About Sublist3r

Sublist3r is python tool that is designed to enumerate subdomains of websites through OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu, and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster and ReverseDNS.

subbrute was integrated with Sublist3r to increase the possibility of finding more subdomains using bruteforce with an improved wordlist. The credit goes to TheRook who is the author of subbrute.

Screenshots

Installation

git clone https://github.com/Plazmaz/Sublist3r.git

Recommended Python Version:

Sublist3r currently supports Python 2 and Python 3.

The recommended version for Python 2 is 2.7.x
The recommened version for Python 3 is 3.4.x

Dependencies:

Sublist3r depends on the requests, dnspython and argparse python modules.

These dependencies can be installed using the requirements file:

Installation on Windows:

c:\python27\python.exe -m pip install -r requirements.txt

Installation on Linux

sudo pip install -r requirements.txt

Alternatively, each module can be installed independently as shown below.

Requests Module (http://docs.python-requests.org/en/latest/)

Install for Windows:

c:\python27\python.exe -m pip install requests

Install for Ubuntu/Debian:

sudo apt-get install python-requests

Install for Centos/Redhat:

sudo yum install python-requests

Install using pip on Linux:

sudo pip install requests

dnspython Module (http://www.dnspython.org/)

Install for Windows:

c:\python27\python.exe -m pip install dnspython

Install for Ubuntu/Debian:

sudo apt-get install python-dnspython

Install using pip:

sudo pip install dnspython

argparse Module

Install for Ubuntu/Debian:

sudo apt-get install python-argparse

Install for Centos/Redhat:

sudo yum install python-argparse

Install using pip:

sudo pip install argparse

for coloring in windows install the following libraries

c:\python27\python.exe -m pip install win_unicode_console colorama

Usage

Short Form	Long Form	Description
-d	--domain	Domain name to enumerate subdomains of
-b	--bruteforce	Enable the subbrute bruteforce module
-p	--ports	Scan the found subdomains against specific tcp ports
-v	--verbose	Enable the verbose mode and display results in realtime
-t	--threads	Number of threads to use for subbrute bruteforce
-e	--engines	Specify a comma-separated list of search engines
-o	--output	Save the results to text file
-h	--help	show the help message and exit
-to	--takover-check	[New] Scan subdomains for unregistered services such as Github Pages

Examples

To list all the basic options and switches use -h switch:

python sublist3r.py -h

To enumerate subdomains of specific domain:

python sublist3r.py -d example.com

To enumerate subdomains of specific domain and show only subdomains which have open ports 80 and 443 :

python sublist3r.py -d example.com -p 80,443

To enumerate subdomains of specific domain and show the results in realtime:

python sublist3r.py -v -d example.com

To enumerate subdomains and enable the bruteforce module:

python sublist3r.py -b -d example.com

To enumerate subdomains and use specific engines such Google, Yahoo and Virustotal engines

python sublist3r.py -e google,yahoo,virustotal -d example.com

Using Sublist3r as a module in your python scripts

Example

import sublist3r 
subdomains = sublist3r.main(domain, no_threads, savefile, ports, silent, verbose, enable_bruteforce, takeover_check, engines)

The main function will return a set of unique subdomains found by Sublist3r

Function Usage:

domain: The domain you want to enumerate subdomains of.
savefile: save the output into text file.
ports: specify a comma-sperated list of the tcp ports to scan.
silent: set sublist3r to work in silent mode during the execution (helpful when you don't need a lot of noise).
verbose: display the found subdomains in real time.
enable_bruteforce: enable the bruteforce module.
takeover_check: enable subdomain takeover checks.
engines: (Optional) to choose specific engines.

Example to enumerate subdomains of Yahoo.com:

import sublist3r 
subdomains = sublist3r.main('yahoo.com', 40, 'yahoo_subdomains.txt', ports= None, silent=False, verbose= False, enable_bruteforce= False, takeover_check=False, engines=None)

License

Sublist3r is licensed under the GNU GPL license. take a look at the LICENSE for more information.

Credits

TheRook - The bruteforce module was based on his script subbrute.
Bitquark - The Subbrute's wordlist was based on his research dnspop.

Thanks

Special Thanks to Ibrahim Mosaad for his great contributions that helped in improving the tool.

Version

Current version is 1.0

sublist3r's People

Contributors

Stargazers

Watchers

sublist3r's Issues

cannot start a process twice

Simply running py sublist3r.py -v -d exmaple.com gives this error:

Traceback (most recent call last):
  File "sublist3r.py", line 78, in <module>
    takeover_check=takeover_check, engines=engines)
  File "sublist3r.py", line 59, in main
    return scanner.scan()
  File "C:\Users\Ben\Desktop\Sublist3r-master\Sublist3r-master\subscann3r.py", line 88, in scan
    enum.start()
  File "C:\Users\Ben\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\process.py", line 99, in start
    assert self._popen is None, 'cannot start a process twice'
AttributeError: 'BaiduEnum' object has no attribute '_popen'

This is on Windows 10, Python 3.6.1

unbound method init()

May i know what am doing wrong?

C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r>c:\python27\python.exe -m pip install -r requirements.txt
Requirement already satisfied: argparse in c:\python27\lib\site-packages (from -r requirements.txt (line 1))
Requirement already satisfied: dnspython in c:\python27\lib\site-packages (from -r requirements.txt (line 2))
Requirement already satisfied: requests in c:\python27\lib\site-packages (from -r requirements.txt (line 3))

C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r>sublist3r.py -d glx.com

                 ____        _     _ _     _   _____
                / ___| _   _| |__ | (_)___| |_|___ / _ __
                \___ \| | | | '_ \| | / __| __| |_ \| '__|
                 ___) | |_| | |_) | | \__ \ |_ ___) | |
                |____/ \__,_|_.__/|_|_|___/\__|____/|_|

                # Coded By Ahmed Aboul-Ela - @aboul3la
                # Rewritten by Dylan Katz  - @Plazmaz

[-] Enumerating subdomains now for glx.com
[-] Searching now in Baidu..
Traceback (most recent call last):
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\sublist3r.py", line 78, in
takeover_check=takeover_check, engines=engines)
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\sublist3r.py", line 59, in main
return scanner.scan()
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\subscann3r.py", line 86, in scan
enums = [enum(self.domain, [], q=subdomains_queue, silent=self.scan_flags.Silent, logger=self.logger) for enum in chosenEnums]
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\engines\engine.py", line 243, in init
logger=logger)
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\engines\engine.py", line 32, in init
multiprocessing.Process.init(self)
TypeError: unbound method init() must be called with Thread instance as first argument (got BaiduEnum instance instead)

C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r>python sublist3r.py -d glx.com

                 ____        _     _ _     _   _____
                / ___| _   _| |__ | (_)___| |_|___ / _ __
                \___ \| | | | '_ \| | / __| __| |_ \| '__|
                 ___) | |_| | |_) | | \__ \ |_ ___) | |
                |____/ \__,_|_.__/|_|_|___/\__|____/|_|

                # Coded By Ahmed Aboul-Ela - @aboul3la
                # Rewritten by Dylan Katz  - @Plazmaz

[-] Enumerating subdomains now for glx.com
[-] Searching now in Baidu..
Traceback (most recent call last):
File "sublist3r.py", line 78, in
takeover_check=takeover_check, engines=engines)
File "sublist3r.py", line 59, in main
return scanner.scan()
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\subscann3r.py", line 86, in scan
enums = [enum(self.domain, [], q=subdomains_queue, silent=self.scan_flags.Silent, logger=self.logger) for enum in chosenEnums]
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\engines\engine.py", line 243, in init
logger=logger)
File "C:\Users\kandasam\Dropbox\bug\Tools\Sublist3r\engines\engine.py", line 32, in init
multiprocessing.Process.init(self)
TypeError: unbound method init() must be called with Thread instance as first argument (got BaiduEnum instance instead)

Error

Getting the following on the Baidu Process:

Process BaiduEnum-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/root/tools/Sublist3r/engines/engine.py", line 39, in run
    domain_list = self.enumerate()
  File "/root/tools/Sublist3r/engines/enumarator_base.py", line 118, in enumerate
    links = self.extract_domains(resp)
  File "/root/tools/Sublist3r/engines/engine.py", line 270, in extract_domains
    return links
UnboundLocalError: local variable 'links' referenced before assignment

Any ideas?

Error when I run Sublist3r.py -d domain.com

Hello,

I have Win 7 with all updates. I also have Python 2.7 + 3.6. When I run sublist3r.py -d domain.com, I get the following error:

Traceback (most recent call last):
File "C:\Users\xxx\Documents\Sublist3r2\sublist3r.py", line 78, in
takeover_check=takeover_check, engines=engines)
File "C:\Users\xxx\Documents\Sublist3r2\sublist3r.py", line 59, in main
return scanner.scan()
File "C:\Users\xxx\Documents\Sublist3r2\subscann3r.py", line 88, in scan
enum.start()
File "C:\Users\xxx\AppData\Local\Programs\Python\Python36-32\lib\multiproces
sing\process.py", line 99, in start
assert self._popen is None, 'cannot start a process twice'
AttributeError: 'BaiduEnum' object has no attribute '_popen'

Your help is appreciated.
Thanks
Ben

ip option -i

Hey @Plazmaz can you grab this feature here and merge? it's super useful for large scoped projects:

https://github.com/aboul3la/Sublist3r/pull/107

Error when running the -p option

hey again @Plazmaz , any ideas on this one?

python sublist3r.py -d TARGET.com -p 80

[-] Starting port scan for the following ports: 80
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/root/tools/Sublist3r/sublister2/Sublist3r/util/port_scanner.py", line 27, in port_scan
    print("%s%s%s - %sFound open ports:%s %s%s%s" % (G, host, W, R, W, Y, ', '.join(openports), W))
NameError: global name 'G' is not defined

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/root/tools/Sublist3r/sublister2/Sublist3r/util/port_scanner.py", line 27, in port_scan
    print("%s%s%s - %sFound open ports:%s %s%s%s" % (G, host, W, R, W, Y, ', '.join(openports), W))
NameError: global name 'G' is not defined

error

c:\python27>python.exe sub2/sub.py -d site.com
[!] Error: Coloring libraries not installed ,no coloring will be used [Check the
readme]
←[91m
____ _ _ _ _ _____
/ | _ | | | ()| ||__ / _ __
___ | | | | '_ | | / | | | | '|
) | || | |) | | _ \ | ) | |
|/ _,|./|||/_|___/||←[0m←[93m

                # Coded By Ahmed Aboul-Ela - @aboul3la

←[94m[-] Enumerating subdomains now for site.com←[0m
←[92m[-] Searching now in Baidu..←[0m
Traceback (most recent call last):
File "sub2/sub.py", line 82, in
takeover_check=takeover_check, engines=engines)
File "sub2/sub.py", line 63, in main
return scanner.scan()
File "c:\python27\sub2\subscann3r.py", line 86, in scan
enums = [enum(self.domain, [], q=subdomains_queue, silent=self.scan_flags.Si
lent, logger=self.logger) for enum in chosenEnums]
File "c:\python27\sub2\engines\engine.py", line 244, in init
logger=logger)
File "c:\python27\sub2\engines\engine.py", line 32, in init
multiprocessing.Process.init(self)
TypeError: unbound method init() must be called with Thread instance as firs
t argument (got BaiduEnum instance instead)

AssertionError: can only join a started process

./sublist3r.py -v -b -p 80,443 -d hackerone.com

[-] Enumerating subdomains now for hackerone.com
[-] verbosity is enabled, will show the subdomains results in realtime
[-] Searching now in Baidu..
[-] Searching now in Google..
[-] Searching now in Virustotal..
[-] Searching now in GoogleTER..
[-] Searching now in Yahoo..
[-] Searching now in Netcraft..
[-] Searching now in SSL Certificates..
[-] Searching now in DnsDB..
[-] Searching now in Ask..
[-] Searching now in ThreatCrowd..
[-] Searching now in DNSdumpster..
[-] Searching now in Bing..
[-] Searching now in HackerTarget..
[-] Searching now in PassiveDNS..
Baidu: support.hackerone.com
Virustotal: a.ns.hackerone.com
Virustotal: b.ns.hackerone.com
Virustotal: api.hackerone.com
Virustotal: links.hackerone.com
Virustotal: support.hackerone.com
Virustotal: info.hackerone.com
Virustotal: www.hackerone.com
Yahoo: www.hackerone.com
Yahoo: support.hackerone.com
Netcraft: www.hackerone.com
SSL Certificates: support.hackerone.com
SSL Certificates: www.hackerone.com
SSL Certificates: links.hackerone.com
SSL Certificates: api.hackerone.com
SSL Certificates: go.hackerone.com
SSL Certificates: info.hackerone.com
ThreatCrowd: api.hackerone.com
ThreatCrowd: www.hackerone.com
DNSdumpster: support.hackerone.com
DNSdumpster: b.ns.hackerone.com
DNSdumpster: a.ns.hackerone.com
DNSdumpster: www.hackerone.com
Bing: support.hackerone.com
Traceback (most recent call last):
  File "./sublist3r.py", line 78, in <module>
    takeover_check=takeover_check, engines=engines)
  File "./sublist3r.py", line 59, in main
    return scanner.scan()
  File "/root/sublister-se/Sublist3r/subscann3r.py", line 90, in scan
    enum.join()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 144, in jo
    assert self._popen is not None, 'can only join a started process'