chrismaddalena / odin Goto Github PK

View Code? Open in Web Editor NEW

622.0 34.0 115.0 2.7 MB

Automated network asset, email, and social media profile discovery and cataloguing.

Home Page: https://posts.specterops.io/gathering-open-source-intelligence-bee58de48e05

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

osint recon reconnaissance asset-discovery neo4j

odin's Introduction

ODIN

Observe, Detect, and Investigate Networks

Current version: v2.0.0 "Huginn"

ODIN is Python tool for automating intelligence gathering, asset discovery, and reporting. Remember, check the dev branch for the bleeding edge, and feedback is welcome!

See the GitHub wiki for details and installation and setup instructions.

What Can ODIN Do?

ODIN aims to automate the basic recon tasks used by red teams to discover and collect data on network assets, including domains, IP addresses, and internet-facing systems. The key feature of ODIN is the data management and reporting. The data is organized in a database and then, optionally, that database can be converted into an HTML report or a Neo4j graph database for visualizing the data.

ODIN performs this in multiple phases:

Phase 1 - Asset Discovery

Collect basic organization information from sources like the Full Contact marketing database.
Check DNS Dumpster, Netcraft, and TLS certificates to discover subdomains for the provided domains.
Resolve domain and subdomains to IP addresses via socket connections and DNS records.
Collect information for all IP addresses, such as ownership and organization data, from RDAP, whois, and other data sources.
Lookup domains and search for IP addresses on Shodan to collect additional data, such as operating systems, service banners, and open ports.
Check for the possibility of takeovers and domain fronting with the domains and subdomains.

Phase 2 - Employee Discovery

Harvest email addresses and employee names for the target organization.
Link employees to social media profiles via search engines and the Twitter API.
Cross check discovered email addresses with Troy Hunt's Have I Been Pwned.

Phase 3 - Cloud and Web Services

Hunt for Office files and PDFs under the target domain, download them, and extract metadata.
Search for AWS S3 buckets and Digital Ocean Spaces using keywords related to the organization.
Take screenshots of discovered web services for a quick, early review of services.

Phase 4 - Reporting

Save all data to a SQLite3 database to allow the data to be easily queried.
Generate an HTML report using default SQL queries to make it simple to peruse the data in a web browser.
Create a Neo4j graph database that ties all of the discovered entities (IP addresses, domains, subdomains, ports, and certificates) together with relationships (e.g. RESOLVES_TO, HAS_PORT).

At the end of all of this you will have multiple ways to browse and visualize the data. Even a simple Neo4j query like MATCH (n) RETURN n (display everything) can create a fascinating graph of the organization's external perimeter and make it simple to see how assets are linked. The Neo4j wiki pages contain better query examples.

odin's People

Contributors

Stargazers

Watchers

Forkers

silogit h0k5 securycore clicknull johnjohnsp1 izogain cacalote nesgley minkione ayooshagarwal boris519 terncgod techlord-rce samyoyo sonahri rajivraj tim1512 ksmaheshkumar star-bob mortymorty geonaute theralfbrown rasarab 5bhuv4n35h thealadeen 0xjukai r3dfruitrollup 4n6strider d3viluke jaredpack iraqnophobia js36 raystyle wsygoogol ro9ueadmin me009 m4rm0k vap0rz nickhakkz marpie slooppe anyedt bbhunter orf53975 reanimat0r eos21 9dheu0aq3 tdefise str3tched-by-design threatinteltest sjas gilfidel oneplush odomrob 0xneox umbrell4 superuser5 github-userx skinny21 mjmaley huseyinrencber mmg1 hernannh quagmire69 databill86 backwardn seabreg atropineal qman69 ntgitdude23 hpycuit mdheller 5l1v3r1 ykankaya masterscott xiaolushuo dhinmn swastikdey175 bcp-infosec-repo pingmuict larl0ch jorson-chen an0nym0u5101 refkywijaya pseudobobsmith kamikazerascal 0xpwn7 bigrayhicks neight-1 misterola hiteshmah-jan thehappydinoa ninja-pandit lkthinh projects-by-abhi paul-q-burns israelccarvalho longjoe76 terrorizer1980 sboriskirov

odin's Issues

Trojan

Is this a Trojan? lol

FEATURE REQ: AWS wordlist option doesn't prepend or append the client name

I noticed if you pass ODIN a list of AWS prefixes to search for, it will look for just those prefixes and not append or prepend the client name when it looks for a S3 bucket.

If /tmp/prefixes contains:
chat
sitemap

And you pass it with the -w parameter like # odin.py osint -c 'CLIENT' -d 'client.com' -w /tmp/prefixes

ODIN will return:

http://chat.s3.amazonaws.com
http://sitemap.s3.amazonaws.com

Possible solution:

Search for bucket names like...

chat-client
client-chat
chat.client
client.chat
chat_client
client_chat

etc.

Dependency problems

Hi, thanks for publishing this and I'm very excited to get it working.
I'm using Kali linux and followed the steps but when i get to pipenv install i get following error. I tested to run "pipenv --python 3.6 install" also since it looks like its using python 2.7.

pipenv install
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
env/utils.py", line 402, in resolve_deps
    req_dir=req_dir
  File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 250, in actually_resolve_deps
    req = Requirement.from_line(dep)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 704, in from_line
    line, extras = _strip_extras(line)
TypeError: 'module' object is not callable

/usr/local/lib/python2.7/dist-packages/pipenv/_compat.py:113: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/pipenv-SezucC-requirements'>
  warnings.warn(warn_message, ResourceWarning)

Tweepy compatibility issues with python 3.7

Aloha,

ODIN produced the following error after homebrew updated python to 3.7:

(ODIN-4yBUt_eV) ~/D/t/ODIN ❯❯❯ python3 odin.py master
Traceback (most recent call last):
File "odin.py", line 27, in
from lib import reporter, asciis, verification, htmlreporter, grapher
File "/tools/ODIN/lib/reporter.py", line 14, in
from lib import domain_tools, email_tools, pyfoca, helpers, screenshots
File "/tools/ODIN/lib/email_tools.py", line 9, in
import tweepy
File "/Users/XXXXXXXX/.virtualenvs/ODIN-4yBUt_eV/lib/python3.7/site-packages/tweepy/init.py", line 17, in
from tweepy.streaming import Stream, StreamListener
File "/Users/XXXXXXXX/.virtualenvs/ODIN-4yBUt_eV/lib/python3.7/site-packages/tweepy/streaming.py", line 358
def _start(self, async):
^
SyntaxError: invalid syntax

After some googling, I found the following issue posted on the tweepy github:
https://github.com/tweepy/tweepy/issues/1017

I attempted to reinstall tweepy via pip3 but it doesn't look like pip has updated tweepy for some time.

In order to fix the issue, I just replaced the tweepy/streaming.py file with this one:
https://github.com/tweepy/tweepy/blob/master/tweepy/streaming.py

That seemed to resolve the issue so far.

Cheers,
Crazian

Database is locked

Running on Ubuntu 18.04.1 and am frequently running into the following error while running against some targets:

Process Employee Hunter:
Traceback (most recent call last):
  File "/home/cht/.local/share/virtualenvs/ODIN-M4UjiRTK/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap
    self.run()
  File "/home/cht/.local/share/virtualenvs/ODIN-M4UjiRTK/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/cht/ODIN/lib/reporter.py", line 794, in create_people_table
    self.c.execute("INSERT INTO email_addresses VALUES (?,NULL,NULL)",(email,))
sqlite3.OperationalError: database is locked

FEATURE REQ: Search for domains by discovered employee/person name

Once a list of people associated with the target org is compiled, it would be great to go back and search domain registration info for each individual to weed out "shadow IT" assets registered by the individual but hosting organizational info/assets.

TLS Table Output Formatting

Please adjust the column widths in the TLS table output so that the "Minimum Symmetric Cipher" column heading only wraps to two lines instead of three.

Boto Exception Not Caught

When credentials for AWS are not provided, ODIN shows a traceback.

`[+] Running urlcrazy for REDACTED
Process Cloud Report:
Traceback (most recent call last):

<--SNIP-->

botocore.exceptions.NoCredentialsError: Unable to locate credentials`

There was a problem with the specified Chrome web driver in your keys.config!

I have downloaded and extracted both gecko driver and chrome driver to ODIN directory, specified the path in either for chrome or geckodriver but odin does not recognize either of them and gives me this error. I am on Kali on VM.

[WebDriver]
driver_path = /root/ODIN/chromedriver

Please help.

URLCrazy path hardcoded in domain_tools.py

The path to urlcrazy seems to be handcoded in domain_tools.py:

$ cat lib/domain_tools.py | grep cmaddy
subprocess.call("/Users/cmaddy/Lab/URLCrazy/urlcrazy")
cmd = "/Users/cmaddy/Lab/URLCrazy/urlcrazy -f csv -o '{}' {}".format(outfile, target)

Domains containing hyphens interpreted as IP range

It seems like when parsing the scope file passed with -sf, ODIN interprets a domain containing a hyphen as an IP range.

[+] blah123-online.com is a range - expanding...
[!] Parsing of scope file failed!
L.. Details: list index out of range

colors module import issue

I tried running the setup function and received an import error. Apparently I can't seem to find the colors package.

Traceback (most recent call last):
File "setup_check.py", line 8, in <module>
from colors import red, green, yellow
ImportError: cannot import name 'red' from 'colors'

I've tried tracking it down and installing with pip to no avail. I'm new to python and this is one of my first projects in it so I apologize if this is a silly error.

Hacktoberfest

Hacktoberfest
Lets join us !

'multiprocess' not found in requirements.txt

The included requirements.txt does not include 'multiprocess' and thus needs to be installed separately on Mac 10.12.6.

AttributeError: 'NoneType' object has no attribute 'split'

Aloha,

I got the following error while running ODIN:

Release v1.9.2, MUNINN
[+] OSINT Module Selected: ODIN will run all recon modules.
[] Headless Chrome browser test was successful!
[+] Beginning initial discovery phase! This could take some time...
[+] Starting new process: Company Info Collector
[+] Starting new process: Employee Hunter
[+] Starting new process: Domain and IP Address Recon
[+] Beginning the harvesting of email addresses for redacted.com...
[] Harvesting Google
Process Domain and IP Address Recon:
Traceback (most recent call last):
File "/Users/XXXXXXXXX/.virtualenvs/ODIN-4yBUt_eV/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap
self.run()
File "/Users/XXXXXXXXX/.virtualenvs/ODIN-4yBUt_eV/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/tools/ODIN/lib/reporter.py", line 316, in create_domain_report_table
if not any(x.strip(",").strip().lower() in whois_privacy for x in org_name.split(" ")):
AttributeError: 'NoneType' object has no attribute 'split'

After some googling, I found the following link that could possibly fix this issue:
https://gis.stackexchange.com/questions/185135/attributeerror-nonetype-object-has-no-attribute-split-from-arcmap-only

So this could be another python 3.7 compatibility issue or the script needs to check for a NULL attribute before splitting.

Cheers!

Metadata issues while downloading multiprocessor

error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.
pip3 install multiprocessing
Defaulting to user installation because normal site-packages is not writeable
Collecting multiprocessing
Downloading multiprocessing-2.6.2.1.tar.gz (108 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 108.0/108.0 kB 131.8 kB/s eta 0:00:00
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [7 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/tmp/pip-install-v5w213gd/multiprocessing_6569445ef88848fc8a94f123a8ab9cf8/setup.py", line 94
print 'Macros:'
^^^^^^^^^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Syntax Error

SyntaxError: keyword can't be an expression
Checked to ensure that I had all of the requirements installed by running the master branch, it worked fine.

https://github.com/chrismaddalena/viper/blob/5c3c0c916c25f3b2f929f8eafbabc67336d05a86/lib/domain_tools.py#L686