Git Product home page Git Product logo

b-fullam / automating-virustotal-apiv3-for-ips-and-urls Goto Github PK

View Code? Open in Web Editor NEW
39.0 3.0 12.0 33 KB

Automating VirusTotal's API v3 for IP address and URL analysis w/HTML Reporting. Python script that functions like a CLI tool to interact programmatically with VirusTotal API v3.

License: MIT License

Python 100.00%
security-tools python virustotal virustotal-api security-automation bulk-operation bulk command-line-tool python3 information-security

automating-virustotal-apiv3-for-ips-and-urls's Introduction

Automating-VirusTotal-APIv3-for-IPs-and-URLs

Automating VirusTotal's API(v3) for IP address and URL analysis with HTML Reporting

Enter a single IP address or URL, or select either a list of IP addresses or URLs to be submitted to VirusTotal's API(v3) for analysis.

Here are the options included in the script:

usage: vt-ip-url-analysis.py [-h] [-s SINGLE_ENTRY] [-i IP_LIST] [-u URL_LIST] [-V]

Python Automated VT API v3 IP address and URL analysis 2.0 by Brett Fullam

optional arguments:
  -h, --help                show this help message and exit
  -s SINGLE_ENTRY, --single-entry SINGLE_ENTRY
                            ip or url for analysis
  -i IP_LIST, --ip-list IP_LIST
                            bulk ip address analysis
  -u URL_LIST, --url-list URL_LIST
                            bulk url analysis
  -V, --version             show program version

Here's a sample of the command to run the script using a single entry, google.com, as the input:

python3 vt-ip-url-analysis.py -s google.com

The results returned by VT's API(v3) are then filtered for high-level analysis to quickly determine whether entries are harmless or need further investigation.

Security Automation Python DNS Lookups

The script also generates a hypertext link to VirusTotal's web-based GUI for each entry allowing the analyst seamless access to additional information directly from the HTML report.

The generated HTML report, named "report.html", is saved in the same directory that the Python script resides.

For more information about this script, check out my article on "Security Automation with Python — IP address and URL analysis via VirusTotal's API v3 with HTML Reporting."

Getting Started

  1. Download the script or use git to clone the repository

  2. Install dependencies. This script was created using Python3, and I've included a requirements.txt file listing the necessary dependencies. More information on how to install dependencies for this project can be found in my article "Security Automation with Python — IP address and URL analysis via VirusTotal's API v3 with HTML Reporting".

  3. You'll also need to create a .env file in the same directory as the Python script. Inside the .env file add the following code and insert your VirusTotal API key as indicated:

API_KEY1=<insert your vt API key here>

ALWAYS remember to make sure you add ".env" to your .gitignore file to keep it from being sent to github and exposing your API key in the repository. For more information on working with .env files, take a look at Drew Seewald's article "Using dotenv to Hide Sensitive Information in Python--Hide your passwords and API tokens to make your code more secure".

  1. I also included 2 text files for testing both the IP address and URL lists functionality. One for URLs, target-urls.txt, which has URLs with intentional formatting errors to test the regex pattern included in the script. One for IP addresses, target-ips.txt, which includes a mix of public and private IP addresses, as well as a handful of improperly formatted IP addresses to test the regex patterns included in the script.

automating-virustotal-apiv3-for-ips-and-urls's People

Contributors

b-fullam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

automating-virustotal-apiv3-for-ips-and-urls's Issues

Error with python3.11

Hi, I'm trying to run the script on a WSL VM.

I have done all the steps indicated to use your script and I am getting the following errors:

─$ python vt-ip-url-analysis.py -h
/home/nueve/.local/lib/python3.11/site-packages/requests/init.py:102: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (5.2.0)/charset_normalizer (2.0.10) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "
Traceback (most recent call last):
File "init.pxd", line 942, in numpy.import_array
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/nueve/Automating-VirusTotal-APIv3-for-IPs-and-URLs/vt-ip-url-analysis.py", line 6, in
import pandas as pd
File "/usr/local/lib/python3.11/dist-packages/pandas/init.py", line 22, in
from pandas.compat import (
File "/usr/local/lib/python3.11/dist-packages/pandas/compat/init.py", line 15, in
from pandas.compat.numpy import (
File "/usr/local/lib/python3.11/dist-packages/pandas/compat/numpy/init.py", line 7, in
from pandas.util.version import Version
File "/usr/local/lib/python3.11/dist-packages/pandas/util/init.py", line 1, in
from pandas.util._decorators import ( # noqa
File "/usr/local/lib/python3.11/dist-packages/pandas/util/_decorators.py", line 14, in
from pandas._libs.properties import cache_readonly # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pandas/_libs/init.py", line 13, in
from pandas._libs.interval import Interval
File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
File "pandas/_libs/hashtable.pyx", line 1, in init pandas._libs.hashtable
File "pandas/_libs/missing.pyx", line 1, in init pandas._libs.missing
File "/usr/local/lib/python3.11/dist-packages/pandas/_libs/tslibs/init.py", line 31, in
from pandas._libs.tslibs.conversion import (
File "pandas/_libs/tslibs/conversion.pyx", line 1, in init pandas._libs.tslibs.conversion
File "pandas/_libs/tslibs/nattype.pyx", line 30, in init pandas._libs.tslibs.nattype
File "init.pxd", line 944, in numpy.import_array
ImportError: numpy.core.multiarray failed to import

Thank you for your work and help.

KeyError: 'data'

Please help me resolve the following error. I followed the tutorial and had no problems with the installation. I tried to enter an IP list from a txt file and got the error below. Please help me fix it as I am a beginner in Python coding.

Traceback (most recent call last):
  File "/home/haha/python-vt/vt-ip-url-analysis.py", line 459, in <module>
    urlReportIPLst(args.ip_list)
  File "/home/haha/python-vt/vt-ip-url-analysis.py", line 322, in urlReportIPLst
    urlReport(i)
  File "/home/haha/python-vt/vt-ip-url-analysis.py", line 104, in urlReport
    epoch_time = (decodedResponse["data"]["attributes"]["last_analysis_date"])
KeyError: 'data'

'favicon' key received inside filteredResponse made Pandas throw an AttributeError

Hi guys, I cloned the repo and install the dependencies correctly but for some reason it gave me an AttributeError: 'int' object has no attribute 'items' error while trying to create the dataframe with the remaining keys stored in the filteredResponse dictionary (pd.DataFrame.from_dict) - see urlReport(arg) function. Looking into the filteredResponse that's being passed to the DataFrame Pandas function, it seems that it didn't like the "favicon" attribute so I added it to the keys_to_remove array and now everything works fine.

The error:

└─$ python3 vt-ip-url-analysis.py -s google.com

+++++ Print filteredResponse +++++
{'favicon': {'raw_md5': '1631ff32e63a6dcc15469d14c7c94e42', 'dhash': 'e89e931939338ee8'}, 'times_submitted': 166200, 'redirection_chain': ['http://google.com/', 'http://www.google.com/'], 'reputation': 2615, 'has_content': False, 'last_analysis_stats': {'harmless': 81, 'malicious': 0, 'suspicious': 0, 'undetected': 8, 'timeout': 0}}
+++++++++++
Traceback (most recent call last):
  File "/home/kali/Desktop/Python-VT-Script/vt-ip-url-analysis.py", line 456, in <module>
    urlReport(args.single_entry)
  File "/home/kali/Desktop/Python-VT-Script/vt-ip-url-analysis.py", line 167, in urlReport
    dataframe = pd.DataFrame.from_dict(filteredResponse, orient='index')
  File "/home/kali/Desktop/Python-VT-Script/lib/python3.10/site-packages/pandas/core/frame.py", line 1747, in from_dict
    data = _from_nested_dict(data)
  File "/home/kali/Desktop/Python-VT-Script/lib/python3.10/site-packages/pandas/core/frame.py", line 12029, in _from_nested_dict
    for col, v in s.items():
AttributeError: 'int' object has no attribute 'items'

Below is the filteredResponse received for google.com:

{'favicon': {'raw_md5': '1631ff32e63a6dcc15469d14c7c94e42', 'dhash': 'e89e931939338ee8'}, 'times_submitted': 166198, 'redirection_chain': ['http://google.com/', 'http://www.google.com/'], 'reputation': 2615, 'has_content': False, 'last_analysis_stats': {'harmless': 81, 'malicious': 0, 'suspicious': 0, 'undetected': 8, 'timeout': 0}}

After adding "favicon" to the keys_to_remove attribute:

┌──(Python-VT-Script)─(kali㉿kali)-[~/Desktop/Python-VT-Script]
└─$ python3 vt-ip-url-analysis.py -s google.com

+++++Print filteredResponse +++++
{'times_submitted': 166200, 'redirection_chain': ['http://google.com/', 'http://www.google.com/'], 'reputation': 2615, 'has_content': False, 'last_analysis_stats': {'harmless': 81, 'malicious': 0, 'suspicious': 0, 'undetected': 8, 'timeout': 0}}
+++++++++++
                                                                     0
times_submitted                                                 166200
redirection_chain         [http://google.com/, http://www.google.com/]
reputation                                                        2615
has_content                                                      False
last_analysis_stats  {'harmless': 81, 'malicious': 0, 'suspicious':...
                                                            google.com
community score      0/89  :  security vendors flagged this as mali...
has_content                                                      False
last_analysis_date                            Thu Oct  6 10:32:49 2022
last_analysis_stats  {'harmless': 81, 'malicious': 0, 'suspicious':...
redirection_chain         [http://google.com/, http://www.google.com/]
reputation                                                        2615
times_submitted                                                 166200
virustotal report    https://www.virustotal.com/gui/url/cf4b367e49b...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.