humble

HTTP Headers Analyzer

"A journey of a thousand miles begins with a single step. - Lao Tzu"

"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"

Features
Screenshots
Installation & Update
Usage
Advanced Usage
Linux: Show only the analysis summary
Windows: In spanish. Show only the analysis summary (PowerShell >= 7 required)
Linux: Show only the URL, date and analysis summary
Linux: Show only the deprecated headers/protocols and insecure values
Linux: Check for HTTP client errors (4XX)
Linux: Analyze multiple URLs and save the results as PDFs
Checks: Missing Headers
Checks: Fingerprint Headers
Checks: Deprecated Headers and Insecure Values
Checks: Empty Values
Guidelines included
To-Do
Further Reading
Contribute
Acknowledgements
License

Features

✔️ 13 checks of missing HTTP response headers.
✔️ 996 checks of fingerprinting through HTTP response headers.
✔️ 85 checks of deprecated HTTP response headers/protocols or with insecure/wrong values.
✔️ Browser compatibility check for enabled security headers.
✔️ Two types of analysis: brief and detailed, along with HTTP response headers.
✔️ Export of analysis to HTML5, PDF 1.4 and TXT.
✔️ The analysis includes dozens of references, official documentation and technical articles.
✔️ i10n: analysis results in English or Spanish.
✔️ Saves each analysis, showing (at the end) the improvements or deficiencies in relation to the last one.
✔️ Shows analysis statistics: either against a specific URL or all of them.
✔️ Shows fingerprint statistics: either against a specific term or the Top 20.
✔️ Code reviewed via pycodestyle, SonarLint and Sourcery.
✔️ Tested, one by one, on thousands of URLs.
✔️ Fully tested and working on Windows (10 20H2 - 19042.985) and Linux (Kali 2021.1).
✔️ All code under one of the most permissive licenses: MIT.
✔️ Regularly updated.
✔️ Developed in my spare time, with no relationship with clients/companies: try it and integrate it into your projects!.
✔️ Technical resource accepted in the OWASP Secure Headers Project and Kali Linux.

Screenshots

.: (Windows) - Brief analysis.

.: (Linux) - Brief analysis and retrieved HTTP headers.

.: (Linux) - Detailed analysis in Spanish.

.: (Linux) - List of HTTP fingerprint headers based on a specific term.

.: (Windows) - Detailed analysis exported to PDF. Example.

.: (Linux) - Detailed analysis exported to HTML. Example.

.: (Linux) - Analysis history file: Date, URL, Missing, Fingerprint, Deprecated/Insecure, Empty headers & Total warnings (the four previous totals).

.: (Linux) - Statistics of the analysis performed against a specific URL.

.: (Linux) - Statistics of the analysis performed against all URLs.

Installation & Update

NOTE: Python 3.9 or higher is required.

# install python3 and python3-pip if not exist
(Windows) https://www.python.org/downloads/windows/
(Linux) if not installed by default, install them via, e.g. Synaptic, apt, dnf, yum ...

# install git
(Windows) https://git-scm.com/download/win
(Linux) https://git-scm.com/download/linux

# clone the repository
$ git clone https://github.com/rfc-st/humble.git

# change the working directory to humble
$ cd humble

# install the requirements
$ pip3 install -r requirements.txt

# update humble (every week, inside humble's working directory)
$ git pull

# or download the latest release (every four to five weeks)
https://github.com/rfc-st/humble/releases

Usage

(Windows) $ py humble.py
(Linux)   $ python3 humble.py

usage: humble.py [-h] [-a] [-b] [-f [TERM]] [-g] [-l {es}] [-o {html,pdf,txt}] [-r] [-u URL] [-v]

humble (HTTP Headers Analyzer) - https://github.com/rfc-st/humble

options:
  -h, --help         show this help message and exit
  -a                 show statistics of the performed analysis (will be global if '-u URL' is omitted)
  -b                 show a brief analysis (if omitted, a detailed one will be shown)
  -f [TERM]          show fingerprint statistics (will be the Top 20 if "TERM", e.g. "Google", is omitted)
  -g                 show guidelines for securing popular web servers/services
  -l {es}            show the analysis in the indicated language (if omitted, English will be used)
  -o {html,pdf,txt}  save analysis to file (with the format URL_headers_yyyymmdd.ext)
  -r                 show full HTTP response headers and a detailed analysis
  -u URL             schema and URL to analyze. E.g. https://google.com
  -v, --version      show the version of this tool and check for updates

Advanced Usage

Linux: Show only the analysis summary

$ python3 humble.py -u https://www.spacex.com | grep -A 8 "\!." | sed $'1i \n'

Windows (in Spanish): show only the analysis summary (PowerShell >= 7 required)

$ py humble.py -u https://www.spacex.com -l es | Select-String -Pattern '!.' -Context 1,8 -NoEmphasis

Linux: Show only the URL, date and analysis summary

$ python3 humble.py -u https://www.spacex.com | grep -A7 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed $'1i \n'

Linux: Show only the deprecated headers/protocols and insecure values

$ python3 humble.py -u https://www.spacex.com | sed '/3. /,/4. /!d' | sed '$d' | sed $'1i \n'

Linux: Check for HTTP client errors (4XX)

$ python3 humble.py -u https://block.fiverr.com | grep -A1 -B5 'Note : \|Nota : ' --color=never

Linux: Analyze multiple URLs and save the results as PDFs

$ datasets=('https://facebook.com' 'https://www.microsoft.com' 'https://www.spacex.com'); for dataset in "${datasets[@]}"; do python3 humble.py -u "$dataset" -o pdf; done

Checks: Missing Headers

Show / Hide


`Cache-Control`	`Clear-Site-Data`	`Content-Type`
`Content-Security-Policy`	`Cross-Origin-Embedder-Policy`	`Cross-Origin-Opener-Policy`
`Cross-Origin-Resource-Policy`	`NEL`	`Permissions-Policy`
`Referrer-Policy`	`Strict-Transport-Security`	`X-Content-Type-Options`
`X-Frame-Options`

Checks: Fingerprint headers

Check this file.

Checks: Deprecated headers/protocols and insecure values

Check this file.

Checks: Empty values

Any HTTP response header.

Guidelines included to enable security HTTP headers

Amazon AWS
Apache HTTP Server
Cloudflare
MaxCDN
Microsoft Internet Information Services
Nginx

To-do

Add more header/value checks (only security-oriented)
Google Style Python Docstrings and maybe documentation via Sphinx.

Contribute

Report a Bug.
Create a Feature request.
Report a Security Vulnerability.
Send me an email with your suggestions!: [email protected]

Thanks for your time!! :).

Acknowledgements

Azathothas for reporting this bug.
bulaktm for this suggestion.
Eduardo, for making possible the first Demo and this example.
gl4nce for this suggestion.
İDRİS BUDAK for reporting the need to this check.

manuel-sommer / humble Goto Github PK

humble's Introduction