Extract valid or partially valid domain names and IPs from malicious or invalid URLs.

Keep in mind that the algorithm is not that perfect, there might be false positives.

Tested on Kali Linux v2021.2 (64-bit).

Check the testing URLs here and the results here.

Made for educational purposes. I hope it will help!

Future plans:

• decode Unicode URLs,
• detect IPv6 addresses.

Open your preferred console from /src/ and run the commands shown below.

Install required packages:

pip3 install -r requirements.txt

Run the script:

python3 domain_extractor.py

Extract hosts from the results:

jq -r '.[].hosts[]' results.json | sort -u -f | tee -a hosts.txt

Extract URLs with valid or partially valid hosts from the results:

jq -r '.[] | if (.hosts != []) then (.original) else (empty) end' results.json | sort -u -f | tee -a valid_urls.txt

Extract URLs with no valid nor partially valid hosts from the results:

jq -r '.[] | if (.hosts == []) then (.original) else (empty) end' results.json | sort -u -f | tee -a invalid_urls.txt

Figure 1 - Help

Figure 2 - Validating