Is there any way that i can add censys secret key in my github (private) , so no need to enter manually in CLI

Hello,
i see this is very good tool to find real ip behind waf. does it have support for rotating API key?, so that when API key reaches quota limit it will use next API keys. thank you

I have been using this tool for awhile and i noticed 50% of the results are false positive. for i,e after cloudflair dump a list of ip addresses for the site origin server it can't detect which one is working so i after i take those ip addresses i test them manually on the both ports 443 & 80 some they work on 443 port and some they only work at 80 port.

Thank you so much for this amazing tool

...
[] Testing candidate origin servers
[] Retrieving target homepage at https://....com
[-] https://....com responded with an unexpected HTTP status code 403

Is Cloudflare detecting your tool ?
I receive the same response when using curl.
But visiting the site from the browser works fine.
I'm running it with docker, if it makes any difference.

if the censys account was non-commercial then a modification of
censys_search.py: line 30
hosts_search_results = censys_hosts.search(hosts_query)
must be modified to
hosts_search_results = censys_hosts.search(hosts_query,max_records=1000)

Cheers!

Cf. https://github.com/christophetd/CloudFlair/blob/master/cloudflare_utils.py#L7

Traceback (most recent call last):
File "cloudflair.py", line 167, in
main(args.domain, args.output_file, censys_api_id, censys_api_secret)
File "cloudflair.py", line 136, in main
hosts = find_hosts(domain, censys_api_id, censys_api_secret)
File "cloudflair.py", line 21, in find_hosts
if not dns_utils.is_valid_domain(domain):
File "/root/Desktop/info gathering/CloudFlair/dns_utils.py", line 5, in is_valid_domain
dns.resolver.query(domain, 'A')
File "/usr/local/lib/python2.7/dist-packages/dns/resolver.py", line 1132, in query
raise_on_no_answer, source_port)
File "/usr/local/lib/python2.7/dist-packages/dns/resolver.py", line 1041, in query
timeout = self._compute_timeout(start)
File "/usr/local/lib/python2.7/dist-packages/dns/resolver.py", line 858, in _compute_timeout
raise Timeout(timeout=duration)
dns.exception.Timeout: The DNS operation timed out after 30.0018880367 seconds

sudo python3 cloudflair.py

[*] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
Traceback (most recent call last):
File "/home/bart/Tools/IP-Scan/CloudFlair/cloudflair.py", line 10, in
from html_similarity import similarity
ModuleNotFoundError: No module named 'html_similarity'

There is an option to expand the timeout time to be more than 3 seconds?

cloudflair.py evil.com
Traceback (most recent call last):
File "C:\Python3\CloudFlair-master\cloudflair.py", line 3, in
import dns_utils
File "C:\Python3\CloudFlair-master\dns_utils.py", line 1, in
import dns.resolver
ModuleNotFoundError: No module named 'dns'

I was able to run this on my host system, but inside of Docker, no matter what I do, it gives me this error. Even with invalid API credentials, same error.

$ docker run --rm -e CENSYS_API_ID=MY_ID -e CENSYS_API_SECRET=MY_SECRET christophetd/cloudflair redacted.com
[*] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
[*] The target appears to be behind CloudFlare.
[*] Looking for certificates matching "redacted.com" using Censys
Traceback (most recent call last):
  File "/data/cloudflair.py", line 184, in <module>
    main(args.domain, args.output_file, censys_api_id, censys_api_secret)
  File "/data/cloudflair.py", line 153, in main
    hosts = find_hosts(domain, censys_api_id, censys_api_secret)
  File "/data/cloudflair.py", line 43, in find_hosts
    cert_fingerprints = censys_search.get_certificates(domain, censys_api_id, censys_api_secret)
  File "/data/censys_search.py", line 17, in get_certificates
    return set([ cert['parsed.fingerprint_sha256'] for cert in certificates_search_results ])
  File "/data/censys_search.py", line 17, in <listcomp>
    return set([ cert['parsed.fingerprint_sha256'] for cert in certificates_search_results ])
  File "/usr/lib/python3.6/site-packages/censys/base.py", line 173, in search
    pages = payload['metadata']['pages']
KeyError: 'metadata'

As seen in https://github.com/pielco11/fav-up, it is possible to get a servers real IP using the hash of a fav-icon.
Perhaps it would be possible to implement something like this using the current API.

see PR #51 for more details

while executing i am getting this error
Traceback (most recent call last):
File "/data/data/com.termux/files/home/cloudflair/cloudflair.py", line 184, in
main(args.domain, args.output_file, censys_api_id, censys_api_secret)
File "/data/data/com.termux/files/home/cloudflair/cloudflair.py", line 153, in main
hosts = find_hosts(domain, censys_api_id, censys_api_secret)
File "/data/data/com.termux/files/home/cloudflair/cloudflair.py", line 32, in find_hosts
if not dns_utils.is_valid_domain(domain):
File "/data/data/com.termux/files/home/cloudflair/dns_utils.py", line 5, in is_valid_domain
dns.resolver.query(domain, 'A')
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/dns/resolver.py", line 1131, in query
return get_default_resolver().query(qname, rdtype, rdclass, tcp, source,
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/dns/resolver.py", line 1041, in query
timeout = self._compute_timeout(start)
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/dns/resolver.py", line 858, in _compute_timeout
raise Timeout(timeout=duration)
dns.exception.Timeout: The DNS operation timed out after 30.002305269241333 seconds

Some websites appear to be blocking requests that don't have a legitimate-looking user agent.

Hi Chris,

I am using the latest Kali 2020.4
My current python2 is 2.7.18 and python3 is 3.8.6U*
I did run the exact command as the instruction
But when I run the command "python cloudflair.py "
And the error comes

File "cloudflair.py", line 3, in
import dns_utils
File "/home/kali/Desktop/cloudflair/dns_utils.py", line 1, in
import dns.resolver
ImportError: No module named 'dns'

Then I tried to run the command "python3 cloudflair.py "
It works but still got error

I am wondering is SyntaxWarning error in line 46, 55, 157 affect the process as well as the result?
How to fix it?

Hi @christophetd!

This is good work, and great projection as well, while using I have noticed it uses certificate to extract hostname, but limited to host using Cloudflare which make sense here as per project, but it would be nice if we can extract all host for any given domain as well (it might be new project) or if you like to add into this only, what you think about it?

Hello,

$ python cloudflair.py myvulnerable.site

on this page please put

$ python3 cloudflair.py myvulnerable.site

because of on python version 2 it was not working so please put python3

It seems like it's nearly easier to go to their website, search domain you want.

Then grab an IP and send curl host header to identify if that's the right one.

Add functionality so it can do that too? Instead of only certs. My domains IP is on the site but it cant find it using this tool

i use python 3.6
python3 cloudflair.py mytarget.tld
[*] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
Traceback (most recent call last):
File "cloudflair.py", line 13, in
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
AttributeError: 'module' object has no attribute 'disable_warnings'

I keep getting this error, both on python2 & 3 on a particular site. first time encountering tho

I have dnspython correctly installed but it shows no module named dns.python?

$ python cloudflair.py --help
Traceback (most recent call last):
File "cloudflair.py", line 3, in
import dns_utils
File "/home/kali/CloudFlair/dns_utils.py", line 1, in
import dns.resolver
ImportError: No module named dns.resolver

I followed every step and everything. It is not working help pls

Traceback (most recent call last):
File "cloudflair.py", line 3, in
import dns_utils
File "/root/Desktop/CloudFlair/dns_utils.py", line 1, in
import dns.resolver
ImportError: No module named 'dns'

python2 cloudflair.py www.ikhwanonline.com
[] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
Traceback (most recent call last):
File "cloudflair.py", line 184, in
main(args.domain, args.output_file, censys_api_id, censys_api_secret)
File "cloudflair.py", line 153, in main
hosts = find_hosts(domain, censys_api_id, censys_api_secret)
File "cloudflair.py", line 32, in find_hosts
if not dns_utils.is_valid_domain(domain):
File "/data/data/com.termux/files/home/CloudFlair/dns_utils.py", line 5, in is_valid_domain
dns.resolver.query(domain, 'A')
File "/data/data/com.termux/files/usr/lib/python2.7/site-packages/dns/resolver.py", line 1132, in query
raise_on_no_answer, source_port)
File "/data/data/com.termux/files/usr/lib/python2.7/site-packages/dns/resolver.py", line 1041, in query
timeout = self._compute_timeout(start)
File "/data/data/com.termux/files/usr/lib/python2.7/site-packages/dns/resolver.py", line 858, in _compute_timeout
raise Timeout(timeout=duration)
dns.exception.Timeout: The DNS operation timed out after 30.0032861233 seconds
$ python3 cloudflair.py www.ikhwanonline.com
cloudflair.py:46: SyntaxWarning: "is" with a literal. Did you mean "=="?
if len(cert_fingerprints) is 0:
cloudflair.py:55: SyntaxWarning: "is" with a literal. Did you mean "=="?
if len(hosts) is 0:
cloudflair.py:157: SyntaxWarning: "is" with a literal. Did you mean "=="?
if len(origins) is 0:
[] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
Traceback (most recent call last):
File "cloudflair.py", line 7, in
import censys_search
File "/data/data/com.termux/files/home/CloudFlair/censys_search.py", line 1, in
import censys.certificates
ModuleNotFoundError: No module named 'censys'

Traceback (most recent call last):
File "cloudflair.py", line 166, in
main(args.domain, args.output_file, censys_api_id, censys_api_secret)
File "cloudflair.py", line 137, in main
origins = find_origins(domain, hosts)
File "cloudflair.py", line 127, in find_origins
page_similarity = similarity(response.text, original_response.text)
File "/usr/local/lib/python2.7/site-packages/html_similarity/similarity.py", line 6, in similarity
return k * structural_similarity(document_1, document_2) + (1 - k) * style_similarity(document_1, document_2)
File "/usr/local/lib/python2.7/site-packages/html_similarity/structural_similarity.py", line 29, in structural_similarity
tags1 = get_tags(document_1)
File "/usr/local/lib/python2.7/site-packages/html_similarity/structural_similarity.py", line 10, in get_tags
for el in doc.getroot().iter():
AttributeError: 'NoneType' object has no attribute 'iter'

After running pip install -r requirements.txt and then trying to run the script, got the following.

File "cloudflair.py", line 3, in <module> import dns_utils File "/Users/[redacted]/cloudflair/dns_utils.py", line 1, in <module> import dns.resolver ImportError: No module named dns.resolver

Only relevant once #9 will be merged.

Each time a commit is pushed, the Docker image should be built and pushed to the Docker hub. Bonus: do it only if the Dockerfile changed. Cf. https://docs.docker.com/docker-hub/builds/

Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.11/dist-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.11/dist-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f7b587c9e50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='search.censys.io', port=443): Max retries exceeded with url: /api/v2/hosts/68.232.201.126 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7b587c9e50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/cloudflair/cloudflair.py", line 198, in
main(args.domain, args.output_file, censys_api_id, censys_api_secret, args.use_cloudfront)
File "/root/cloudflair/cloudflair.py", line 167, in main
hosts = find_hosts(domain, censys_api_id, censys_api_secret, use_cloudfront)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/cloudflair/cloudflair.py", line 68, in find_hosts
hosts = censys_search.get_hosts(cert_fingerprints, censys_api_id, censys_api_secret)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/cloudflair/censys_search.py", line 46, in get_hosts
hosts_search_results = censys_hosts.search(hosts_query).view_all()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/censys/search/v2/api.py", line 212, in view_all
result = task.result()
^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/censys/search/v2/api.py", line 262, in view
return self._get(self.view_path + document_id, args)["result"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/censys/common/base.py", line 208, in _get
return self._make_call(self._session.get, endpoint, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/censys/common/base.py", line 41, in _wrapper
return _impl()
^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/backoff/_sync.py", line 94, in retry
ret = target(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/censys/common/base.py", line 39, in _impl
return method(self, *args, *kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/censys/common/base.py", line 165, in _make_call
res = method(url, **request_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='search.censys.io', port=443): Max retries exceeded with url: /api/v2/hosts/68.232.201.126 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7b587c9e50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

Traceback (most recent call last):
File "cloudflair.py", line 3, in
import dns_utils
File "/root/cloudflair/dns_utils.py", line 1, in
import dns.resolver
ImportError: No module named dns.resolver

it's working out but it seems it can't find the SSL certificates on every site i tried

[*] 0 certificates matching "myvulnerable.site" found.
Exiting.

is there any clue to this problem?

On Kali it works fine but on my Ubuntu machine it keep asking for secret key.
I already added it to bashrc file but it keep asking me. Also I tried adding with command line but it did not help.
Any ideas?

e.g. https://travis-ci.org/christophetd/CloudFlair/jobs/530628905

File "cloudflair.py", line 3, in
import dns_utils
File "/home/kali/cloudflair/dns_utils.py", line 1, in
import dns.resolver
ImportError: No module named dns.resolver

This needs to be enclosed in a try / except block to avoid throwing an uncaught exception when the page content isn't retrieved for some reason.

Hello,
I already installed the requirements and are ok but, when i run it as follows it get an error:

python cloudflair.py www.website.com
[*] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
Traceback (most recent call last):
File "cloudflair.py", line 7, in
import censys_search
File "/root/Scripts/CloudFlare/CloudFlair/censys_search.py", line 1, in
import censys.certificates
ImportError: No module named censys.certificates

Regards.

When i was running the 'python cloudflair.py url', the python printed this result:ImportError: No module named dns.resolver
What can i do next?

[*] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4
Traceback (most recent call last):
File "cloudflair.py", line 7, in
import censys_search
File "/Users/mohamedrahuman/bugbountry/tools/wafBypass/cloudflair/censys_search.py", line 3, in
from censys.common import version
ModuleNotFoundError: No module named 'censys'

Getting above error when trying to run this tool.
Tried to install the packages but still same error.

I am using MacOS Montery

Any tip, still googling to fix this. This tools seems cool, really want it work for me, thanks

File "/data/data/com.termux/files/home/cloudflair/cloudflair.py", line 184, in
main(args.domain, args.output_file, censys_api_id, censys_api_secret)
File "/data/data/com.termux/files/home/cloudflair/cloudflair.py", line 153, in main hosts = find_hosts(domain, censys_api_id, censys_api_secret) File "/data/data/com.termux/files/home/cloudflair/cloudflair.py", line 32, in find_hosts
if not dns_utils.is_valid_domain(domain): File "/data/data/com.termux/files/home/cloudflair/dns_utils.py", line 5, in is_valid_domain
dns.resolver.query(domain, 'A') File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/dns/resolver.py", line 1131, in query return get_default_resolver().query(qname, rdtype, rdclass, tcp, source,
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/dns/resolver.py", line 1041, in query timeout = self._compute_timeout(start)
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/dns/resolver.py", line 858, in _compute_timeout raise Timeout(timeout=duration)
dns.exception.Timeout: The DNS operation timed out after 30.001431703567505 seconds

i enter python cloudflair.py myvulnerable.site
[*] Retrieving Cloudflare IP ranges from https://www.cloudflare.com/ips-v4 Traceback (most recent call last): File "/home/iu/cloudflair/cloudflair.py", line 4, in <module> import cloudflare_utils File "/home/iu/cloudflair/cloudflare_utils.py", line 47, in <module> cloudflare_subnets = [ipaddress.ip_network(ip_range) for ip_range in cloudflare_ip_ranges] File "/home/iu/cloudflair/cloudflare_utils.py", line 47, in <listcomp> cloudflare_subnets = [ipaddress.ip_network(ip_range) for ip_range in cloudflare_ip_ranges] File "/usr/lib/python3.9/ipaddress.py", line 83, in ip_network raise ValueError('%r does not appear to be an IPv4 or IPv6 network' % ValueError: '<!DOCTYPE html>' does not appear to be an IPv4 or IPv6 network

I've already completed pip install -r requirements.txt

Write at least a basic set of unit tests

Sample usage:

docker run --rm christophetd/cloudflair myvulnerable.site

Once #6 will be done