obheda12 / gitdorker Goto Github PK
View Code? Open in Web Editor NEWA Python program to scrape secrets from GitHub through usage of a large repository of dorks.
A Python program to scrape secrets from GitHub through usage of a large repository of dorks.
i am getting this error with valid token that never used:
https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
also which one of token type need? Fine-grained or classic
[#] 0 organizations found.
[#] 0 users found.
[#] 240 dorks found.
[#] 0 keywords found.
[#] 1 queries ran.
[#] 240 urls generated.
[#] 1 tokens being used.
[#] running 1 threads.
[#] 29 requests per minute allowed
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
[-] error occurred: https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits
Hello
Thanks for the amazing Tool
I wonder if i can , Launch it on Only one Repository , not the entire Organisation ??
Best regards
To run the GitDorker script, author has given sample command on the main page. https://github.com/obheda12/GitDorker
The command is python3 GitDorker.py -tf TOKENSFILE -q tesla.com -d dorks/DORKFILE -o tesla
If user runs this command as it is, it will give error because the dorks folder in main directory is named as Dorks and in command its written as dorks
Please make D capital in command OR rename the folder from Dorks to dorks
Problem: Every time I run gitdorker I get a 0 byte CSV file
python3 GitDorker.py -tf example_tf -org example_org -d alldorksv3 -output example_out
but example_out.csv is always empty, despite seeing lots of results in the console output
i am using 5 tokens from 2 different accounts
command i tried-
python3 GitDorker.py -tf /root/git_tokens.txt -org orgname -d /root/alldorksv3 -o githubdorks.txt
also error same with "-u mygithubaccount"
weird thing is with -t single_token both above options works fine
but quite slow.
please help i want to use -tf option somehow.
It would be awesome if this supported the Github Enterprise endpoint so that orgs can use this tool to identify internal threats
used this command
python3 GitDorker.py -tf token.txt -org companyname -d Dorks/medium_dorks.txt -lb
Rate limiting errors I am getting after 30 requests
Github api allows up to 10 requests per minute for unauthenticated requests https://docs.github.com/en/rest/reference/search#rate-limit
so it is possible to add a free decrease in time by adding requests that doesn't contain Auth header
It would be really nice to have the ability to run this via Docker with the ability to just pass in the env variables. Would this be something you'd like to see contributed?
Bro whats command in terminal ? if i want to found or grab Bulk Query???
How to overcome rate limit issue (github) ? tell me some suggestions
/ _ ___ ___ __ / / /__
/ , / -|-</ // / / __(-<
//||_//_,//_/__/
[+] SUCCESS | RESULTS RETURNED
[#] NEUTRAL | NO RESULTS RETURNED
[-] FAILURE | RATE LIMITS OR API FAILURE
QUERY PROVIDED: *.com
[-] Traceback (most recent call last):
File "GitDorker.py", line 404, in
sys.stdout.write(colored('%s' % new_url, 'white'))
NameError: name 'new_url' is not defined
am I missing something with the keyword? (-k) tried from dork file results and it finds nothing:
python3 GitDorker.py -tf tf/TOKENSFILE -q tesla.com -k filename:sshd_config -o tesla
python3 GitDorker.py -tf tf/TOKENSFILE -q tesla.com -k language:yaml -o tesla
python3 GitDorker.py -tf tf/TOKENSFILE -q tesla.com -k "pwd" -o tesla
python3 GitDorker.py -tf tf/TOKENSFILE -q tesla.com -k "slack_api" -o tesla
python3 GitDorker.py -tf tf/TOKENSFILE -q tesla.com -k "private_key" -o tesla
with and without quotes and alot of combos.
any ideas?
just want to search a single term/keyword.
I have included pictures of the same errors in powershell and on centOS8.
I've also included one picture of the pool.py errors at line 48 "in mapstar return list(map(*args))" and the GitDorker.py error at line 325 " line 325, in pool.map(api_search, url_dict)".
The other picture is the errors in GitDorker.py at line 140 and 164
" line 140, in token_round_robin current_token = tokens_list[n] IndexError: list index out of range"
"line 164, in api_search headers = {"Authorization": "token " + token_round_robin()}"
←[0mTraceback (most recent call last):$$$$$$$$ Dorking In Progress $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ 3/240
File "C:\Users\GuyWhoNotSoSmart\GitDorker\GitDorker.py", line 325, in
pool.map(api_search, url_dict)
File "C:\Users\GuyWhoNotSoSmart\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "C:\Users\GuyWhoNotSoSmart\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 771, in get
raise self._value
File "C:\Users\GuyWhoNotSoSmart\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "C:\Users\GuyWhoNotSoSmart\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 48, in mapstar
return list(map(*args))
File "C:\Users\GuyWhoNotSoSmart\GitDorker\GitDorker.py", line 164, in api_search
headers = {"Authorization": "token " + token_round_robin()}
File "C:\Users\GuyWhoNotSoSmart\GitDorker\GitDorker.py", line 140, in token_round_robin
current_token = tokens_list[n]
IndexError: list index out of range
It would be cool if there were a function --results whitch outputs in terminal/file only the + results instead of # and -
When try to install the requirements. got this error message.
ERROR: Could not find a version that satisfies the requirement itertools (from -r requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for itertools (from -r requirements.txt (line 1))
Hello, it would be cool if there were a function "-qf", so you could pass the program many domains like
tesla.com
teslamtoros.com
and so on and the program iterates throught them instead creating a new proccess for each domain with copy-paste
Hi @obheda12 ,
Any plans to support Github enterprise in near future?
technically for this feature, it need to search only dorks without any query (or query optional) for internal git servers.
This will be very helpful feature for PT or red teams.
Wondering if anyone is having this issue:
File "GitDorker.py", line 485, in
sys.stdout.write(colored('%s' % new_url, 'white'))
NameError: name 'new_url' is not defined
Thanks
[-] error occurred: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /search/code?q=org%3Acompany+access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f589053c940>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
ihave a good internet
Already similar kind of one issue #19 is there and it is closed. But the solution given in the issue was increase the no. of personal tokens, but i have already done that. I first run the command (python3 GitDorker.py -d Dorks/akeyless-dorks.txt -q akeyless.io -tf tf/TOKENSFILE -o akeyless-output.txt) with 2 tokens, it showed me the same error and then i run the command with four tokens, then also i got the same error. And the token is placed in a text file as per your guidelines only(i.e., without spaces and without newlines). I dont know what is the issue here. Pls help...
Hey
I tried your project and generate a access key as advised, it worked and produced some output with potential results. However, when I copy and past the link it always says that no results were found. I'm logged with the same account and also tried with a new incognito window.
Ant advise?
The tool is awesome and it was working but after sometime it giving me error like this
[-] Traceback (most recent call last):
File "/root/Tools/GitDorker/GitDorker.py", line 404, in
sys.stdout.write(colored('%s' % new_url, 'white'))
NameError: name 'new_url' is not defined
File "GitDorker.py", line 325, in
pool.map(api_search, url_dict)
File "D:\python38\lib\multiprocessing\pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "D:\python38\lib\multiprocessing\pool.py", line 771, in get
raise self._value
File "D:\python38\lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "D:\python38\lib\multiprocessing\pool.py", line 48, in mapstar
return list(map(*args))
File "GitDorker.py", line 164, in api_search
headers = {"Authorization": "token " + token_round_robin()}
File "GitDorker.py", line 140, in token_round_robin
current_token = tokens_list[n]
IndexError: list index out of range
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.