josehelps / git-wild-hunt Goto Github PK

All warnings for matches are displayed twice in the output, and duplicate entries exist in results.json, even after it is deleted before the script has run.

2020-08-22 11:16:43,640 - INFO - git-wild-hunt - processing potential leak #1 on redacted
2020-08-22 11:16:43,906 - INFO - git-wild-hunt - processing potential leak #2 on redacted
2020-08-22 11:16:44,135 - INFO - git-wild-hunt - processing potential leak #3 on redacted url A
2020-08-22 11:16:44,321 - WARNING - git-wild-hunt - url: redacted
check: Amazon AWS Access Key ID matches: ['redacted key A']
2020-08-22 11:16:44,321 - WARNING - git-wild-hunt - url: redacted
check: AWS API Key matches: ['redacted key A']
2020-08-22 11:16:44,908 - INFO - git-wild-hunt - processing potential leak #4 on redacted
2020-08-22 11:16:45,282 - INFO - git-wild-hunt - processing potential leak #5 on redacted
2020-08-22 11:16:45,616 - INFO - git-wild-hunt - processing potential leak #6 on redacted

(venv) ~/git/git-wild-hunt$ cat results.json | jq '.[] | .url' | wc -l
203
(venv) ~/git/git-wild-hunt$ cat results.json | jq '.[] | .url' | sort -u | wc -l
100
(venv) ~/git/git-wild-hunt$ cat results.json | jq '.[] | .matches' | grep '"' | wc -l
219
(venv) ~/git/git-wild-hunt$ cat results.json | jq '.[] | .matches' | grep '"' | sort -u | wc -l
104

How do I get Github credentials? I'm not getting in any way

https://semgrep.dev/p/secrets as well as: https://github.com/odomojuli/RegExAPI, https://github.com/daffainfo/Git-Secret/blob/main/Dorks/dorks.txt

If you’ll indulge me by leaving this open a bit,

I’d like to contribute 1) a GitHub action .yml and 2) see about wiring up Docker release images to publish on the repo.
...
Both are straightforward tasks.
Shouldn’t be more than an hour this weekend if I play my cards right.
—-
Why?
I don’t have python on my host, and would like to use your project. Which means the developing the above contributions preempt me using it much anyways. 🙂
Wrt this issue, in a more meta sense: Just, saw 0 issues, and would love to not only use this but contribute upstream if you’re interested.
—-
Would you like an upstream ~PR? Lmk thoughts on this facet and if you’d like that to take a certain form.
Feel free to delete, mark wontfix, etc, nbd.

Wanted to stub this for provenances’ sake prior to allocating the dev cycles.

Cheers!
e

It fails if I mention the organization search fitler. I am trying to use this to scan our private Github organization:

python git-wild-hunt.py -s " extension:yml org:myorg filename:deploy.yml language:YAML"

Here is the error code:

git-wild-hunt is using config at path git-wild-hunt.conf
2020-06-21 03:20:30,636 - INFO - git-wild-hunt - INIT - git-wild-hunt v1
Traceback (most recent call last):
File "git-wild-hunt.py", line 190, in
results = search_github(github_token, s)
File "git-wild-hunt.py", line 30, in search_github
log.info("total results: {}".format(result['total_count']))
KeyError: 'total_count'

My questions:
Can I use it to scan my private repo with the right token?
Does it support organization search filter?

python3 git-wild-hunt.py -s "path:.ssh filename:id_rsa"  100%  144.4 Mbps
git-wild-hunt is using config at path git-wild-hunt.conf
2021-07-26 22:34:31,597 - INFO - git-wild-hunt - INIT - git-wild-hunt v1
Traceback (most recent call last):
File "//git-wild-hunt/git-wild-hunt.py", line 216, in
regexes = load_regexes(config['regexes'])
File "//git-wild-hunt/git-wild-hunt.py", line 19, in load_regexes
regexes = json.loads(f.read())
File "/usr/lib/python3.9/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 3 column 5 (char 52)

would be nice to have a match on strings with high entroy additionally to regex!