GitMiner will query Github useful source code data at the Enterprise, Organization or Repository level.
The gitminer.py is called as a CLI app. It can be run with your local python.
All output gets saved to the logs/ dir in the working directory
usage: gitminer.py [-h] --hostname HOSTNAME [--org ORG_NAME] [--repo REPO_NAME] [-a]
optional arguments:
-h, --help show this help message and exit
--hostname HOSTNAME Hostname URL: acme
--org ORG_NAME Organization Name(ex: DevOps)
--repo REPO_NAME Repository Name(ex: GitMiner)
-a, --all To run in all orgs and repos, set this flag.
-n, --noprompt Do not prompt.
-c, --csv Write CSV data to file.
-j, --json Write JSON data to file.
Python 3.6+
pip3 install -r requirements.txt
All queries require authentication. You will NEED a Personal Token in the config.json file.
Make sure you do the following first:
- Copy
config.json.exampletoconfig.json - Update
config.jsonwith Github tokens
{
"ACME_GITHUB_HOSTNAME": "github.ACME.co",
"ACME_GITHUB_TOKEN": "{Your personal ACME Github Token}"
}
Get a single repo from an org:
Local run:
$ python3 gitminer.py --hostname acme --org DevOps --repo ghe-miner
Get all repos for a single org:
$ python3 gitminer.py --hostname acme --org DevOps
Get all repos for all orgs:
$ python3 gitminer.py --hostname acme --all
**************************************************
GITMINER:
A tool to mine useful data from Github Enterprise
**************************************************
[.] Are you sure you want to run this script in github.acme.co: (y/n)
y
[.] Using: github.acme.co
Organization: DevOps
Owner: John.Doe
Org Url: https://github.acme.co/DevOps
Repos:
|
|- Repo Name: Maestro
|- Core Language: Java
|- Languages:
|
|- Language: Java
|- Byte Size: 3.1650390625 KB
|
|- Language: Kotlin
|- Byte Size: 1.2666015625 KB
|
|- Language: Dockerfile
|- Byte Size: 0.333984375 KB
|- Root Content Files:
|
| * .gitattributes
| * .gitignore
| * .idea
| * Dockerfile
| * Jenkinsfile.norfolk
| * README.md
| * build.gradle.kts
| * build.yaml
| * docker-compose.debug.yml
| * docker-compose.yml
| * gradle
| * gradlew
| * gradlew.bat
| * settings.gradle.kts
| * src
|
|- Repo Name: example-test
...
**************************************
[*] Grand Total number of Orgs: 1
[*] Grand Total number of Repos: 84
**************************************
Language Count Percentage
---------------- ------- ------------
Python 34 40%
Empty 22 26%
Java 8 9%
JavaScript 6 7%
HCL 5 5%
HTML 4 4%
Lua 1 1%
Rich Text Format 1 1%
TypeScript 1 1%
Dockerfile 1 1%
SaltStack 1 1%
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent