Quickly install your favorite Security repos on a new system

Kick off multiple concurrent git clone tasks utilizing Python

Add/remove Security related Github repos to the default.xt list as needed in order to address indivudual use case/project needs

Clone repos to review offline while traveling without internet connectivity

This program was tested with Python3 version 3.8.9 64-bit

Ensure the Python3 virtual environment package is installed (Ubuntu)

sudo apt-get install python3-venv

Ensure git is installed (Ubuntu)

sudo apt-get install git

Clone code repo

git clone https://github.com/tbalz2319/RapidRepoPull.git

Change directory into RapidRepoPull

cd RapidRepoPull

The script will run in a minimal Alpine Docker container (126 MB) and extract the dirs in the current working dir

docker-compose up --build

Clone code repo

git clone https://github.com/tbalz2319/RapidRepoPull.git

Change directory into RapidRepoPull

cd RapidRepoPull

Execute the script below

./install.sh

Clone code repo

git clone https://github.com/tbalz2319/RapidRepoPull.git

Change directory into RapidRepoPull

cd RapidRepoPull

Create a virtual Python3 environment to run this code

python3 -m venv venv

Activate the virual enivornment

source venv/bin/activate

Install requirements

pip install -r requirements.txt

Run program

python rapid.py

Clone code repo

git clone https://github.com/tbalz2319/RapidRepoPull.git

Change directory into RapidRepoPull

cd RapidRepoPull

Execute the script below

./option4.sh

The script will exit while leaving the program in a python virtual env indicated by (venv)

Use the help menu below to navigate options

Usage: rapid.py [OPTIONS]

   Options:
   -v, --verbose      Will print verbose messages.
   -f, --file TEXT    Specify a text file with a list of user selected Github repos
   -t, --thread TEXT  Specify the number of CPU threads to use
   -u, --url TEXT     Specify a url to scrape containing Github repos to clone
   --help             Show this message and exit.```

Example specifying a custom text file with a list of repos and how many threads the script should use

The text file must reside in the same directory as rapid.py and can be called anything

The example file "bob.txt" is there to testing an example purposes

python rapid.py -f bob.txt -t 25

The example below with scrape a url containing Github to clone and provide verbose output, utilizing 75 threads

python rapid.py -v -u https://apsdehal.in/awesome-ctf -t 75

Run the following script

./update.sh

Run the command to maintain all existing repos by attempting to pull latest version

./update_repos.sh

Add error handling

Clean up code

Implement new features to enhance functionality via cli

The code for this project is constantly being cleaned up, refactored and refined, there are most likely lots of bugs that need to fixed and identified.

Any help with the project is greatly appreciated

Pull requests welcomed