This README provides instructions on how to run the LLaMa model on a Windows machine, with support for both CPU and GPU.

Clone the repo and run ./launch.ps1.

git clone [email protected]:treadon/llama-7b-example.git
cd llama-7b-example
./launch.ps1

This will create a virtual environment (venv), install the required packages, and run the script that will reveal the capital of Canada.

If the script fails to run, ensure that you have the following packages installed:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install transformers
pip install sentencepiece

Then, run the llama.py script:

python .\src\llama.py