Repo that allows me to build AI tools on top of Hugging Face and OpenAI Whisper
"When you use a pretrained model, you train it on a dataset specific to your task. This is known as fine-tuning, an incredibly powerful training technique. In this tutorial, you will fine-tune a pretrained model with a deep learning framework of your choice:"
Tried conda with and failed:
https://huggingface.co/docs/transformers/training
Use split terminal and do:
nvidia-smi -l 1
in one
htop
in another
- Reference PyTorch site
numba -s | grep cuda
- run
python utils/verify_cuda_pytorch.py
- run
nvidia-smi
should show a GPU
conda create -n hf python=3.9
conda activate hf
conda env config vars set CONDA_SUBDIR=osx-arm64
#install nightly
pip install -U --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu
#install rust if not installed
curl — proto ‘=https’ — tlsv1.2 -sSf https://sh.rustup.rs | sh
#install via brew
brew install cmake
brew install pkg-config