Note
The legacy APIs no longer work with the latest version of the
Text Generation Web UI. They were deprecated in
November 2023 and have now been completely removed.
If you want to use the LEGACY APIs, please set the image tag
to 1.9.5
. You will also have to add port 6000 for the
legacy REST API and/or port 6005 for the legacy Websockets API.
- Ubuntu 22.04 LTS
- CUDA 12.1.1
- Python 3.11.9
- Text Generation Web UI v1.11
- Torch 2.2.2
- Jupyter Lab
- code-server
- runpodctl
- OhMyRunPod
- RunPod File Uploader
- croc
- rclone
- speedtest-cli
- screen
- tmux
This image is designed to work on RunPod. You can use my custom RunPod template to launch it on RunPod.
Note
You will need to edit the docker-bake.hcl
file and update REGISTRY_USER
,
and RELEASE
. You can obviously edit the other values too, but these
are the most important ones.
# Clone the repo
git clone https://github.com/ashleykleynhans/text-generation-docker.git
# Log in to Docker Hub
docker login
# Build the image, tag the image, and push the image to Docker Hub
cd text-generation-docker
docker buildx bake -f docker-bake.hcl --push
# Same as above but customize registry/user/release:
REGISTRY=ghcr.io REGISTRY_USER=myuser RELEASE=my-release docker buildx \
bake -f docker-bake.hcl --push
docker run -d \
--gpus all \
-v /workspace \
-p 3000:3001 \
-p 5000:5001 \
-p 7777:7777 \
-p 8888:8888 \
-p 2999:2999 \
-e VENV_PATH="/workspace/venvs/text-generation-webui" \
ashleykza/oobabooga:latest
You can obviously substitute the image name and tag with your own.
Connect Port | Internal Port | Description |
---|---|---|
3000 | 3001 | Text Generation Web UI |
5000 | 5001 | Open AI Compatible API |
7777 | 7777 | Code Server |
8888 | 8888 | Jupyter Lab |
2999 | 2999 | RunPod File Uploader |
Variable | Description | Default |
---|---|---|
VENV_PATH | Set the path for the Python venv for the app | /workspace/venvs/text-generation-webui |
JUPYTER_LAB_PASSWORD | Set a password for Jupyter lab | not set - no password |
DISABLE_AUTOLAUNCH | Disable Web UI from launching automatically | (not set) |
DISABLE_SYNC | Disable syncing if using a RunPod network volume | (not set) |
Text Generation Web UI creates a log file, and you can tail the log instead of killing the service to view the logs.
Application | Log file |
---|---|
Text Generation Web UI | /workspace/logs/textgen.log |
For example:
tail -f /workspace/logs/textgen.log
Pull requests and issues on GitHub are welcome. Bug fixes and new features are encouraged.