Model Host is a template for mock-deployment of a machine learning model built with Docker and FastAPI. Its main purpose is to launch a docker container running a uvicorn server that can accept model inputs via POST requests.

To use this template, all you need to do is replace your model code in model.py and write (or generate) a Pydantic data model in data_model.py. This data model should specify the input to your model's entrypoint.

I hope to make this into a Cookiecutter template one day, but until then use it by modifying the parts you need by hand.

For sending data, install the client requirements:

pip install -r requirements/client.txt

Optionally, if you want to use datagen to make your Pydantic data models,

pip install -r "requirements/client.txt[datagen]"

Everything under the app directory is related to the docker container. The main.py script is concerned with the FastAPI app itself, and the app.model.py file is the place for you to insert the code that will run your model.

Since this is built with FastAPI, you can specify a Pydantic data model for requests to this app. See the next session if you want a quick way to do it, or you can do it manually. If you have a pydantic data model setup for your model input, go ahead and use that. Optionally, you can use datamodel-code-generator to generate one from a sample_data.json file:

make data_model

Then read over the generated python script and double-check it matches what you are expecting.

If you want to launch the uvicorn server without docker, use

make server

and it will run the app with uvicorn.

When you're ready to make it a docker container, build the image and run the container with

make docker

You can modify the send_data.py script as needed to send data to the model. All you need to do is send a POST request (with your json input) on port 8000. If you wish, you can use make send to run the script.