Konan is your one-stop platform to bring your AI models to life. Typically, one would be required to do some tedious boring work in order to prepare the machine learning or data science model for deployment. Not with us! With Konan, gone are the days of worrying about a model's dependencies, battling with containerization or obsessively hand-holding the infrastructure. This repo will guide you in preparing your machine learning model in as seamless a manner as possible! Of course, don't forget to also checkout Konan's rich documentation for additional info.

This repository contains the source code and tooling necessarily to generate the konan-housing-model docker image, a Konan-ready ML model packaged and distributed as a Docker image. The model aims to estimate the sale price of house based on the popular Ames Housing Dataset and the Jupyter notebook titled #1 House Prices Solution [top 1%], generously provided to the public by Nanashi.

You can also find the built image and its tags on Dockerhub directly.

• notebook.ipynb contains the original Jupyter notebook as provided by Nanashi. It is used as a reference for the python scripts produced by this repo in the housing-model/src/playground directory.
• The .konan.example environment file is provided as a template for end users, should they wish to replicate building this repository and generate a different Konan-compatible docker image from it.
• build.sh is the script that the Github Workflow for this repo uses to generate the new ML-model to be used in the docker images.
• The tests/ directory contains the unit and integration tests that the Github Workflow will use to test any new changes added to the repo.
• The data/ directory contains the datasets used by this repository
  • The data/kaggle directory contains a copy of the original Ames Housing Dataset, sourced from Kaggle directly.
  • The data/final directory contains cleaned .csv files that are Konan-compatible. In particular, some columns are renamed, others are dropped. This is the dataset that is used to train, validate and test the Konan-models generated by this repo.
  • The data/metadata.yml file contains some crucial information for the encoders used by the ML-model.
• housing-model is the main directory for the repo. Some highlights about its contents include:
  • The housing-model/retrain.sh script file includes the command to trigger a Model Retraining on Konan for the generated model.
    • Check konan-docs for more information on Model Retraining.
  • The housing-model/src/playground directory contains the python scripts that generates the cleaned datasets (data/final/* files), given the original datasets from Kaggle. (data/kaggle/* files) and the data/metadata.yml file.
  • The housing-model/src/app directory contains the python scripts for the Konan model's web-server, including the retrain.py and server.py files.
    • Check konan-docs for more information on the requirements for generating a Konan Model.
  • The housing-model/src/serving directory contains python scripts that uses konan-sdk to interact with a Konan deployment created using a konan-housing-model image.
    • Check the documentation of konan-sdk to learn more about how to use it.
    • Make sure to generate a .env file from the provided .env.example example file.

To ensure that your local environment contains all of the required dependencies, run poetry install to install the dependencies in a virtual environment. You can learn more about poetry here.

1. Go to the Dockerhub page for this image and select the tag you want to deploy, and make note of the image url. The image url of your chosen tag is the part after the corresponding docker pull.

   • For example, if you are using the latest tag, then the image url will be:

     konanai/konan-housing-model:latest

   • Don't forget to updated the latest tag. It is always recommended to use a specific, versioned tag and not latest.

2. Go to Konan, login and deploy this image as a new Deployment! Perhaps you'd find the relevant instructions on Konan docs useful.
   • Use the image url that you took note of in the previous step.
   • Leave the Container Registry Credentials blank, as you are using a publicly-available image.

After following Konan's documentation on deploying the konan-housing-model on Konan, you may wish to use the provided housing-model/src/serving/main.py script to interact with your newly-created deployment.

1. Log in to Konan and retrieve the UUID of the Deployment you created using the konan-housing-model.

2. Create a .env file in the housing-model/src/serving directory using the provided .env.example example file there.

3. Export the needed environment variables

   . ./housing-model/src/serving/.env

4. Trigger some predictions and provide their feedback using the housing-model/src/serving/main.py script file

   python housing-model/src/serving/main.py

python housing-model/src/playground/prepare.py

1. Export some needed environment variables

   export KONAN_MODEL_REGRESSOR_NAME="svr"  # only svr and xgboost are currently supported

2. Create an empty artifacts/<ml-algorithm> directory

   mkdir -p artifacts/"svr"  # must match the name of the above regressor name

3. Run the train script

   python housing-model/src/playground/train.py

1. Ensure you have trained the model, as described in the above steps

2. Export the same needed environment variables, with the same values

   export KONAN_MODEL_REGRESSOR_NAME="svr"  # only svr and xgboost are currently supported

3. Run the validate script

   python housing-model/src/playground/validate.py

Although you may use the official Konan docker image for your use, you may of course use the provided source to build (and optionally) publish your own version of the docker image.

1. Create a .konan environment variable from the provided .konan.example example, and don't forget to update it with your container registry credentials and desired image name and tag.

2. Use the Makefile to build the image

   make build

3. Use the Makefile to login to Dockerhub

   make repo-login

4. Use the Makefile to publish the image

   make publish-version

Of course, feel free to run make --help to familiarize yourself with the other options the Makefile provides.

You are more than welcome to contribute to this repo's source code, documentation, and tooling scripts. We ask, however, that you keep your commits self-contained and to adhere to the conventional commits standard. This helps us automate pushing new tags and builds through semantic versioning.

If you have any questions, or feel that this repo needs some improvement, feel free to raise an issue with the repository's Issue Tracker.