The tagline on hero section should be either:

• Visual Data Preparation Made for All
• Where Visual Data Preparation is Made for All

The copy under the title 'Integration Vision AI into existing data stack in minutes' should be separated into two sentences not one.

The title 'Fetch, Deploy and Share with the community' needs a comma before and.

If we run the make tests multiple times, there is a high possibility the tests in model-backend-rest.js won't pass.

     █ Model Backend API: Load model online

       ✓ POST /models (multipart) cls response Status
       ✓ POST /models (multipart) cvtask cls response Name
       ✓ POST /models (multipart) cvtask cls response FullName
       ✓ POST /models (multipart) cvtask cls response CVTask
       ✓ POST /models (multipart) cvtask cls response Versions
       ✓ PATCH /models (multipart) cls response Status
       ✓ PATCH /models (multipart) cvtask cls response Name
       ✓ PATCH /models (multipart) cvtask cls response FullName
       ✗ PATCH /models (multipart) cvtask cls response CVTask
        ↳  0% — ✓ 0 / ✗ 1
       ✗ PATCH /models (multipart) cvtask cls response Versions
        ↳  0% — ✓ 0 / ✗ 1
       ✗ PATCH /models (multipart) cvtask cls response Version 1 Status
        ↳  0% — ✓ 0 / ✗ 1

Summary

Now, pipeline-backend only supports inference by multipart image uploading, we want to support inference by providing image URL or image in base64 format.

{
  "contents": [
    {
      "url": "https://artifacts.instill.tech/dog.jpg"
    },
    {
      "base64": "/9j/4AAQ...Kkdf/9k="
    }
  ]
}

Relevant issue

• https://github.com/instill-ai/model-backend/issues/27

Now update model version method only support update status ONLINE/OFFLINE. But model version also has description fields to explain what is changed in this version. User wants to update the version description aslo.

We will release a holistic version for all protocol buffer files for all backends.

This is a flavour decision for our API practice. We prefer using a fine-grained version control package instill.pipeline over an imposed-upstream control package instill.pipeline.v1.

When protobufs is released, a GitHub Action workflow will trigger auto-gen for all gRPC client-server codes for all language repos (e.g., protogen-go) and also update all related gRPC/OpenAPI documents

Why

Right now newsletter archive is SSG, every time we release new newsletter we have to manually re-deploy the whole site. This will cause overhead.

How

• Implement SSR
• Use pagination or staged fetching to reduce the initial load time of SSR

Deploy the model

If I run the following script in quick start

# Deploy the model
go run deploy-model/main.go --model-path yolov4-onyx-cpu.zip --model-name yolov4

I get a model with model name yolov4 with 1 version.

2022/02/21 00:16:13 model has been created, the response is: id:1 name:"yolov4" full_Name:"local-user/yolov4" cv_task:DETECTION versions:{version:1 model_id:1 description:"YoloV4 for object detection" created_at:{seconds:1645402547 nanos:80792000} updated_at:{seconds:1645402547 nanos:80807000}}

The response missed the status of version 1 of the model.

If I run the above script the second time, I get a model with model name yolov4 with 2 versions.

bash 2022/02/21 00:20:07 model has been created, the response is: id:1 name:"yolov4" full_Name:"local-user/yolov4" cv_task:DETECTION versions:{version:1 model_id:1 description:"YoloV4 for object detection" created_at:{seconds:1645402547 nanos:80792000} updated_at:{seconds:1645402625 nanos:977017000} status:ONLINE} versions:{version:2 model_id:1 description:"YoloV4 for object detection" created_at:{seconds:1645402779 nanos:961661000} updated_at:{seconds:1645402779 nanos:961692000}}

The response only includes the status of version 1, but no status of version 2.

Test the model

# Test the model
go run test-model/main.go --model-name yolov4 --test-image dog.jpg --model-version 2

Get response

2022/02/21 00:24:15 error when triggering predict: rpc error: code = Code(400) desc = {"status":400,"title":"PredictModel","detail":"Model is offline"}

Note: shouldn't we use status code 422 instead of 400 for the above scenario?

But when GET /models/yolov4

{
    "id": 1,
    "name": "yolov4",
    "full_Name": "local-user/yolov4",
    "cv_task": "DETECTION",
    "versions": [
        {
            "version": 1,
            "model_id": 1,
            "description": "YoloV4 for object detection",
            "created_at": "2022-02-21T00:15:47.080792Z",
            "updated_at": "2022-02-21T00:20:09.486272Z",
            "status": "ONLINE"
        },
        {
            "version": 2,
            "model_id": 1,
            "description": "YoloV4 for object detection",
            "created_at": "2022-02-21T00:19:39.961661Z",
            "updated_at": "2022-02-21T00:20:09.486272Z",
            "status": "ONLINE"
        }
    ]
}

The response shows both model versions are online.

Related to model backend issue

This seems to be a missing correction.

When we do

make all
make test

The tests won't pass because the tests rely on an existing online yolov4 model.

We should use a tiny dummy model for testing and fix this issue to make the tests self-contained.

Why

• We don't have instill cloud section on the current landing page.

How

• Implement new version of landing page design ASAP

Related PR instill-ai/protobufs#34

When running the quick start example, sometimes it produces errors, for example, Triton server not-ready error (503). We need to improve the example or provide troubleshooting guidelines.

Currently, we only validate model in the recipe. We also need to validate source and destination.

Related issue: #202

Summery

Here we collect a list of tasks that needs to be completed to release the VDP project

Tasks

VDP

Quick start example

• #181

Improve Documentation

• How VDP works
  • instill-ai/vdp#52
  • #184
  • #183
• Quick start
  • #182

Protobufs

• https://github.com/instill-ai/protobufs/issues/8

Protogen-go

• https://github.com/instill-ai/protogen-go/issues/20

• Related to vdp #177
• Tested with Triton image 22.01-py3 and 21.09-py3
• Tried to increase ulimit stack and shm but have same result
  Note: have no logging before Triton exited

Protobufs

• #277
• https://github.com/instill-ai/protobufs/issues/33

model-backend

• https://github.com/instill-ai/model-backend/issues/44
• https://github.com/instill-ai/model-backend/issues/33
• https://github.com/instill-ai/model-backend/issues/45
• https://github.com/instill-ai/model-backend/issues/30

pipeline-backend

• #206
• #205

vdp

• Use the latest docker images in vdp

model-backend has a docker command "./openapi" at the moment.

It should be model-backend to cohere other backends.

Why

An online model (or model version) should not be allowed to be set offline nor to be deleted when it is being used by any activated pipeline.

We need to introduce the above critical dependency to avoid breaking a working pipeline.

• #45
• #12

Why

• We need a career page to display we are looking for new role

Date -> Data

What

• magic flow is missing at mobile.
• It may be caused by non-standard height or non-specific height.

https://github.com/instill-ai/vdp/blob/d81f5649265c2deb1ff9b08fd31b2ad779450922/README.md?plain=1#L63

Image pulling is a one-time effort.

After creating an object detection pipeline by following the quick start tutorial, a user would want to deploy its own model to create a pipeline.

Given a model, we need to provide guidelines on

• how to verify whether the model is supported or not
• how to prepare the model to be deployed
• how to deploy the model and create a pipeline with the model

Add unit testing for all backends. Please tick off the check box when it is done

• mgmt-backend @xiaofei-du
• connector-backend @pinglin
• model-backend @Phelan164
• pipeline-backend @pinglin

Current redoc deploys the OpenAPI docs from the main branch of the protobufs.
It should deploy the corresponding OpenAPI docs that are compatible with the pipeline-backend and model-backend in the docker-compose.yml.

Why

• landing page banner is too big, it needs to be responsive

https://github.com/instill-ai/vdp/blob/814ea12e94b7ebb4656c7208f5c160c646f8523a/Makefile#L73

Related issues

• #179
• #180

buf has provided a list of rules and categories as well as style guidelines, which we should follow.

https://github.com/instill-ai/model-backend/blob/9eb75d70b00198b6a0362e294104d07a082f7ba4/README.md?plain=1#L16

The behaviour of unload is different from load for an ensemble model.

load will load all the ensemble model's dependent models into Triton:
https://github.com/instill-ai/model-backend/blob/66b034d0f9ef9af61c2823ae01df111db6c5abaa/pkg/services/model.go#L368

, while unload will unload only the ensemble itself and leave the dependent in the Triton:
https://github.com/instill-ai/model-backend/blob/66b034d0f9ef9af61c2823ae01df111db6c5abaa/pkg/services/model.go#L388

This causes OOM after a few patching on an ensemble model if its dependents are not unloaded together or purged regularly.

The current supported Computer Vision (CV) tasks include image classification and object detection.

We need to provide guidelines on

• how to verify whether an image classification or detection model is supported or not
• what to do if a model is not supported

Test highlights (including but not limited to):

• CV task parser
• use a tiny dummy model for testing, no need to download remote model

Why

Right now, we have IconBase component defined like below, we need to remove default my-auto and make it more flexible

import { FC } from "react";
import * as classNames from "classnames";

interface Props {
  viewBox: string;
  styleName: string;
}

export const IconBase: FC<Props> = ({ children, viewBox, styleName }) => {
  return (
    <svg
      xmlns="http://www.w3.org/2000/svg"
      viewBox={viewBox}
      className={classNames.default("fill-current my-auto", styleName)}
    >
      {children}
    </svg>
  );
};

Issues to be resolved for v0.2.0-alpha release will be added here.

Backends

• #45

pipeline-backend

• Support creating a SYNC pipeline with http as source, MySQL as destination.

vdp

• Use the latest docker images for all backends
• #46
• [ ] Add Roadmap

The model type is mistakenly set as tensorrt.
https://github.com/instill-ai/model-backend/blob/9eb75d70b00198b6a0362e294104d07a082f7ba4/examples-go/grpc_client.go#L65

The model type is mistakenly set as tensorrt.
https://github.com/instill-ai/vdp/blob/814ea12e94b7ebb4656c7208f5c160c646f8523a/examples-go/deploy-model/main.go#L65

https://github.com/instill-ai/vdp/blob/22549306b42b3a1424f284b6e2c57ba71ff1ee98/examples-go/test-model/main.go#L78

Why

GSAP is a open-source package we use for product website's animation, it is charged once we have paid users, so we need to consider adapt other animation package in the future

The VDP project span over multiple repos including

• vdp
• pipeline-backend
• model-backend
• protobufs
• protogen-go

It would be nice if we provide docs to explain the role of each repo in the project, so the community has a better understanding of the VDP project.