PyPi package improvement

Work through these items for discovery of this package at PyPi. These stats were obtained from here.

Basic info present?         0
Source repository present?  0
Readme present?             0
License present?            1
Has multiple versions?      1
Follows SemVer?             1
Recent release?             1
Not brand new?              0
1.0.0 or greater?           0
Dependent Packages          0
Dependent Repositories      0
Stars                       0
Contributors                0
Libraries.io subscribers    0

visionai-grafana | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied

Redis grafana docker-compose file

version: '3.4'

services:
redis:
container_name: visionai-redis
image: redis
ports:
- '6379:6379'

grafana:
container_name: visionai-grafana
image: grafana/grafana
ports:
- '3000:3000'
environment:
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_BASIC_ENABLED=false
- GF_ENABLE_GZIP=true
- GF_USERS_DEFAULT_THEME=dark
- GF_DEFAULT_APP_MODE=development
- GF_INSTALL_PLUGINS=redis-datasource
volumes:
- ./provisioning/datasources:/etc/grafana/provisioning/datasources
- ./data:/var/lib/grafana
user: "$UID:$GID"

Add user to grafana service

Motivation

Scenario module currently supports adding/deleting new scenarios. It would be good to add a support for following type of command. This would make development easier, and also for new folks to contribute to the scenarios.

Example

$ pip install visionai 
$ visionai scenarios list     # list available scenarios
$ visionai scenario download smoke-and-fire-detection  # download smoke-and-fire-detection scenario
$ visionai models serve # serve available models
$ visionai scenario test smoke-and-fire-detection # test with available camera.

This type of flow would provide an easy & intuitive approach for an end-user to try out the scenario independently & quickly.

What needs to be done?

1. Add support for downloading a single model for a scenario
2. Add support for implementing scenario cli command test or validate

• On your local machine - this should open a OpenCV imshow window where you can see how the scenario is performing.

Running visionai out of box should be quick and have very minimal dependencies.

• Torch package is too big (1GB). Takes a long time to install
• OpenCV package is too big as well.
• Basically while running individual commands - we can install these packages on demand. We have support for check_depdencies and install_dependencies in util/general.py file. Need to make use of it to enable automatic installation as needed.

Motivation

Pipelines are a natural extension of what we are doing right now. We are directly managing scenarios & cameras currently. We can apply a scenario to a camera. We can apply multiple scenarios to a camera.
However, there seems to be a missing logical abstraction here. People tend to think in terms of pipelines nowadays - and I think it is a great way to make our solution much more usable.

Example

Following sequence of steps indicate how one might enable running pipelines.

visionai pipeline create --name pipe1
visionai pipeline add-camera --pipeline pipe1 --camera OFFICE-01
visionai pipeline add-camera --pipeline pipe1 --camera OFFICE-02
visionai pipeline add-scenario --piepeline pipe1 --scenario smoke-and-fire-detection
    events = [minimal]/[all]?       [all]
    duration = [always]/[schedule]?      [always]
    focus_area = [full-frame]/[bounding-box]   [full-frame]
visionai pipeline test pipe1   # validate pipeline on usb-camera
visionai pipeline start  pipe1 # start command
visionai pipeline start-daemon pipe1 # start in background
visionai pipeline stop pipe1  # stop 
visionai pipeline disable pipe1 # disable a pipeline
visionai pipeline enable pipe1 # enable a pipeline

Architecture

• Maintain pipeline definitions along with other dynamic JSON files in the same folder.
• Right now the pipelines can be within JSON files. Later we can think about adding some DB support.
• Implement each of the commands above - by following the CLI layout we have put forth.

Support for Python3.6 and 3.7

A lot of people are still using Python 3.6 and Python 3.7 models. Need to add support for this.

What to implement

We need to implement pipeline changes in Poetry - so we can compile and test the build for all these versions and push them in. We can do this only when a PR is created to be merged to main branch. Changes will be primarily for the Azure pipelines yml file. There might be some cases where things might break for Python 3.6, 3.7 -- in those cases we need to make sure to fix our code to support them.

when run visionai init it fails asking for env file

Support for face-blur model.

Since people privacy is very critical to any Vision AI solution - we first need to protect any person's faces. To implement this - we need to detect any faces identified and blur them before we run them through additional pipelines.

This is different than other models since the output of this model needs to be fed to subsequent stages of the pipeline.

How to blur faces

We will utilize yolov5-face module. This will allow us to use something that is pre-trained, already validated with several different types of images (day vs night, male vs female vs children, indoor vs outdoor scenarios).

Motivation

Once the web-app support is implemented - next enhancement on the web-app & edge-application would be the ability to stream from the device or any running scenario to the cloud.

Use case would be something like this:

• User installs visionai
• User starts web-app using visionai start web-app
• User adds cameras and scenarios on the web-app
• User may want to test out if the RTSP or local camera is working by just clicking PLAY button on the web-app. This should somehow stream everything from the edge-device to the cloud.

Architecture

At a high-level, I want to use rtsp-streamer application. Since it provides support for majority of different IP cameras, it would be an ideal source to build upon.

However, after that I am not very sure how this would be implemented overall. Several questions remain:

• visionai application is a CLI application - should we start some rtsp streamer application in the background when starting?
• How would web-app communicate with the edge-machine? Through web-sockets or redis pubsub? We implemented something similar on Azure IoTEdge - but want to keep this framework extensible to other areas as well

Prior codebase

We have some codebase available at https://github.com/visionify/opencv-livestream
This implements opencv based livestream to a web-app. I think some portions of this might be usable.

Events framework

• When an event of interest is detected - fire an event
• The event should be pretty generic JSON payload
• Add support for pushing the event into redis pubsub or Azure Event hub.
• Add support for pretty printing the event

Scenarios repo migration

Currently all scenarios are maintained in visionai-scenarios repo. Even though it is a public repo - it is difficult to have scenarios in a difficult experience for the end-user than having all of them in a single place.

Also having scenarios in a single repo - would give us options to create a new scenario browser web-app that can be launched from the web-app directly.

Scenario

• We have some models already downloaded on the system.
• User starts the models by visionai models serve
• User downloads one more model
• The new model is not automatically served by the model server. User has to restart the model server. However this is not very intuitive to the end user.
• Impact of this issue is more frequent when you don't have any models and have started the service, then download additional models. There is non indication of what is going on.

Proposed solution:

• Anytime a new model is downloaded - then restart the model serve application. This would ensure that models being served are same as the new models that are downloaded. This would be more intuitive

After installation API is not running.

executing visionai init prompts about Env file.

Where I can find any docs about configuration and running demo with external camera?
Almost every command fails with error:

VisionAI API Failed to establish a connection to the server. Please check if the server is running and reachable.

I managed to execute some demos with webcam, but unable to connect to my rtsp cam or video from file.

Include images, annotation examples on main page README.
Check for the working commands
Update documentation in such a way that it reflects the latest working CLI commands

• Tested the application on MacOS platform.
• It works good until a point - when triton server needs to be run.
• It downloads the triton server, but is unable to run it. Need to think through how to enable this. We may need a Triton server docker build for Mac OSx instead.

visionai models serve

Add support for visionai models serve in CI/CD tests. Right now we have not added any of these test cases to CI/CD because it involves pulling triton docker image which may be too large. Try to optimize this process as much as possible so we can run this as part of CI/CD on occasions (likely weekly once etc).

Motivation

Currently all the activity for configuring pipelines, running scenarios etc happens through the CLI. Need support to implement it through web-app. We have a high-level web-app built which shows available scenarios, and allows to create/delete cameras & adding scenarios for them. We need to support opening the web-app through the CLI.

For example:

visionai webapp start  # start the web-app. 

and

visionai webapp stop # stop the web-app, save any pending states

Architecture

Use Docker compose type of architecture where we can just bring up couple of containers and web-app will be available at some end-point for us locally. Need to come up with exact architecture definition for this.

Resolve conflicts after pull main branch

Overview

There is a Github action which uses a NPM package that can check for any broken links on our website. We want to integrate support for this in our CI/CD pipeline.

Reference

For example, neptune-ai package has this support. We need to go through how they have implemented it and do the same changes for our documentation modules as well.

• Test the VisionAI application on Windows platform and see if there are any issues observed.
• How do we support for CPU only machines vs GPU enabled machines

Support an API framework for visionai application

When we open the web-app, it needs an API to talk to. This API needs to provide basic functionality with the following:

• Adding removing cameras
• Adding removing scenarios from cameras
• Managing pipelines (adding removing cameras/scenarios from a pipeline).

We need a REST API based framework for this.

How will it integrate with the rest of the application?

I think this needs to be done through some Docker magic. Say port 3001 will be for front-end application, port 3002 will be for the API application. API application itself will be a Dockerhub image that anyone can pull and use.

What database to use for the API application?

JSON file is sufficient for version 1. For any organization - even if we have 1000 cameras, it is still managed easily with JSON files. Later we can add support for some other database. For now let's stick with JSON

Hi, something about shoplifting detection ?

Support for following REST APIs is needed

Cameras
GET /cameras - get all cameras available
POST /cameras - add a camera
PUT /cameras/<camera_name> - Update a camera
DELETE /cameras/<camera_name> - Delete a camera
POST /cameras/<camera_name>/start-live-stream
POST /cameras/<camera_name>/stop-live-stream

Scenarios for cameras
GET /cameras/<camera_name>/scenarios - Get all scenarios for cameras
POST /cameras/<camera_name>/scenarios - Add a scenario to a camera
DELETE /cameras/<camera_name>/scenarios/<scenario_name> - Delete a scenario from camera

Model server start/stop
POST /model-server/start - Start all available models
POST /model-server/stop - Stop all available models
POST /model-server/restart - Restart model server
GET /model-server/status - Get different models running as well as server status

CV2 & Torch are very big packages

Including dependency on these two packages is causing the installation time go through the roof. This is also true during CI/CD tests when package needs to be built.

• Remove the dependency from these.
• Add documentation in our readme that user needs to install Docker, CV2 and Torch in advance.
• When we are using any of these libraries (torch, cv2 etc): then use the method from util/general.py that allows you to install a package on demand: check_requirements('torch', install=True). This would allow us to reduce the size of the package significantly and make our testing efforts easier.