This project demonstrates how to detect people in queues (in order to redirect them to shortest queue) using inference on pre-trained neural network with Intel OpenVINO framework.

The purpose of this project is to choose the right hardware suitable for a particular scenario. See Scenarios.md

The following pages will walk you through the steps of the project. At a high level, you will:

• Propose a possible hardware solution
• Build out your application and test its performance on the DevCloud using multiple hardware types
• Compare the performance to see which hardware performed best
• Revise your proposal based on the test results

• Scenario 1: Manufaturing

  • Hardware: FPGA

• Scenario 2: Retail Sector
  • Hardware: CPU and IGPU

• Scenario 3: Transport Sector
  • Hardware: VPU

The application was tested on a number of hardware using the Intel DevCloud and the results can be accessed here.

• This project makes the use of Intel DevCloud to test on CPU, GPU, FPGA and VPU so no specific hardware is required..

• Intel® Distribution of OpenVINO™ toolkit 2019 R3 release.
• Python > 3.5, 3.6

The Intel® DevCloud for the Edge is a cloud service designed to help developers prototype and experiment with computer vision applications using the Intel® Distribution of OpenVINO™ Toolkit. Once registered, developers can access a series of Python and C++ based Jupyter Notebook tutorials and sample solutions and execute them directly from a web browser. Then, developers can create their own Jupyter Notebooks and quickly try them out on a variety of hosted Intel® hardware solutions specifically designed for deep learning inferencing.

The Intel® DevCloud for the Edge provides you with access to everything needed to begin working with sample applications, prototypes and tutorials. This includes pre-trained models, source code, test input images, video and data streams. Additionally, users can apply any of the pre-trained deep learning models available through the Intel® Distribution of OpenVino™ toolkit, or upload their own customized pre-trained deep-learning models to develop and test their own computer vision applications.

Reduced time to access comprehensive Intel® development solutions, hardware and software, for deep learning and computer vision application development with just an internet connection. Access to fully configured physical edge machines pre-installed with the Intel® Distribution of OpenVINO™ Toolkit (CPU, iGPU, VPU and FPGA) hosted in the cloud powered by Intel® Xeon® Scalable processors. Ability to evaluate and choose the right Intel® hardware acceleration option for your application. A vast library of pre-trained models from the Intel® Distribution of OpenVINO™ Toolkit and ability to upload your own custom pre- trained models to evaluate the best framework, topology, and hardware acceleration solution for your unique application.

Install Intel® Distribution of OpenVINO™ toolkit Utilize the classroom workspace, or refer to the relevant instructions for your operating system for this step.

• Linux/Ubuntu
• Mac
• Windows

Much of the Intel® DevCloud for the Edge documentation can be accessed without registering. You will need to register for an Intel® DevCloud for the Edge account to explore, run the examples, upload your own code and test the hardware.

• On the Home page, click Sign in on the top right corner.
• Click Register and follow the prompts to enter the information requested.
• Within 48 hours you will receive an invitation email to your Intel® DevCloud for the Edge account.
• For increased security, the Intel® DevCloud for the Edge is protected by 2-factor authentication. Please check your email for the 6- digit security code. Copy/paste the full URL from that email containing the uuid argument into a browser window. All current web browsers are supported.
• Follow the prompts to complete your Intel DevCloud account registration.
• Once you have completed account registration, you can return any time to the Home page and click Sign in at the top right corner to access your account.
• Each time you sign in, the top right corner displays the total number of days you have access to the Intel® DevCloud for the Edge resource. You can request an extension from within the portal.

The figure below illustrates the user workflow for code development, job submission and viewing results.

• The person_detect.py file does the person counting part for you. Try to experiment around with the threshold value and see how the predictions turn up.

• The queue_job.sh is the utility which helps in the submission of the job to multiple devices on the DevCloud.

• The project includes 3 notebooks for different use cases-

• Retail Scenario

• Manufacturing Scenario

• Transportation Scenario

Each of the notebooks follow this process, be sure to change the original video location according to your system in each case.

We write a script to submit a job to an IEI Tank* 870-Q170 edge node with an Intel Core™ i5-6500TE processor. The inference workload should run on the CPU.

cpu_job_id = !qsub queue_job.sh -d . -l nodes=1:tank-870:i5-6500te -F "[model_path] CPU [original_video_path] /data/queue_param/manufacturing.npy [output_path] 2" -N store_core

We write a script to submit a job to an IEI Tank* 870-Q170 edge node with an Intel® Core i5-6500TE. The inference workload should run on the Intel® HD Graphics 530 integrated GPU.

gpu_job_id = !qsub queue_job.sh -d . -l nodes=tank-870:i5-6500te:intel-hd-530 -F "[model_path] GPU [original_video_path] /data/queue_param/manufacturing.npy [output_path] 2" -N store_core

We write a script to submit a job to an IEI Tank 870-Q170 edge node with an Intel Core i5-6500te CPU. The inference workload should run on an Intel Neural Compute Stick 2 installed in this node.

vpu_job_id = !qsub queue_job.sh -d . -l nodes=tank-870:i5-6500te:intel-ncs2 -F "[model_path] MYRIAD [original_video_path] /data/queue_param/manufacturing.npy [output_path] 2" -N store_core

We write a script to submit a job to an IEI Tank 870-Q170 edge node with an Intel Core™ i5-6500te CPU. The inference workload will run on the IEI Mustang-F100-A10 FPGA card installed in this node.

fpga_job_id = !qsub queue_job.sh -d . -l nodes=1:tank-870:i5-6500te:iei-mustang-f100-a10 -F "[model_path] HETERO:FPGA,CPU [original_video_path] /data/queue_param/manufacturing.npy [output_path] 2" -N store_core

We then compare performance on these devices on these 3 metrics-

• Frames Per Second
• Model Load Time
• Inference Time

The contents of this repository are covered under the MIT License