When I try run "dpu_mnist_classifier.ipynb" in "pynq_dpu/edge/notebooks", I got the following error message:

ModuleNotFoundError Traceback (most recent call last)
in ()
----> 1 from pynq_dpu import DpuOverlay
2 overlay = DpuOverlay("dpu.bit")
3 overlay.load_model("dpu_mnist_classifier.xmodel")

/usr/local/lib/python3.6/dist-packages/pynq_dpu-1.3.2-py3.6-linux-aarch64.egg/pynq_dpu/init.py in ()
14
15
---> 16 from .dpu import DpuOverlay
17
18

/usr/local/lib/python3.6/dist-packages/pynq_dpu-1.3.2-py3.6-linux-aarch64.egg/pynq_dpu/dpu.py in ()
24 from typing import List
25 import pynq
---> 26 import vart
27 import xir
28

ModuleNotFoundError: No module named 'vart'

I know this is because "vart" is not installed and I need install it. But after I operated followed Vitis AI UG and readme, this problem is still not solved. Very eager for your guidance!

Hello,
Is there any examples showing how to deploy the compiled models on the Ultra96 board. If not where can I find it? I have seen the zcu104/2 board examples using Vitis AI library and VART. If I want to do the same with Ultra96, is it possible?

Hello,

will Visit AI support PYNQ-Z2 Board? thanks.

Hi, all:
Is there anyone who met the following problem:

root@pynq:/home/xilinx# git clone --recursive --shallow-submodules https://github.com/Xilinx/DPU-PYNQ
Cloning into 'DPU-PYNQ'...
remote: Enumerating objects: 325, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 325 (delta 7), reused 26 (delta 5), pack-reused 285
Receiving objects: 100% (325/325), 3.55 MiB | 1.28 MiB/s, done.
Resolving deltas: 100% (143/143), done.
Submodule 'vitis-ai-git' (https://github.com/Xilinx/Vitis-AI.git) registered for path 'vitis-ai-git'
Cloning into '/home/xilinx/DPU-PYNQ/vitis-ai-git'...
remote: Enumerating objects: 26536, done.
remote: Counting objects: 100% (26536/26536), done.
remote: Compressing objects: 100% (18395/18395), done.
remote: Total 26536 (delta 7271), reused 25480 (delta 7055), pack-reused 0
Receiving objects: 100% (26536/26536), 563.72 MiB | 2.06 MiB/s, done.
Resolving deltas: 100% (7271/7271), done.
fatal: unable to access 'https://github.com/Xilinx/Vitis-AI.git/': Empty reply from server
Unable to fetch in submodule path 'vitis-ai-git'
root@pynq:/home/xilinx# cd DPU-PYNQ/upgrade
root@pynq:/home/xilinx/DPU-PYNQ/upgrade# make
bash ./check_env.sh
Error: please make sure you have valid Internet connection.
Makefile:7: recipe for target 'check_env' failed
make: *** [check_env] Error 1

After successfully running the MNIST classifier example on the ZCU104 board, I wanted to try running a hybrid CNN-SVM model
using the extracted features of all the test samples ( the output features of the MNIST classifier example) to be used as training and test examples for the SVM classifier using PYNQ-DPU
as shown in the code below.
However, the obtained accuracy of this model was about 34.34.% compared to CNN model provided in the MNIST classifier example (98.61%)

from time import time
import sklearn
import numpy as np
import mnist
import cv2
from sklearn import svm
import matplotlib.pyplot as plt
%matplotlib inline
from six.moves import urllib
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)
from pynq_dpu import DpuOverlay
overlay = DpuOverlay("dpu.bit")
overlay.load_model("dpu_mnist_classifier.xmodel")
raw_data = mnist.test_images()
normalized_data = np.asarray(raw_data/255, dtype=np.float32)
test_data = np.expand_dims(normalized_data, axis=3)
test_label = mnist.test_labels()
dpu = overlay.runner

inputTensors = dpu.get_input_tensors()
outputTensors = dpu.get_output_tensors()

shapeIn = tuple(inputTensors[0].dims)
shapeOut = tuple(outputTensors[0].dims)
outputSize = int(outputTensors[0].get_data_size() / shapeIn[0])
output_data = [np.empty(shapeOut, dtype=np.float32, order="C")]
input_data = [np.empty(shapeIn, dtype=np.float32, order="C")]
image = input_data[0]
total = test_data.shape[0]
array2 = np.empty_like (output_data)
print("Classifying {} digit pictures ...",(total))
for i in range(total):
    image[0,...] = test_data[i]
    job_id = dpu.execute_async(input_data, output_data)
    dpu.wait(job_id)
     array2 = np.vstack((array2, output_data))

array2 = array2.reshape(array2.shape[0], -1)
exTrain = array2[:9900]
y_train = test_label[:9900]
exTest = array2[9901:10000]
y_test = test_label[9901:10000]
from sklearn import svm
clf = svm.SVC(kernel='linear') # Linear Kernel
clf.fit(exTrain, y_train)
predictions = clf.predict(exTest)

I don't get what's the reason for this bad accuracy. Any help please?
Thanks

Hello,

Are there any plans to support models compiled by Vitis AI 2.0 Compiler?

Thank you!

I managed to run multiple subgraphs on DPU by changing the dpu.py code. However, is there anyway to run the subgraphs on CPU?

Hello
I could see that vitis ai 1.3 was available and I saw in an issue that it was a constraint to have pynq-DPU which would be compatible with the Z2. Can you estimate the release date of a version for the pynq Z2?

I follow the instructions with the pre-build HW (dpu.bit dpu.hwh, dpu.xclbin) on ZCU104; it implements successfully.

And I follow the Rebuild instructions to generate my own HW (dpu.bit dpu.hwh, dpu.xclbin), while implement on the example, it shows the "Kernel Restarting The kernel appears to have died. It will restart automatically."

Can anyone tell me what the problem is?

Hi,

After successfully building a single core DPU for ZCU104, I manually copied DPU-PYNQ directory to my board I ran this command:

pip3 install /home/xilinx/DPU-PYNQ --no-build-isolation

I opened a notebook, created a overlay object to load the dpu.bit, and investigate its properties by overlay? command. It showed:

Type:            DpuOverlay
String form:     <pynq_dpu.dpu.DpuOverlay object at 0xffff659435b0>
File:            /usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq_dpu/dpu.py
Docstring:      
Default documentation for overlay /usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq_dpu/dpu.bit. The following
attributes are available on this overlay:

IP Blocks
----------
axi_vip_2            : pynq.overlay.DefaultIP
DPUCZDX8G_1          : pynq.overlay.DefaultIP
DPUCZDX8G_2          : pynq.overlay.DefaultIP
axi_vip_1            : pynq.overlay.DefaultIP
ps_e_0               : pynq.overlay.DefaultIP

So I went to the Jupyter terminal and manually copied the required files:

cd /home/xilinx/DPU-PYNQ/boards/ZCU104
cp dpu.bit dpu.hwh ../vitis_platform/dpu/dpu.tcl binary_container_1/dpu.xclbin /usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq_dpu

I restarted the resnet50 again, ran overlay? command again and it showed:

Type:            DpuOverlay
String form:     <pynq_dpu.dpu.DpuOverlay object at 0xffff659434f0>
File:            /usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq_dpu/dpu.py
Docstring:      
Default documentation for overlay /usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq_dpu/dpu.bit. The following
attributes are available on this overlay:

IP Blocks
----------
axi_vip_2            : pynq.overlay.DefaultIP
DPUCZDX8G_1          : pynq.overlay.DefaultIP
axi_vip_1            : pynq.overlay.DefaultIP
ps_e_0               : pynq.overlay.DefaultIP

My question is how can we support to looking for the local bitstream files before getting online prebuilt ones?

Thank you in advance.

I tried dpu_mnist_classifier.ipynb and it worked well.
But dpu_mnist_classifier.ipynb with keras.datasets.mnist ( original ipynb used mnist ) had poor accuracy. All prediction results were '1'.

Does anyone have any idea?

I am facing a lot of time delay in installing pynq dpu . Is it normal or not? I was able to successfully build the pynq upgrade. Please let me know.

Hello. I am not clear about the model format.
In the "host" folder, the target is to convert the model into .xmodel file. However, in the "pynq_dpu" folder, the examples use ".elf" model as the input.
I wonder what's the relationship between them, and how to convert?

Hi,

When working with dpu_inception_v1.ipynb I have seen that !dexplorer -w has the next output:

[DPU IP Spec]
IP  Timestamp            : 2020-03-26 13:30:00
DPU Core Count           : 1

[DPU Core Configuration List]
DPU Core                 : #0
DPU Enabled              : Yes
DPU Arch                 : B1600
DPU Target Version       : v1.4.1
DPU Freqency             : 300 MHz
Ram Usage                : Low
DepthwiseConv            : Enabled
DepthwiseConv+Relu6      : Enabled
Conv+Leakyrelu           : Enabled
Conv+Relu6               : Enabled
Channel Augmentation     : Disabled
Average Pool             : Enabled

I am a little bit confuse about this output because I have created another dpu design with other DPU frequencies and another architectures and when I tried dexplorer after loading the overlay, all the time the output is the same, it doesn't change.

Is this dexplorer output hardcoded?

Also I was having reboot problems when using 300/600 MHz frequency designs, that I am not having when I use the default design so how it is possible to be DPU Freqency equal to 300MHz?

PYNQ V2.5 on Ultra96V2. Follow the steps in README. In last step to build up notebook.

pynq get-notebooks pynq-dpu -p .

root@pynq:/home/xilinx/jupyter_notebooks# pynq get-notebooks pynq-dpu -p .
Delivering notebooks './pynq-dpu'...
Traceback (most recent call last):
File "/usr/local/bin/pynq-get-notebooks", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/pynq/_cli/get_notebooks.py", line 240, in main
raise e
File "/usr/local/lib/python3.6/dist-packages/pynq/_cli/get_notebooks.py", line 209, in main
ext_mod = ext.load()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2324, in load
return self.resolve()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2330, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/usr/local/lib/python3.6/dist-packages/pynq_dpu/init.py", line 17, in
from .dpu import DpuOverlay
File "/usr/local/lib/python3.6/dist-packages/pynq_dpu/dpu.py", line 20, in
import runner
ModuleNotFoundError: No module named 'runner'

There is no module named 'runner' on pypi . Only have one module named "Runner"

Hello,
I was trying to run a custom CNN model using PYNQ-DPU on ZCU104 board following this example. And I was trying to retrieve the data from the layer before Softmax.
As far as I know, the output data here is the output data of the layer before Softmax , which is the dense layer in the attached example of a model architecture. The shape of the output data will be (1,128). Am I correct ???
job_id = dpu.execute_async(input_data, output_data) dpu.wait(job_id) temp = [j.reshape(1, outputSize) for j in output_data] softmax = calculate_softmax(temp[0][0])

When I tried to reshape the output data to be (1,128), there was NaN in the data.

So I tried this:
After the training of the full model using tensorflow, I run vai_q_tensorflow to obtain a .pb quantized model. I truncate this model to delete softmax layer. The truncated model (conv+pool+dense1) is compiled using

!vai_c_tensorflow \
    --frozen_pb custom_dense1.pb \
    --arch /opt/vitis_ai/compiler/arch/DPUCZDX8G/ZCU104/arch.json \
    --output_dir . \
    --net_name mnist_dense1_modif

However I got the following error

• VITIS_AI Compilation - Xilinx Inc.

[INFO] Namespace(inputs_shape=None, layout='NHWC', model_files=['custom_dense1.pb'], model_type='tensorflow', out_filename='./mnist_dense1_modif_org.xmodel', proto=None)
[INFO] tensorflow model: custom_dense1.pb
[INFO] parse raw model : 8%| | 1/12 [00:00<00:00, 3675.99it/s]
Traceback (most recent call last):
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/bin/xnnc-run", line 33, in
sys.exit(load_entry_point('xnnc==1.3.0', 'console_scripts', 'xnnc-run')())
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/main.py", line 194, in main
normal_run(args)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/main.py", line 178, in normal_run
in_shapes=in_shapes if len(in_shapes) > 0 else None,
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/xconverter.py", line 131, in run
xmodel = CORE.make_xmodel(model_files, model_type, _layout, in_shapes)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/core.py", line 104, in make_xmodel
model_files, layout, in_shapes=in_shapes, model_type=model_t
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 97, in to_xmodel
model_name, raw_nodes, layout, in_shapes, model_fmt, model_type
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 161, in create_xmodel
xmodel = cls.__create_xmodel_from_tf1(name, layers, layout, in_shapes)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 243, in __create_xmodel_from_tf1
xmodel_name, layout, layers, const_layer_dict, super_const_dict, in_shapes
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 1847, in __generate_xmodel
), f"[ERROR] TF Conv2d requires two inputs: actual: {bottom}."
AssertionError: [ERROR] TF Conv2d requires two inputs: actual: ['input_data'].

Any suggestions how to solve this problem please?
Thanks.

Loading Pytorch and Tensorflow xmodels from Model Zoo don't work. Only loading caffe xmodels seem to work.

After setting up the Vitis AI CPU docker container, I ran it from host/ using ./docker_run.sh xilinx/vitis-ai-cpu:latest following the guide in host/README.md

In the container I ran the compile.sh once, so it copies the board files, then the other commands to start the notebook

conda activate vitis-ai-tensorflow
./compile.sh Ultra96 tf_yolov3_voc_416_416_65.63G_1.3
yes | pip install matplotlib keras==2.2.5
jupyter notebook --ip=0.0.0.0 --port=8080

I had to slightly modify the compile.sh to run where the MODEL_NAME is referenced with normal braces instead of curly braces.

I also tried to manually copy the board files and had the same result.

The notebook mostly runs fine, there are some warnings about deprecated functions, but everything works until the very last part. I modified it slightly to point to the Ultra96 board images instead of the default.

!vai_c_tensorflow \
    --frozen_pb quantized/deploy_model.pb \
    --arch /opt/vitis_ai/compiler/arch/DPUCZDX8G/Ultra96/Ultra96.json \
    --output_dir . \
    --net_name mnist_classifier

result:

**************************************************
* VITIS_AI Compilation - Xilinx Inc.
**************************************************
[INFO] Namespace(inputs_shape=None, layout='NHWC', model_files=['quantized/deploy_model.pb'], model_type='tensorflow', out_filename='./mnist_classifier_org.xmodel', proto=None)
[INFO] tensorflow model: quantized/deploy_model.pb
[INFO] parse raw model     : 11%| | 1/9 [00:00<00:00, 5857.97it/s]              
Traceback (most recent call last):
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/bin/xnnc-run", line 33, in <module>
    sys.exit(load_entry_point('xnnc==1.3.0', 'console_scripts', 'xnnc-run')())
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/__main__.py", line 194, in main
    normal_run(args)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/__main__.py", line 178, in normal_run
    in_shapes=in_shapes if len(in_shapes) > 0 else None,
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/xconverter.py", line 131, in run
    xmodel = CORE.make_xmodel(model_files, model_type, _layout, in_shapes)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/core.py", line 104, in make_xmodel
    model_files, layout, in_shapes=in_shapes, model_type=model_t
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 97, in to_xmodel
    model_name, raw_nodes, layout, in_shapes, model_fmt, model_type
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 161, in create_xmodel
    xmodel = cls.__create_xmodel_from_tf1(name, layers, layout, in_shapes)
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 243, in __create_xmodel_from_tf1
    xmodel_name, layout, layers, const_layer_dict, super_const_dict, in_shapes
  File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/xnnc/translator/tensorflow_translator.py", line 1847, in __generate_xmodel
    ), f"[ERROR] TF Conv2d requires two inputs: actual: {bottom}."
AssertionError: [ERROR] TF Conv2d requires two inputs: actual: ['input_data'].

Any ideas what's causing the error? Anything I can do to mitigate it?

When will PYNQ-Z2 be supported? Thanks.

Hi, I would like to know the possibilities of using Vitis AI library with the DPU-PYNQ package say for Ultra96? Is it possible by any means? Thanks in advance.

Hello,

I want to create a custom hardware design because I need some GPIO. I wanted to create the design with this tutorial and was wondering if I can just build the hardware like that and use it for pynq ?

Greetings

Hello. Thanks for your work. I am a newcomer of DPU and Vitis-AI, and I feel confused about the procedure for deployment.

You mentioned that "we first need to make sure Vitis and XRT have been installed and their settings are sourced properly."

source /Vitis/2021.1/settings64.sh
source /xilinx/xrt/setup.sh

I wonder are there any references related to them? I could not find the corresponding repo or files. And how to check whether Vitis and XRT are installed?

Thank you.

Hi, I am currently working with DPU-PYNQ on ZCU104. I am working with a project of Face Tracking based on this project avnet_face_tracking but I have a problem with the line "job_id = dpu.execute_async( inputData, outputData )" (inside the function "faces = dpu_face_detector.process(frame)"). When it reaches this part of the code, the kernel stops working, like if it was in an infinite loop, but it doesnt show any error.
Can you tell me what the dpu.execute_async() function is expecting to do? I will send you my code:
``
import sys
if '/usr/lib/python3.6/site-packages' not in sys.path:
sys.path.append('/usr/lib/python3.6/site-packages')

import os
import subprocess
from ctypes import *
from typing import List
import pynq
import vart
import xir
from ctypes import *
from typing import List
import cv2
import numpy as np
import pathlib
import math
import threading
import time
import sys
import argparse

from imutils.video import FPS
from pyimagesearch.centroidtracker import CentroidTracker

from vitis_ai_vart.facedetect import FaceDetect
from vitis_ai_vart.utils import get_child_subgraph_dpu

from pynq_dpu import DpuOverlay
from matplotlib import pyplot as plt # show image

construct the argument parse and parse the arguments

inputId = 0
detThreshold = 0.55
nmsThreshold = 0.35

densebox_xmodel = "densebox_640_360.xmodel"
overlay = DpuOverlay("dpu.bit")
print("[INFO] dpu overlay loaded")
overlay.load_model(densebox_xmodel)

Initialize Vitis-AI/DPU based face detector

densebox_dpu = overlay.runner
dpu_face_detector = FaceDetect(densebox_dpu,detThreshold,nmsThreshold) # Check face detect function
dpu_face_detector.start()

Initialize our centroid tracker and frame dimensions

ct = CentroidTracker()
print("[INFO] Begin dpu process")

Initialize the camera input

print("[INFO] starting camera input ...")
cam = cv2.VideoCapture(inputId)
cam.set(cv2.CAP_PROP_FRAME_WIDTH,640)
cam.set(cv2.CAP_PROP_FRAME_HEIGHT,480)
print("Capture device is open: " + str(cam.isOpened()))
if not (cam.isOpened()):
print("[ERROR] Failed to open camera ", inputId )
exit()

START TRUE LOOP

Capture image from camera

ret,frame = cam.read()

Show image inside notebook using Matplotlib

plt.imshow(frame[:,:,[2,1,0]])
plt.show()

Vitis-AI/DPU based face detector

faces = dpu_face_detector.process(frame) #---------------THIS LINE IS NOT WORKING`

Inside this dpu_face_detector.process(frame) function you can find this:
`def process(self,img):
#print("[INFO] facefeature process")

dpu = self.dpu
#print("[INFO] facefeature runner=",dpu)

inputChannels = self.inputChannels
inputHeight = self.inputHeight
inputWidth = self.inputWidth
inputShape = self.inputShape
output0Channels = self.output0Channels
output0Height = self.output0Height
output0Width = self.output0Width
output0Size = self.output0Size
output0Shape = self.output0Shape
output1Channels = self.output1Channels
output1Height = self.output1Height
output1Width = self.output1Width
output1Size = self.output1Size
output1Shape = self.output1Shape

imgHeight = img.shape[0]
imgWidth  = img.shape[1]
scale_h = imgHeight / inputHeight
scale_w = imgWidth / inputWidth

""" Image pre-processing """
#print("[INFO] process - pre-processing - normalize ")
# normalize
img = img - 128.0
#print("[INFO] process - pre-processing - resize ")
# resize
img = cv2.resize(img,(inputWidth,inputHeight))

""" Prepare input/output buffers """
#print("[INFO] process - prep input buffer ")
inputData = []
inputData.append(np.empty((inputShape),dtype=np.float32,order='C'))
inputImage = inputData[0]
inputImage[0,...] = img

#print("[INFO] process - prep output buffer ")
outputData = []
outputData.append(np.empty((output0Shape),dtype=np.float32,order='C'))
outputData.append(np.empty((output1Shape),dtype=np.float32,order='C'))

""" Execute model on DPU """
#print("[INFO] process - execute ")
job_id = dpu.execute_async( inputData, outputData )                                                 #-----------THIS LINE IS NOT WORKING
dpu.wait(job_id)

""" Retrieve output results """    
#print("[INFO] process - get outputs ")
OutputData0 = outputData[0].reshape(1,output0Size)
bboxes = np.reshape( OutputData0, (-1, 4) )
#
outputData1 = outputData[1].reshape(1,output1Size)
scores = np.reshape( outputData1, (-1, 2))

""" Get original face boxes """
gy = np.arange(0,output0Height)
gx = np.arange(0,output0Width)
[x,y] = np.meshgrid(gx,gy)
x = x.ravel()*4
y = y.ravel()*4
bboxes[:,0] = bboxes[:,0] + x
bboxes[:,1] = bboxes[:,1] + y
bboxes[:,2] = bboxes[:,2] + x
bboxes[:,3] = bboxes[:,3] + y

""" Run softmax """
softmax = softmax_2( scores )

""" Only keep faces for which prob is above detection threshold """
prob = softmax[:,1] 
keep_idx = prob.ravel() > self.detThreshold
bboxes = bboxes[ keep_idx, : ]
bboxes = np.array( bboxes, dtype=np.float32 )
prob = prob[ keep_idx ]

""" Perform Non-Maxima Suppression """
face_indices = []
if ( len(bboxes) > 0 ):
    face_indices = nms_boxes( bboxes, prob, self.nmsThreshold );

faces = bboxes[face_indices]

# extract bounding box for each face
for i, face in enumerate(faces):
    xmin = max(face[0] * scale_w, 0 )
    ymin = max(face[1] * scale_h, 0 )
    xmax = min(face[2] * scale_w, imgWidth )
    ymax = min(face[3] * scale_h, imgHeight )
    faces[i] = ( int(xmin),int(ymin),int(xmax),int(ymax) )

return faces`

Hello,

When trying the example of Training a CNN for DPU compilation, I wanted to save and load the trained model using:

from keras.models import load_model
model.save('float_model.h5') 
model = keras.models.load_model('float_model.h5')

However, I got the following error:
ValueError: You are trying to load a weight file containing 5 layers into a model with 0 layers
Also, after the quantization of this trained model I tried to convert the quantized model (.pb) to (.h5) model using:

from tensorflow.keras.models import save_model, Sequential
model_path = './quantized_modif/quantize_eval_model'
model = tf.keras.models.load_model(model_path)
save_model(model,model_path + r"\quantized_model.h5", save_format='h5')

I got the following error: OSError: SavedModel file does not exist at: ./quantized_modif/quantize_eval_model/{saved_model.pbtxt|saved_model.pb}

Any suggestions how to fix this please?

Hello,
Is the model used in this example came from the trained model on this notebook https://github.com/Xilinx/DPU-PYNQ/blob/master/host/train_mnist_model.ipynb?
I wanted to know the architecture of the used model ("dpu_mnist_classifier.xmodel").

Thanks!

Hi, I'm very interrested on using DPU on the PYNQ-Z2 plateform and according to xilinx the DPU v3.3 is supported on zynq-7000 series (https://www.xilinx.com/html_docs/vitis_ai/1_3/dpu_intro.html#gub1565042860537). Can anyone tell me how to modify the sources to get it working with this board ?

When I was trying to the installation of DPU_PYNQ on ZCU104, I got the following error :

fatal: unable to access 'https://github.com/Xilinx/DPU-PYNQ.git/': Could not resolve host: github.com
I tried :
root@pynq:/home/xilinx# git config --system --get https.proxy
root@pynq:/home/xilinx# git config --system --get http.proxy
root@pynq:/home/xilinx# git config --global --get https.proxy
Also
git config --global --unset http.proxy
And restarting the terminal.
However the problem persists, any help please?

Hi @yunqu ,

First of all thank you for your effort supporting DPU on Pynq boards it is an incredible work!

I've been able to run different dpu notebooks but now when I try to load models using VART and libvart-dpu-runner.so like in this example, resnet50_mt_py, I am having problems when running import runner. It says that:

ModuleNotFoundError: No module named 'runner'

I've been digging and I have seen that runner.so is installed in /usr/lib/python3.5/site-packages/runner.so but notebooks use Python3.6. Could this runner.so be installed/supported in Python 3.6?

Best regards,
Jorge.

Hello,

the ULTRA96 json file references:

"target"   : "DPUCZDX8G",
"dcf"      : "/opt/vitis_ai/compiler/arch/DPUCZDX8G/Ultra96/Ultra96.dcf",
"cpu_arch" : "arm64"

alas Vitis 1.2 does not have an ULTRA96 directory in /opt/vitis_ai/compiler/arch/DPUCZDX8G/
it appears necessary to not only post the pynqdpu.dpu.ultra96.hwh file but the dcf as well ?

Thank you for your support, Gerd

Hello!
I'm trying to compare performance between an Ultra96-V2 board and a desktop GPU running the same network and model.
To achieve that on the board, I am trying to compile an item from the Vitis AI model zoo to an *.elf file for the DPU.

I followed the guide in host/README.md up until the point where I am supposed to run compile.sh in the docker container:

./compile.sh Ultra96 dk_yolov4_coco_416_416_60.1G_1.3

The first error I got was ./compile.sh: line 20: MODEL_NAME: command not found. Then I opened the file with vim and found that on line 20 and 21 () was used instead of {}. I corrected that and the script progressed.

The next issue was Error: currently only caffe and tensorflow are supported. so I tried the suggested example instead:

> ./compile.sh Ultra96 cf_resnet50_imagenet_224_224_7.7G_1.1
[DLet]Generate DPU DCF file dpu-03-26-2020-13-30.dcf successfully.
Archive:  cf_resnet50_imagenet_224_224_7.7G_1.1.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of cf_resnet50_imagenet_224_224_7.7G_1.1.zip or
        cf_resnet50_imagenet_224_224_7.7G_1.1.zip.zip, and cannot find cf_resnet50_imagenet_224_224_7.7G_1.1.zip.ZIP, period.

So I decided to try any tensorflow model from the zoo:

> ./compile.sh Ultra96 tf_yolov3_voc_416_416_65.63G_1.3
[DLet]Generate DPU DCF file dpu-03-26-2020-13-30.dcf successfully.
Archive:  tf_yolov3_voc_416_416_65.63G_1.3.zip
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/quantize/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/quantize/config.ini.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/quantize/evaluate_quantize_model.sh.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/quantize/input_fn.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/quantize/quantize.sh.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/quantize/readme.md.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/config.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/dataset_tools/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/dataset_tools/convert_voc_anno.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/dataset_tools/prepare_data.sh.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/evaluation.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/run_eval.sh.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/tf_prediction.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/tf_prediction_pil.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/code/test/yolo3_predictor.py.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/data/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/data/calib_list.txt.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/data/demo_list.txt.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/float/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/float/yolov3_voc.pb.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/quantized/.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/quantized/deploy_model.pb.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/quantized/quantize_eval_model.pb.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/README.md.
checkdir error:  tf_yolov3_voc_416_416_65.63G_1.3 exists but is not directory
                 unable to process tf_yolov3_voc_416_416_65.63G_1.3/requirements.txt.

So I decided to try a different model and got this:

> ./compile.sh Ultra96 cf_ssdpedestrian_coco_360_640_0.97_5.9G_1.3
...
**************************************************
* VITIS_AI Compilation - Xilinx Inc.
**************************************************
[INFO] Namespace(inputs_shape=None, layout='NCHW', model_files=['cf_ssdpedestrian_coco_360_640_0.97_5.9G_1.3/quantized/deploy.caffemodel'], model_type='caffe', out_filename='./ssdpedestrian_org.xmodel', proto='cf_ssdpedestrian_coco_360_640_0.97_5.9G_1.3/quantized/deploy.prototxt')
[INFO] caffe model: cf_ssdpedestrian_coco_360_640_0.97_5.9G_1.3/quantized/deploy.caffemodel
[INFO] caffe model: cf_ssdpedestrian_coco_360_640_0.97_5.9G_1.3/quantized/deploy.prototxt
[INFO] parse raw model     :100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 123/123 [00:00<00:00, 1823.06it/s]
[INFO] infer shape (NCHW)  :100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 123/123 [00:00<00:00, 17733.99it/s]
[INFO] infer shape (NHWC)  :100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 123/123 [00:00<00:00, 17090.12it/s]
[INFO] generate xmodel     :100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 123/123 [00:00<00:00, 11883.80it/s]
[INFO] generate xmodel: /workspace/ssdpedestrian_org.xmodel
[UNILOG][INFO] The compiler log will be dumped at "/tmp/vitis-ai-user/log/xcompiler-20210121-075318-141"
[UNILOG][INFO] Target architecture: DPUCZDX8G
[UNILOG][FATAL][TARGET_FACTORY_UNREGISTERED_TARGET][Unregistered target!] Cannot find target with name DPUCZDX8G, valid names are: {DPUCAHX8H_ISA2=>0x20200000000002a,DPUCAHX8H_ISA2_ELP2=>0x20200000000002e,DPUCAHX8L_ISA0=>0x30000000000001d,DPUCVDX8G_ISA0_B16384C64B1=>0x600000076080812,DPUCVDX8G_ISA0_B8192C32B1=>0x600000076080811,DPUCVDX8G_ISA0_B8192C32B1_ELP4=>0x600000076040411,DPUCVDX8G_ISA0_B8192C32B3=>0x600000076080831,DPUCVDX8G_ISA0_B8192C32B3_DW=>0x6000000f6088831,DPUCVDX8G_ISA0_B8192C32B3_I4W8B2=>0x600000276080831,DPUCVDX8G_ISA0_B8192C32B3_I8W4B2=>0x600000376080831,DPUCVDX8G_ISA0_B8192C32B3_I8W8B2=>0x600000176080831,DPUCVDX8H_ISA0=>0x5000000000007ee,DPUCZDI4G_ISA0_B4096_DEMO_SSD=>0x400002003220206,DPUCZDI4G_ISA0_B8192D8_DEMO_SSD=>0x400002003220207,DPUCZDX8G_ISA0_B1024_MAX=>0x1000020f7014402,DPUCZDX8G_ISA0_B1024_MIN=>0x100002022010102,DPUCZDX8G_ISA0_B1152_MAX=>0x1000020f7012203,DPUCZDX8G_ISA0_B1152_MIN=>0x100002022010103,DPUCZDX8G_ISA0_B1600_MAX=>0x1000020f7014404,DPUCZDX8G_ISA0_B1600_MIN=>0x100002022010104,DPUCZDX8G_ISA0_B2304_MAX=>0x1000020f7014405,DPUCZDX8G_ISA0_B2304_MAX_BG2=>0x1000020f6014405,DPUCZDX8G_ISA0_B2304_MIN=>0x100002022010105,DPUCZDX8G_ISA0_B3136_MAX=>0x1000020f7014406,DPUCZDX8G_ISA0_B3136_MAX_BG2=>0x1000020f6014406,DPUCZDX8G_ISA0_B3136_MIN=>0x100002022010106,DPUCZDX8G_ISA0_B4096_MAX=>0x1000020f7014407,DPUCZDX8G_ISA0_B4096_MAX_BG2=>0x1000020f6014407,DPUCZDX8G_ISA0_B4096_MAX_EM=>0x1000030f7014407,DPUCZDX8G_ISA0_B4096_MIN=>0x100002022010107,DPUCZDX8G_ISA0_B512_MAX=>0x1000020f7012200,DPUCZDX8G_ISA0_B512_MIN=>0x100002022010100,DPUCZDX8G_ISA0_B800_MAX=>0x1000020f7012201,DPUCZDX8G_ISA0_B800_MIN=>0x100002022010101}
*** Check failure stack trace: ***
This program has crashed!
Aborted

So I guess my question is am I doing something wrong? Would I have different results with the GPU image or it's unrelated? How could I generate the .elf files to use with the DPU?

Hello,

I've been trying to run the tf2_ssdincepion_v2 model using dpu-pynq on an Ultra96-V2 using a modified version of the dpu_tf_inceptionv1.ipynb file.

By using the defaults of shapeOut = tuple(outputTensors[0].dims) I was able to get an output array of shape (1,1917,91) which I assume is the confidence score for each box.

If I access the second output tensor shapeOut2 = tuple(outputTensors[1].dims) it will give me a shape of (1,1917,4) which I assume will be the box locations.

However while trying to run with the second set of outputensors, the following error will be thrown:

 job_id = dpu.execute_async(input_data, output_data2)
double free or corruption (out)
Aborted (core dumped)

I am not sure how to proceed with doing the detection as from the graph generated using analyze_subgraphs.sh the subgraph has 2 outputs that require CPU processing to obtain the final result.

Any help will be appreciated. Thank you!

Graph Image

Hello, if I want to use the tf_yolov3_voc model in the Vitis AI Model Zoo to deploy and train on the PYNQ-Z2 board, should I follow this tutorial step by step, right? Does train_mnist_model.ipynb apply to my own custom dataset? For example, if I want to train the tf_yolov3_voc and detect the smoke on PYNQ-Z2.

I read in UG1414 about debugging ncube:

_It involves the following five steps.

1. Run Vitis AI quantizer to generate the golden baseline from the quantized model.
2. Build the model as debug mode DPU kernel by Vitis AI compiler with option --dump
   fused_graph_info specified.
3. Before launching the running of DPU application, run command dexplorer -m debug to
   switch runtime N2Cube into debug mode, or calling dpuEnableTaskDebug() to enable
   debug mode for the dedicated DPU task only (other tasks will not be affected).
4. Run the DPU application and get the raw dump data for DPU task’s each node.
5. Compare DPU raw dump data with the golden baseline from quantizer_

As it turns out the kernel dies even if there is just a typo in the nodes for ncube.
It would be really good to get access to that feature.

I tried to at the line:
!dexplorer -m
in the nodebook, but that seems to spawn its own shell and does not affect the notebook.

I tried
n2cube.dpuSetInputTensorInHWCFP32(task, KERNEL_CONV_INPUT, test_data[j], input_len)
n2cube.dpuEnableTaskDebug(task)
n2cube.dpuRunTask(task)

again no effect, would you please so kind to point out how to get maximum debugging into from ncube ?

Thank you so much for your help, Gerd

I found that DPU-PYNQ1.2 has notebook examples of YOLOV3, but it uses .elf files. Are there any notebook examples of YOLOV3 or V4 in DPU-PYNQ1.3 by using .xmodel files?

Hi for some reason I have to use PYNQ 2.5 image, Will this upgrade work for version 2.5 too?

Hi,
I'm using dpu-pynq v1.1.2 for pynq 2.5 image. I want to make cutom DPU as suggested here https://github.com/Xilinx/DPU-PYNQ/tree/v1.1.2/boards
but unfortunately when i run the make command for Ultra96 v2, it tries to clone this repository https://github.com/yunqu/PYNQ-derivative-overlays.git and this has been moved.
Can you please give me new path of this repository for pynq-dpu v1.1.2 ?

Hi I have followed the readme.md of DPU-PYNQ
Here are my status. I used PYNQ2.6 image for Ultra96v2.

1. [step 1 -- succeed] After successfully finishing step 1 of the readme.md of DPU-PYNQ to upgrade the DPU-PYNQ for PYNQ2.6
2. [step 2 -- succeed] And I use wget to download the released .bit, .hwh, .xclbin, and .xmodel files for ultra96.
3. [step 3 -- kernel died in model_load] When I open the dpu_resnet50.ipynb in jupyter_note and change the .xmodel and .bit files into correct name. And click on run button. The kernel died. Might I ask how to fix that?

Here is the overlay info

IP Blocks
----------
axi_vip_2            : pynq.overlay.DefaultIP
DPUCZDX8G_1          : pynq.overlay.DefaultIP
axi_vip_1            : pynq.overlay.DefaultIP
ps_e_0               : pynq.overlay.DefaultIP

Here are the files that I used.

1. .model

"Ultra96": {
        "url": "https://www.xilinx.com/bin/public/openDownload?filename=pynqdpu.dpu_resnet50.ultra96.1.3.0.xmodel",
        "md5sum": "489ed4dc13e755d82e3a4cf87bf6866e"
    },

2. dpu.bit

"Ultra96": {
        "url": "https://www.xilinx.com/bin/public/openDownload?filename=pynqdpu.dpu.ultra96.1.3.0.bit",
        "md5sum": "20f59c652cab14cbf75d308be7f2dd79"
    }

3. dpu.hwh

"Ultra96": {
        "url": "https://www.xilinx.com/bin/public/openDownload?filename=pynqdpu.dpu.ultra96.1.3.0.hwh",
        "md5sum": "5e680bb06d92450d44701ec355e68136"
    }

4. dpu.xclbin

	"Ultra96": {
        "url": "https://www.xilinx.com/bin/public/openDownload?filename=pynqdpu.dpu.ultra96.1.3.0.xclbin",
        "md5sum": "c1c9069ab68b7826b77538407e72061f"
    },

Any instruction is appreciated!
Thanks in advance!
JT

I have a notebook using the DPU. When I run this notebook from the main Jupyter instance it works fine.
For easier development I'm trying to use VS Code over SSH remote with the Jupyter extension, which creates a separate Jupyter server and and displays the notebook within the IDE.
When I run the same notebook that way, the DpuOverlay cannot find the dpu bitstream.
Here's the exception message:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-5-d7286a9e2014> in <module>()
----> 1 overlay = DpuOverlay("dpu.bit")
      2 overlay.load_model("dpu_tf_inceptionv1.xmodel")

/home/xilinx/.local/lib/python3.6/site-packages/pynq_dpu/dpu.py in __init__(self, bitfile_name, dtbo, download, ignore_version, device)
     78             abs_bitfile_name = os.path.join(OVERLAY_PATH, bitfile_name)
     79         else:
---> 80             raise FileNotFoundError('Cannot find {}.'.format(bitfile_name))
     81         super().__init__(abs_bitfile_name,
     82                          dtbo=dtbo,

FileNotFoundError: Cannot find dpu.bit.

Any ideas what could be the problem?

Hello,

I was working with this notebook as a template to understand the workflow,
alas as of today it does not run anymore. I am running:

./docker_run.sh xilinx/vitis-ai:1.2.82

and within

pip install keras 2.2.5

First indication is that history.history does not have 'acc' anymore but 'accuracy' instead,
so the plots fail.

That seems to be a feature of keras 2.3.x ?
According to the 2.3.0 Release Notes:
"Metrics and losses are now reported under the exact name specified by the user (e.g. if you pass metrics=['acc'], your metric will be reported under the string "acc", not "accuracy", and inversely metrics=['accuracy'] will be reported under the string "accuracy"."

But most importantly I get for the freeze:
Traceback (most recent call last):
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/bin/freeze_graph", line 10, in
sys.exit(run_main())
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/tools/freeze_graph.py", line 487, in run_main
app.run(main=my_main, argv=[sys.argv[0]] + unparsed)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/tools/freeze_graph.py", line 486, in
my_main = lambda unused_args: main(unused_args, flags)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/tools/freeze_graph.py", line 378, in main
flags.saved_model_tags, checkpoint_version)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/tools/freeze_graph.py", line 361, in freeze_graph
checkpoint_version=checkpoint_version)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/tools/freeze_graph.py", line 190, in freeze_graph_with_def_protos
var_list=var_list, write_version=checkpoint_version)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 482, in _build_internal
names_to_saveables)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saving/saveable_object_util.py", line 343, in validate_and_slice_inputs
for converted_saveable_object in saveable_objects_for_op(op, name):
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saving/saveable_object_util.py", line 206, in saveable_objects_for_op
variable, "", name)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saving/saveable_object_util.py", line 83, in init
self.handle_op = var.op.inputs[0]
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 2154, in getitem
return self._inputs[i]
IndexError: list index out of range

Thanks for looking into this, Gerd

The DPU-PYNQ uses DpuOverlay for loading the DPU bitstream and I would like to know that is the bitstream containing DPU and other accelerator (custom HLS IP) still working? Can the custom accelerator running with the same code in PYNQ?
Thanks for your hard work!

Hi,
I Know that on DPU-PYNQ , Softmax operations was done on CPU and the other operations by DPU.
I was wondering is it possible to re-configure the DPU hardware design so all the operations of softmax and DPU are done in hardware?
In that case how to modify the inference file ?
Regards

Hello,

After successfully running the mnist example on the ZCU104 board, I was trying to do some modifications to the example by importing the sklearn module (to import SVM classifier). I got the following error:

It seems that scikit-learn has not been built correctly. If you have installed scikit-learn from source, please do not forget to build the package before using it: run python setup.py install or make in the source directory. If you have used an installer, please check that it is suited for your Python version, your operating system, and your platform.

I tried to install scikit-learn using: pip3 install scikit-learn. I found that it was already installed.

Any suggestions please?

I have been trying to retrain ResNet50 for MNIST classification using the code below following the provided example Build Machine Learning Models for DPU
However I got the following message
The kernel appears to have died. It will restart automatically.

import os
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
import keras
from keras.layers import Dense, Conv2D, InputLayer, Flatten, MaxPool2D

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data() 
x_train = np.expand_dims(x_train, axis=-1)
x_test = np.expand_dims(x_test, axis=-1)
x_train = np.repeat(x_train, 3, axis=-1)
x_test = np.repeat(x_test, 3, axis=-1)
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255
x_train = tf.image.resize(x_train, [32,32]) 
x_test = tf.image.resize(x_test, [32,32])
y_train = tf.keras.utils.to_categorical(y_train , num_classes=10)
y_test = tf.keras.utils.to_categorical(y_test , num_classes=10)
print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)

input = tf.keras.Input(shape=(32,32,3))
efnet = tf.keras.applications.ResNet50(weights='imagenet',
                                             include_top = False, 
                                             input_tensor = input)
gap = tf.keras.layers.GlobalMaxPooling2D()(efnet.output)

output = tf.keras.layers.Dense(10, activation='softmax', use_bias=True)(gap)
func_model = tf.keras.Model(efnet.input, output)
func_model.compile(optimizer='adam',
              loss="sparse_categorical_crossentropy", 
              metrics=['accuracy'])
func_model.fit(x_train, y_train,epochs=5,validation_data=(x_test,y_test),steps_per_epoch = 1)

Any suggestions how to solve this problem please?
Thanks in advance.

Hi, how can i get the wheel of dpu-pynq 1.2.0?

Hi, I am using Pynq in the ZCU104 board and have a problem when I try to import the vart module, when I write "import vart" this error appears
"ModuleNotFoundError: No module named 'vart'". What should I do?

The dpu_resnet50.ipynb file inside board image says this line:
"We can also run it for multiple images as shown below. In this example we have only used 1 thread; in principle, users should be able to boost the performance by employing more threads."

How can I achieve this to get FPS around 30?

The darknet models I've tried so far from the model zoo actually include the caffe prototxt and model files.
Just by changing if [ $FRAMEWORK = 'cf' ]; then to if [ $FRAMEWORK = 'cf' ] || [ $FRAMEWORK = 'dk' ]; then in both 2 places in the compile.sh I could use it to compile dk models from the zoo.
Granted I have not yet tested if the models work but at least the compilation ran without errors.

Hi,
I tried to run the mnist classifier example on ZCU104 board however I got the folowing error when loading the test data

gaierror Traceback (most recent call last)
/usr/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1317 h.request(req.get_method(), req.selector, req.data, headers,
-> 1318 encode_chunked=req.has_header('Transfer-encoding'))
1319 except OSError as err: # timeout error

/usr/lib/python3.6/http/client.py in request(self, method, url, body, headers, encode_chunked)
1238 """Send a complete request to the server."""
-> 1239 self._send_request(method, url, body, headers, encode_chunked)
1240

/usr/lib/python3.6/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
1284 body = _encode(body, 'body')
-> 1285 self.endheaders(body, encode_chunked=encode_chunked)
1286

/usr/lib/python3.6/http/client.py in endheaders(self, message_body, encode_chunked)
1233 raise CannotSendHeader()
-> 1234 self._send_output(message_body, encode_chunked=encode_chunked)
1235

/usr/lib/python3.6/http/client.py in _send_output(self, message_body, encode_chunked)
1025 del self._buffer[:]
-> 1026 self.send(msg)
1027

/usr/lib/python3.6/http/client.py in send(self, data)
963 if self.auto_open:
--> 964 self.connect()
965 else:

/usr/lib/python3.6/http/client.py in connect(self)
935 self.sock = self._create_connection(
--> 936 (self.host,self.port), self.timeout, self.source_address)
937 self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

/usr/lib/python3.6/socket.py in create_connection(address, timeout, source_address)
703 err = None
--> 704 for res in getaddrinfo(host, port, 0, SOCK_STREAM):
705 af, socktype, proto, canonname, sa = res

/usr/lib/python3.6/socket.py in getaddrinfo(host, port, family, type, proto, flags)
744 addrlist = []
--> 745 for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
746 af, socktype, proto, canonname, sa = res

gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

URLError Traceback (most recent call last)
in ()
----> 1 raw_data = mnist.test_images()
2 normalized_data = np.asarray(raw_data/255, dtype=np.float32)
3 test_data = np.expand_dims(normalized_data, axis=3)
4 test_label = mnist.test_labels()
5

/usr/local/lib/python3.6/dist-packages/mnist/init.py in test_images()
174 columns of the image
175 """
--> 176 return download_and_parse_mnist_file('t10k-images-idx3-ubyte.gz')
177
178

/usr/local/lib/python3.6/dist-packages/mnist/init.py in download_and_parse_mnist_file(fname, target_dir, force)
141 Numpy array with the dimensions and the data in the IDX file
142 """
--> 143 fname = download_file(fname, target_dir=target_dir, force=force)
144 fopen = gzip.open if os.path.splitext(fname)[1] == '.gz' else open
145 with fopen(fname, 'rb') as fd:

/usr/local/lib/python3.6/dist-packages/mnist/init.py in download_file(fname, target_dir, force)
57 if force or not os.path.isfile(target_fname):
58 url = urljoin(datasets_url, fname)
---> 59 urlretrieve(url, target_fname)
60
61 return target_fname

/usr/lib/python3.6/urllib/request.py in urlretrieve(url, filename, reporthook, data)
246 url_type, path = splittype(url)
247
--> 248 with contextlib.closing(urlopen(url, data)) as fp:
249 headers = fp.info()
250

/usr/lib/python3.6/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
221 else:
222 opener = _opener
--> 223 return opener.open(url, data, timeout)
224
225 def install_opener(opener):

/usr/lib/python3.6/urllib/request.py in open(self, fullurl, data, timeout)
524 req = meth(req)
525
--> 526 response = self._open(req, data)
527
528 # post-process response

/usr/lib/python3.6/urllib/request.py in _open(self, req, data)
542 protocol = req.type
543 result = self._call_chain(self.handle_open, protocol, protocol +
--> 544 '_open', req)
545 if result:
546 return result

/usr/lib/python3.6/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
502 for handler in handlers:
503 func = getattr(handler, meth_name)
--> 504 result = func(*args)
505 if result is not None:
506 return result

/usr/lib/python3.6/urllib/request.py in http_open(self, req)
1344
1345 def http_open(self, req):
-> 1346 return self.do_open(http.client.HTTPConnection, req)
1347
1348 http_request = AbstractHTTPHandler.do_request_

/usr/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1318 encode_chunked=req.has_header('Transfer-encoding'))
1319 except OSError as err: # timeout error
-> 1320 raise URLError(err)
1321 r = h.getresponse()
1322 except:

URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

Any help please?
Thanks

Hi,
Follow the README.md and successfully finished all the commands before the following one. When I run the following cmd, the u96_v2 complains

pynq get-notebooks pynq-dpu -p .

xilinx@pynq:~/jupyter_notebooks$ sudo pynq get-notebooks pynq-dpu -p .          
WARNING: No notebooks available, nothing can be delivered 

Might I get some help.
Thanks!

According to [(https://github.com/Xilinx/Vitis-AI/issues/133)], in Vitis AI 1.2, DPUCZDX8G is used instead of DPUv2.