Does TIM-VX utilizes NPU of vim3(A311D)? about tim-vx HOT 12 CLOSED

1. Yes
2. Yes
3. Not at the moment
4. if the entire graph is offloaded to the NPU, then CPU usage during inference is near 0. The CPU usage comes from the onetime cost of graph compilation.

from tim-vx.

You will not get optimal performance when you build graphs with a single layer since compiler cannot apply further optimizations such as fusion and scheduling.

from tim-vx.

@janoslim our project Tengine has been integrated the TIM-VX for Khadas VIM3(A311D inside), you can reference it, thanks.
https://github.com/OAID/Tengine/blob/tengine-lite/doc/npu_tim-vx_user_manual.md

from tim-vx.

Thank you for your reply!!

I do have just a question that, I am willing to use TIM-VX with seperated graphs layerwisely (such as conv-graph or Relu-graph),
Would I have a performance degradation when I create Graphs seperately compare to build graph with full dnn model?

Thank you so much!

from tim-vx.

@thezha @BUG1989 Thank you so much for your answers.
This would be the last question and I'm afraid my questions might bother you.
Does TIM-VX available on Android11 on VIM3?
I already deploy Android11 from aosp on vim3 and I want to use TIM-VX on android 11, Is it available to do this?
Thanks.

from tim-vx.

@BUG1989
./tm_classification_timvx -m ./models/mobilenet_uint8.tmfile -i ./models/face1.jpg -g 224,224 -s 0.017,0.017,0.017 -w 104.007,116.669,122.679 -r 500
Tengine plugin allocator TIMVX is registered.
tengine-lite library version: 1.2-dev
TIM-VX prerun.
[ 1] Failed to open device: No such file or directory, Try again...
[ 2] Failed to open device: No such file or directory, Try again...
[ 3] Failed to open device: No such file or directory, Try again...
[ 4] Failed to open device: No such file or directory, Try again...
[ 5] _OpenDevice(1130): FATAL: Failed to open device, errno=No such file or directory.
Can‘t find the npu?

u have to make sure that the galcore.ko is the right version for tim-vx, it seems your galcore.ko is not 6.4.3.p0.286725. cmd and print like below:

khadas@Khadas:~$ sudo dmesg | grep "Galcore"
[sudo] password for khadas: 
[   16.888126] Galcore version 6.4.3.p0.286725

when you download & decompress vim3 prebuild-sdk of A311D, galcore.ko should in folder /prebuild-sdk-a311d/lib.
default galcore.ko in vim3 is in

/usr/lib/modules/4.9.241/kernel/drivers/amlogic/npu/galcore.ko

so u can remove this kernel module and insert new one in /prebuild-sdk-a311d/lib/galcore.ko.

cd /usr/lib/modules/4.9.241/kernel/drivers/amlogic/npu
sudo rmmod galcore.ko
cd <your download & decompress path>/prebuild-sdk-a311d/lib
sudo insmod  galcore.ko

or you can simply replace this galcore and reboot, than check the galcore version.

and because of Tengine was already integrated inside VIM3 since Dec. 2020, export LD_LIBRARY_PATHto avoid path conflict is recommended. suppose your Tengine path is /home/khadas/tengine, and build path is /home/khadas/tengine/build, than cmd is below:

export LD_LIBRARY_PATH=/home/khadas/tengine/3rdparty/lib/aarch64:/home/khadas/tengine/build/src:$LD_LIBRARY_PATH

also you can join our qq group(829565581) for more info.

from tim-vx.

after remote assistance, issue solved.

from tim-vx.

@thezha @BUG1989 Thank you so much for your answers. This would be the last question and I'm afraid my questions might bother you. Does TIM-VX available on Android11 on VIM3? I already deploy Android11 from aosp on vim3 and I want to use TIM-VX on android 11, Is it available to do this? Thanks.

Hello,

I have the same question, if google android aosp version 11 support NPU in khadas board? If not, is it possible to work on its support? (Is there manual or link to work on it)

Thanks,
Ehsan

from tim-vx.

Android NN support is available, but not via TIM-VX API. We supply a separate software package for SoC vendor to integrate with Android NN, I think Amlogic/Khadas already has this support.

from tim-vx.

Android NN support is available, but not via TIM-VX API. We supply a separate software package for SoC vendor to integrate with Android NN, I think Amlogic/Khadas already has this support.

Thanks for your reply.
The AOSP that khadas repository published:
https://github.com/khadas/android_manifest.git
supports npu.
However I should use AOSP of google repository:
https://android.googlesource.com/platform/manifest

I am wondering that could I use npu with this AOSP? If yes, would you please tell me how?

Best regards,
Ehsan

from tim-vx.

@Ehsan-aghapour ,

We should be able to work with Android 11 and before. It depends on Khadas/AML add it or not. We don't have the latest AOSP for VIM3 yet.

from tim-vx.

Close issue since inactive over months.

from tim-vx.