Does ExecuteNetwork support "GpuAcc" runtime ? about armnn HOT 13 CLOSED

@FrancisMurtagh-arm: The issue is fixed. There was a hardware bug.

from armnn.

Hello.

Yes ExecuteNetwork does support acceleration on Mali GPU's. In the example you show it appears the a part of the model is not supported. Can I suggest you try specifying both CPU and GPU: "-c GpuAcc,CpuAcc" Any layer that is not supported on GPU will fall back to CPU.

Colm.

from armnn.

@Colm-in-Arm : Thanks for your response. As I mentioned, I want to execute the Neural Network in my SoC. Right I cannot install python, numpy or any other packages as my SoC is isolated from the web access.

Kindly help me with an alternate option to execute my NeuralNetwork in the Mali GPU g710.

Also Does ExecuteNetwork help to accelarate the NeuralNetowrk in any other ARM based GPU ?

from armnn.

@somasundaram1702

From the error message there looks to be an issue with OpenCl on your device:
"failed creating base context during opening of kernel driver"

This might result in GpuAcc backend failing to register.

Can you show us the output from calling clinfo?

Your picture also cuts off the ending of the message "Current platform provides:.." what does that say?
If GpuAcc isnt in that list then it confirms it didnt register.

Thanks,
Francis.

from armnn.

@FrancisMurtagh-arm: Yes you are right, the "GpuAcc" isn't listed there.

It says "current platform provides [CpuAcc, CpuRef]". Please note that I am using the opencl files generated by utilizing the DDK specific to Mali G710 GPU.

Also the test application "mali_base_csf_cqs_test" comes along with the DDK gets passed. Which indicates that the certain operations are functional on the GPU.

when I check for clinfo:
-sh: clinfo: not found. I believe the CL utilities are not installed.

@FrancisMurtagh-arm :Is this why the ExecuteNetwork is not able to identify the GPU ? Kindly give some information, Thanks

from armnn.

Hi @somasundaram1702,

Are you using a distro like ubuntu or debian? Can you get it via sudo apt install clinfo ?

Our GPU Backend dynamically loads one of these "libOpenCL.so", "libGLES_mali.so", "libmali.so" from your LD_LIBRARY_PATH, if it can't find at least one of them or there is an issue with it it may fail to register.

Francis.

from armnn.

@FrancisMurtagh-arm : No I am not using any of the distro. I am building Linux-5.10.198 from scratch. Also as I mentioned earlier, my SoC is not connected to web, so "sudo apt install --" does not work. (Trying my best to include "Clinfo" while building rootfs)

Note: "ExecuteNetwork works fine when I use "CpuAcc" or "CpuRef".

W.r.t Dynamic Loading: Yes as you mentioned, I set LD_LIBRARY_PATH pointing to "libOpenCL.so", "libGLES_mali.so", "libmali.so" files. Initially I used to get the below error,

Info: ArmNN v33.0.0
Couldn't find any of the following OpenCL library: libOpenCL.so libGLES_mali.so libmali.so

But after setting the LD_LIBRARY_PATH with proper ".so" files generated by the DDK, the driver is able to pick the files and the above error disappeared. Which proves that the driver is able to load the files.

Note that I am using "insmod" to insert my drivers from my linux terminal. Below are the drivers that I am inserting on my SoC along with firmware,

1. dma-buf-test-exporter.ko
2. memory_group_manager.ko
3. protected_memory_allocator.ko

• Do I need more drivers (which will invoke opencl capabilities) to execute the NeuralNetwork on the GPU ?
• Is "Clinfo" utility necessary ?
• Is there any register that I can read to validate that the integrated "Mali GPU" is available for OpenCL computations ?

Kindly guide me with way forward.

Thanks !

from armnn.

Hi,

clinfo would be the easiest way I can think of for seeing if OpenCl is configured correctly.

Can you copy over a .deb archive of clinfo and dpkg -i install it?

Another user had a similar issue:
#173

Regards,
Francis.

from armnn.

Hello @FrancisMurtagh-arm,

I was able to recompile the DDK by enabling OpenCL capabilities. I get the below error while executing Mobilenet application through ExecuteNetwork,

Error: An error occurred attempting to execute a workload: CL error: clFlush. Error code: -36 at function Execute [/devenv/armnn/src/backends/cl/workloads/ClFullyConnectedWorkload.cpp:110]
Info: Execution time: 34389.63 ms.
terminate called after throwing an instance of 'armnn::Exception'
what(): IRuntime::EnqueueWorkload failed
[ 676.399109] mali 70000000.gpu: Failed to soft-reset GPU (timed out after 500 ms), now attempting a hard reset
[ 676.399355] mali 70000000.gpu: reloading firmware
[ 676.413358] mali 70000000.gpu: Reset complete
Aborted

Could you direct me for a resolution ?

from armnn.

Hi,

That seems like an issue with OpenCL configuration rather than on ArmNN side, can you share the Mobilenet model you are using and I can try run it to debug?

It might be worth asking in https://github.com/ARM-software/ComputeLibrary/issues

Regards,
Francis.

from armnn.

@FrancisMurtagh-arm:

Please find the mobilenet model used. Note that this model runs successfully on CpuAcc runtime. Also I have given the complete log below. Kindly assist.
MobileNet.zip

0:02:00: Warning: DEPRECATED: The program option 'model-format' is deprecated and will be removed soon. The model-format is now automatically set.
0:02:00: Warning: No input files provided, input tensors will be filled with 0s.
0:02:00: Info: ArmNN v33.0.0
0:02:01: [ 89.391250] mali 70000000.gpu: Loading Mali firmware 0x1010000
0:02:01: [ 89.412008] mali 70000000.gpu: Mali firmware git_sha: ba6471e0f3fa3a974709abd2628da574543b3c1d
0:02:09: Info: Initialization time: 495.02 ms.
0:02:12: Info: Optimization time: 580.56 ms
0:02:12:
0:02:13: [ 91.178812] random: crng init done
0:05:41: Warning: The input data was generated, note that the output will not be useful
0:05:41: ===== Network Info =====
0:05:41: Inputs in order:
0:05:41: InputLayer, [1,3,224,224], Float32
0:05:41: Outputs in order:
0:05:41: OutPutLayer, [1,1000], Float32
0:05:41:
0:06:58: [ 230.783654] mali 70000000.gpu: AS_ACTIVE bit stuck for as 1. Might be caused by unstable GPU clk/pwr or faulty system
0:06:58: [ 230.783768] mali 70000000.gpu: Preparing to soft-reset GPU
0:06:58: [ 230.783874] mali 70000000.gpu: Wait for AS_ACTIVE bit failed for as 1, before sending MMU command 4
0:06:58: [ 230.783980] mali 70000000.gpu: Flush for GPU page table update did not complete
0:06:58: [ 230.785483] mali 70000000.gpu: Unhandled Page fault in AS1 at VA 0x00007FDFFA80A980
0:06:58: [ 230.785483] Reason: Memory is not growable
0:06:58: [ 230.785483] raw fault status: 0x230002C3
0:06:58: [ 230.785483] exception type 0xC3: TRANSLATION_FAULT at level 3
0:06:58: [ 230.785483] access type 0x2: READ
0:06:58: [ 230.785483] source id 0x2300
0:06:58: [ 230.785483] pid: 1291
0:06:58: [ 230.785704] mali 70000000.gpu: Failed to lock AS 1 for ctx 1291_0
0:08:15: [ 231.310622] mali 70000000.gpu: Stuck waiting on CLEAN_CACHES_COMPLETED bit, might be due to unstable GPU clk/pwr or possible faulty FPGA connector
0:08:15: [ 231.310759] mali 70000000.gpu: Failed to flush GPU cache when disabling AS 1 for ctx 1291_0
0:08:15: [ 231.317143] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.320093] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.324444] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.328293] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.331804] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.335644] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.339814] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.343293] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.347092] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.351120] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.354593] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.358192] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.361243] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.362342] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.366294] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.369592] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.373812] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.377661] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.380943] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.384593] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.388692] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.391753] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.395848] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.400042] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.402593] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.406943] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.410293] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.413870] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.417643] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.421817] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.425258] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.429093] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.433093] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.435662] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.437093] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.440658] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.444593] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.448443] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.451804] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.455856] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.459693] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.463104] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.466393] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.470641] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.473693] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.477768] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.481947] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.484896] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.489093] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:15: [ 231.493170] mali 70000000.gpu: Flush for GPU page table update did not complete
0:08:51: [ 263.158760] mali 70000000.gpu: [74390095] Suspend request sent on CSG slots 0x1 timed out for slots 0x1
0:08:51: [ 263.158874] mali 70000000.gpu: Timeout waiting for CSG slots to suspend before reset, slot_mask: 0x01
0:08:51: [ 263.702757] mali 70000000.gpu: Cache clean timed out. Might be caused by unstable GPU clk/pwr or faulty system
0:08:51: [ 263.702868] mali 70000000.gpu: [74479056] Timeout waiting for CACHE_CLN_INV_L2_LSC
0:08:51: [ 263.703011] mali 70000000.gpu: Quit idle for failing to prevent gpu reset.
0:10:07: [ 264.227966] mali 70000000.gpu: AS_ACTIVE bit stuck for as 0. Might be caused by unstable GPU clk/pwr or faulty system
0:10:07: [ 264.228081] mali 70000000.gpu: Flush for GPU page table update did not complete
0:10:07: [ 264.229444] mali 70000000.gpu: Flush for GPU page table update did not complete
0:10:07: [ 264.229554] mali 70000000.gpu: Evicting context 1291_0 slots: 0x01
0:10:07: [ 264.254933] mali 70000000.gpu: Resetting GPU (allowing up to 500 ms)
0:10:07: [ 264.255022] mali 70000000.gpu: Register state:
0:10:07: [ 264.255119] mali 70000000.gpu: GPU_IRQ_RAWSTAT=0x00040200 GPU_STATUS=0x00000001 MCU_STATUS=0x00000001
0:10:07: [ 264.255238] mali 70000000.gpu: JOB_IRQ_RAWSTAT=0x00000000 MMU_IRQ_RAWSTAT=0x00000000 GPU_FAULTSTATUS=0x00000000
0:10:07: [ 264.255365] mali 70000000.gpu: GPU_IRQ_MASK=0x00000000 JOB_IRQ_MASK=0x00000000 MMU_IRQ_MASK=0x00000000
0:10:07: [ 264.255476] mali 70000000.gpu: PWR_OVERRIDE0=0x00000000 PWR_OVERRIDE1=0x00000000
0:10:07: [ 264.255583] mali 70000000.gpu: SHADER_CONFIG=0x00000000 L2_MMU_CONFIG=0x00000000 TILER_CONFIG=0x00000000
0:10:07: Error: An error occurred attempting to execute a workload: CL error: clFlush. Error code: -36 at function Execute [/devenv/armnn/src/backends/cl/workloads/ClFullyConnectedWorkload.cpp:110]
0:10:07: Info: Execution time: 34094.88 ms.
0:10:07: terminate called after throwing an instance of 'armnn::Exception'
0:10:07: what(): IRuntime::EnqueueWorkload failed
0:10:08: [ 264.755764] mali 70000000.gpu: Failed to soft-reset GPU (timed out after 500 ms), now attempting a hard reset
0:10:08: [ 264.756624] mali 70000000.gpu: reloading firmware
0:10:08: [ 264.835837] mali 70000000.gpu: Reset complete
0:10:10: Aborted

from armnn.

@FrancisMurtagh-arm : Where you able to execute the model ? I am stuck here badly

from armnn.

Hi @somasundaram1702,

I ran your model successfully on an Odroid n2. I would suggest you ask for help in https://github.com/ARM-software/ComputeLibrary/issues as this doesn't appear to be an ArmNN issue rather an OpenCL configuration issue.

Regards,
Francis.

from armnn.