ermig1979 / synet Goto Github PK

After making some testing I found that the performance of the x64 build is substantially better than win32 build (about 50% difference). The input was a standard, 4x down scaled 720p video.

Is this an expected behavior? Any ways to improve the win32 performance?

Hello,

I am trying to convert the latest Face detection model and getting this error:
Unsupported version 11 of IE model
The model is here:
https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/face-detection-0200/FP32/

I try to compile Synet on win10, and test_onnx can't recognize convert mode;

E:\projects\synet\bin\v143\x64\Release>.\TestOnnx.exe -m convert -fw best.onnx -sm best.xml -sw best.bin
[000] Error: Unknown mode :

An Improved One millisecond Mobile Backbone

https://arxiv.org/abs/2206.04040

https://github.com/facebookresearch/mobile-vision/blob/4d5207c/mobile_cv/arch/fbnet_v2/mobileone_block.py#L15

Hi. We run converted from OpenVino framework 'face-detection-retail-0005' network and get next inference results on aarch64:

For fp32: 243,4ms
For int8: 856ms.

It is strange that for int8 the calculation is slower. Is that how it should be?

Testing on Win10 x64 system.

I could not build latest code, So I used Simd v6.1.139 and commit c995873 (the one just before Simd v6.1.139 was released).

x64 binary worked like a charm (I had already converted other.onnx to Synet format under Linux).

⠕ TestOnnx.exe -m=compare
Start  and Synet(fp32) single-thread comparison tests :
Tests are finished successfully!

Synet(fp32) memory usage: 31.4 MB.
----- Performance Report -----
Simd Library: 6.1.139; CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz; System Sockets: 1, Cores: 6, Threads: 12; Cache L1D: 32 KB, L2: 256 KB, L3: 12288 KB, RAM: 32605 MB; Available SIMD: AVX2 FMA AVX SSE4.1 SSSE3 SSE3 SSE2 SSE
Test::SynetNetwork::Init{ fp32 }: 21 ms / 1 = 20.598 ms {min = 20.598; max = 20.598}
Test::SynetNetwork::Predict{ fp32 }: 110 ms / 10 = 10.971 ms {min = 10.732; max = 11.125} 72.8 GFlops

But Win32 binary gave me a 0xC0000005 exception here:

Error C3861 '_mm_cvtsi64_si128': identifier not found

Reproduced in Release, Win32, VS2019 16.6.4

Hello!
I'm trying to run the network on this model. I successfully converted the model, but when I run the network I receive the following error

Access violation writing location

in the file ImgToCol.h:194

Hello,

I can see that the ONNX support has been added.
But how do I convert from the .onnx model file? Your converter requires .bin and .xml

Thank you

Ошибка при конвертировании person-detection-retail-0002.bin/ person-detection-retail-0002.xml - https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/person-detection-retail-0002/FP32/

Cant't convert layer : type = Proposal, name = proposal, id = 222 !

VS 2019 Win32

I tried to convert my model to synet format and it worked:

Convert network from Onnx to Synet : OK.

However when loading the model, net.Load("synet.xml", "synet.bin"); throws exception:

Assertion failed: startAxis <= endAxis && (size_t)endAxis <= _shape.size(), file D:\Coding\git\Synet\src\Synet\Tensor.h, line 396

Onnx model file if needed:
https://we.tl/t-DrVIgiWe7h

the im2col + gemm maybe a little slow.

https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/person-detection-0303/FP32/

It says

Not implemented layer : name = Divide_4866 ; type = Divide ; id = 282
Can't convert layer : id = 285 , name = 610 , type = Interpolate , version = opset4 !
Can't convert IE model v11!
Result: 0

Hello, I am trying to compile Synet.sln in vs 2019, but I have an error and part of the project is not compiled.

16>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace''
16>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found
16>C:\Synet\src\Synet\Converters\InferenceEngineV10.h(299,35): warning C4018: '<': signed/unsigned mismatch
17>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ..\..\src\Test\TestQuantization.cpp) 18>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ....\src\Test\TestPrecision.cpp)
17>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestQuantization.cpp)
16>Done building project "UseFaceDetection.vcxproj" -- FAILED.
18>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestPrecision.cpp)
20>------ Build started: Project: TestOnnx, Configuration: Release Win32 ------
19>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' 17>C:\Synet\src\Synet\Converters\Deoptimizer.h(149,65): warning C4018: '<': signed/unsigned mismatch (compiling source file ..\..\src\Test\TestQuantization.cpp) 19>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found 15>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ....\src\Test\TestInferenceEngine.cpp)
17>C:\Synet\src\Test\TestQuantization.h(333,39): warning C4018: '<=': signed/unsigned mismatch (compiling source file ....\src\Test\TestQuantization.cpp)
18>C:\Synet\src\Test\TestDetectionPrecision.h(475,13): warning C4018: '<': signed/unsigned mismatch (compiling source file ....\src\Test\TestPrecision.cpp)
15>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestInferenceEngine.cpp)
17>Done building project "TestQuantization.vcxproj" -- FAILED.
18>Done building project "TestPrecision.vcxproj" -- FAILED.
19>Done building project "TestPerformanceDifference.vcxproj" -- FAILED.
14>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ..\..\src\Test\TestDarknet.cpp) 14>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ..\..\src\Test\TestDarknet.cpp) 20>TestOnnx.cpp 20>TestImage.cpp 14>Done building project "TestDarknet.vcxproj" -- FAILED. 15>C:\Synet\src\Synet\Converters\InferenceEngineV10.h(299,35): warning C4018: '<': signed/unsigned mismatch (compiling source file ..\..\src\Test\TestInferenceEngine.cpp) 15>Done building project "TestInferenceEngine.vcxproj" -- FAILED. 20>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ....\src\Test\TestOnnx.cpp)
20>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestOnnx.cpp)
20>Done building project "TestOnnx.vcxproj" -- FAILED.
========== Build: 13 succeeded, 7 failed, 0 up-to-date, 0 skipped ==========`

Hello,

How do grayscale frames (instead of bgra) affect performance and detection accuracy of Synet algorithms?

Thank you

Hello, is it possible to run a quantized YOLOv8 network without creating a custom layer?

Ошибка при конвертировании text-detection-0002.bin/text-detection-0002.xml

Cant't convert layer : type = Interp, name = pixel_cls/resize_images/ResizeBilinear, id = 122 !

VS 2019 x64 Debug

face_detection конвертируется нормально

I trained a custom model with yolov5 and export to onnx format, then convert to synet format.
The following code detected nothing, but the onnx model works fine with detect.py of yolov5.

   Net net;
    net.Load("test.xml", "test.bin");

    net.Reshape(1920, 1920, 1);

    Shape shape = net.NchwShape();

    View original;
    original.Load("test0.png");

    View resized(shape[3], shape[2], original.format);
    Simd::Resize(original, resized, ::SimdResizeMethodArea);

    net.SetInput(resized, 0.0f, 255.0f);

    net.Forward();

    Regions objects = net.GetRegions(original.width, original.height, 0.5f, 0.5f);
    uint32_t white = 0xFFFFFFFF;
    for (size_t i = 0; i < objects.size(); ++i)
    {
        const Region & object = objects[i];
        ptrdiff_t l = ptrdiff_t(object.x - object.w / 2);
        ptrdiff_t t = ptrdiff_t(object.y - object.h / 2);
        ptrdiff_t r = ptrdiff_t(object.x + object.w / 2);
        ptrdiff_t b = ptrdiff_t(object.y + object.h / 2);
        Simd::DrawRectangle(original, l, t, r, b, white);
    }
    original.Save("test0r.ppm");

    return 0;

Hello, I would love to use your library to run onnx models without GPU. Unfortunatly I have no idea where to start. How do I go from a compiled onnx model to making predictions with a converted Synet model

Hello Igor!

What do you think about making a brief instruction on how to annotate and train models to be compatible with Synet?
Which software to use for annotation, which for training, general recommendations/requirements, etc.

It would be awesome to have that.

Thank you!

When using your build script after cloning, every target but test_performance difference builds. test_performance difference fails to build with error /usr/bin/ld: libSimd.so: undefined reference to pthread_create' also there is no documentation on what that target does

Hello,

I tried to use the following model:
Expected output:
Name: age_conv3, shape: 1, 1, 1, 1 - Estimated age divided by 100.
Name: prob, shape: 1, 2, 1, 1 - Softmax output across 2 type classes [0 - female, 1 - male].

Received output example:
age_conv3 = [0.349784911, 0.328657120, 0.373537451, 0.369158268] (4 values instead of one)
prob = [0.0271999519, 0.972800076, 0.0115807839, 0.988419175, 0.0321026519, 0.967897415, 0.0114557156, 0.988544285] (8 values instead of 2)

The question is, why do I receive such output? Maybe I use wrong output processing for StubLayer?

Thank you

Can't convert layer : id = 210 , name = Gather_199549/Cast_122100_const , type = Const , version = opset1 !
Can't convert IE model v10!

Unknown element_type = i32 !
Can't convert layer : id = 383 , name = up_sampling2d/Shape/GatherNCHWtoNHWC/Cast_123490_const , type = Const , version = opset1 !
Can't convert IE model v10!

Basically I can convert only the oldest, early 2019 models.

Hello Igor!

Can you suggest what kind of tracker is best to use with your detector?

Or maybe there is some kind of generic tracker incorporated with SIMD library?

Thank you!

I am wondering which configuration is build when using your build script. Is there a way to change the configuration?

When I try to load any model on Intel Celeron J4125 (x64) the function SynetConvolution32fNhwcDirect::OldReorderWeight() hangs on an infinite loop (looks like a.macroD parameter always equals to 0).

I try to convert my own custom onnx model but I get an error.

Convert network from Onnx to Synet : Can't found layer onnx !
Can't found layer onnx !
Can't found layer onnx !
Can't found layer onnx !
Can't convert node[4]: type: Slice, name: Slice_4 ( images onnx::Slice_127 onnx::Slice_128 onnx::Slice_126 onnx::Slice_129 ) -> ( onnx::Slice_130 ) { } !

Conversion finished with errors!
Do you have any idea why I get this error and maybe how I could fix it?

Hello,

I get the following during the compilation under VS2019:

1>------ Rebuild All started: Project: Synet, Configuration: Release x64 ------
1>Synet.cpp
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,30): error C2039: 'SimdTensorDataType': is not a member of '`global namespace''
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,56): error C3646: 'Convert': unknown override specifier
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,57): error C2275: 'Synet::TensorType': illegal use of this type as an expression
1>C:\Synet-master\src\Synet\Params.h(237): message : see declaration of 'Synet::TensorType'
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,68): error C2146: syntax error: missing ')' before identifier 'type'
1>C:\Synet-master\src\Synet\Utils\Permute.h(89,9): error C2334: unexpected token(s) preceding '{'; skipping apparent function body
1>Done building project "Synet.vcxproj" -- FAILED.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

ggml.ai can quantize a model to int4/8, and can seed up the inference of a model.

Hi. There is a problem: On visual studio 2019 and 2015 (32 bit, not tested on others, 64 bit works) it is impossible to convert and load synet network. Here is screenshot of converting error:

The error while trying load is similar.

In memory windows there is something strange: the pointer to virtual table begins with 4 byte offset. But after that there is a trash, and only after - there is a first "params" data. So, I think it is the compiler bug, but not sure. On gcc everything works fine. Can you test your framework on visual studio to investigate this bug?

Hello!

Is working with INT8 models currently supported?

In files perf.sh, test.sh, check.sh the quantization check is disabled, as well as test_003i and test_009i.
If I try to enable them manually, it start require file quant.xml, but it is missing from the project.

As the title.