ermig1979 / synet Goto Github PK
View Code? Open in Web Editor NEWA small framework to infer neural network
License: MIT License
A small framework to infer neural network
License: MIT License
After making some testing I found that the performance of the x64 build is substantially better than win32 build (about 50% difference). The input was a standard, 4x down scaled 720p video.
Is this an expected behavior? Any ways to improve the win32 performance?
Hello,
I am trying to convert the latest Face detection model and getting this error:
Unsupported version 11 of IE model
The model is here:
https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/face-detection-0200/FP32/
I try to compile Synet on win10, and test_onnx can't recognize convert mode;
E:\projects\synet\bin\v143\x64\Release>.\TestOnnx.exe -m convert -fw best.onnx -sm best.xml -sw best.bin
[000] Error: Unknown mode :
An Improved One millisecond Mobile Backbone
Hi. We run converted from OpenVino framework 'face-detection-retail-0005' network and get next inference results on aarch64:
For fp32: 243,4ms
For int8: 856ms.
It is strange that for int8 the calculation is slower. Is that how it should be?
Testing on Win10 x64 system.
I could not build latest code, So I used Simd v6.1.139 and commit c995873 (the one just before Simd v6.1.139 was released).
x64 binary worked like a charm (I had already converted other.onnx
to Synet format under Linux).
⠕ TestOnnx.exe -m=compare
Start and Synet(fp32) single-thread comparison tests :
Tests are finished successfully!
Synet(fp32) memory usage: 31.4 MB.
----- Performance Report -----
Simd Library: 6.1.139; CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz; System Sockets: 1, Cores: 6, Threads: 12; Cache L1D: 32 KB, L2: 256 KB, L3: 12288 KB, RAM: 32605 MB; Available SIMD: AVX2 FMA AVX SSE4.1 SSSE3 SSE3 SSE2 SSE
Test::SynetNetwork::Init{ fp32 }: 21 ms / 1 = 20.598 ms {min = 20.598; max = 20.598}
Test::SynetNetwork::Predict{ fp32 }: 110 ms / 10 = 10.971 ms {min = 10.732; max = 11.125} 72.8 GFlops
Error C3861 '_mm_cvtsi64_si128': identifier not found
Reproduced in Release, Win32, VS2019 16.6.4
Hello!
I'm trying to run the network on this model. I successfully converted the model, but when I run the network I receive the following error
Access violation writing location
in the file ImgToCol.h:194
Program: \bin\x64\Debug\Simd.dll
File: \src\Simd\SimdMemory.h
Line: 161
Expression: nose[i] == NO_MANS_LAND_WATERMARK
Simd.dll!Simd::Free(void * ptr) Line 161 C++
Simd.dll!Simd::Array<float>::~Array<float>() Line 47 C++
Simd.dll!Simd::Avx2::SynetSoftmaxLayerForwardX1(const float * src, unsigned __int64 outer, unsigned __int64 count, float * dst) Line 278 C++
Simd.dll!Simd::Avx2::SynetSoftmaxLayerForward(const float * src, unsigned __int64 outer, unsigned __int64 count, unsigned __int64 inner, float * dst) Line 290 C++
Simd.dll!SimdSynetSoftmaxLayerForward(const float * src, unsigned __int64 outer, unsigned __int64 count, unsigned __int64 inner, float * dst) Line 6763 C++
Lib.dll!Synet::Detail::SoftmaxLayerForwardCpu<float>(const float * src, unsigned __int64 outer, unsigned __int64 count, unsigned __int64 inner, float * dst) Line 133 C++
Lib.dll!Synet::SoftmaxLayer<float>::ForwardCpu(const std::vector<Synet::Tensor<float> *,std::allocator<Synet::Tensor<float> *>> & src, const std::vector<Synet::Tensor<float> *,std::allocator<Synet::Tensor<float> *>> & buf, const std::vector<Synet::Tensor<float> *,std::allocator<Synet::Tensor<float> *>> & dst) Line 168 C++
Lib.dll!Synet::Layer<float>::Forward(const std::vector<Synet::Tensor<float> *,std::allocator<Synet::Tensor<float> *>> & src, const std::vector<Synet::Tensor<float> *,std::allocator<Synet::Tensor<float> *>> & buf, const std::vector<Synet::Tensor<float> *,std::allocator<Synet::Tensor<float> *>> & dst) Line 147 C++
Lib.dll!Synet::Network<float>::Forward() Line 406 C++
Hello,
I can see that the ONNX support has been added.
But how do I convert from the .onnx model file? Your converter requires .bin and .xml
Thank you
Ошибка при конвертировании person-detection-retail-0002.bin/ person-detection-retail-0002.xml - https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/person-detection-retail-0002/FP32/
Cant't convert layer : type = Proposal, name = proposal, id = 222 !
VS 2019 Win32
I tried to convert my model to synet format and it worked:
Convert network from Onnx to Synet : OK.
However when loading the model, net.Load("synet.xml", "synet.bin");
throws exception:
Assertion failed: startAxis <= endAxis && (size_t)endAxis <= _shape.size(), file D:\Coding\git\Synet\src\Synet\Tensor.h, line 396
Onnx model file if needed:
https://we.tl/t-DrVIgiWe7h
the im2col + gemm maybe a little slow.
It says
Not implemented layer : name = Divide_4866 ; type = Divide ; id = 282
Can't convert layer : id = 285 , name = 610 , type = Interpolate , version = opset4 !
Can't convert IE model v11!
Result: 0
Hello, I am trying to compile Synet.sln in vs 2019, but I have an error and part of the project is not compiled.
16>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of '
global namespace''
16>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found
16>C:\Synet\src\Synet\Converters\InferenceEngineV10.h(299,35): warning C4018: '<': signed/unsigned mismatch
17>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ..\..\src\Test\TestQuantization.cpp) 18>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of '
global namespace'' (compiling source file ....\src\Test\TestPrecision.cpp)
17>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestQuantization.cpp)
16>Done building project "UseFaceDetection.vcxproj" -- FAILED.
18>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestPrecision.cpp)
20>------ Build started: Project: TestOnnx, Configuration: Release Win32 ------
19>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' 17>C:\Synet\src\Synet\Converters\Deoptimizer.h(149,65): warning C4018: '<': signed/unsigned mismatch (compiling source file ..\..\src\Test\TestQuantization.cpp) 19>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found 15>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of '
global namespace'' (compiling source file ....\src\Test\TestInferenceEngine.cpp)
17>C:\Synet\src\Test\TestQuantization.h(333,39): warning C4018: '<=': signed/unsigned mismatch (compiling source file ....\src\Test\TestQuantization.cpp)
18>C:\Synet\src\Test\TestDetectionPrecision.h(475,13): warning C4018: '<': signed/unsigned mismatch (compiling source file ....\src\Test\TestPrecision.cpp)
15>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestInferenceEngine.cpp)
17>Done building project "TestQuantization.vcxproj" -- FAILED.
18>Done building project "TestPrecision.vcxproj" -- FAILED.
19>Done building project "TestPerformanceDifference.vcxproj" -- FAILED.
14>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of 'global namespace'' (compiling source file ..\..\src\Test\TestDarknet.cpp) 14>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ..\..\src\Test\TestDarknet.cpp) 20>TestOnnx.cpp 20>TestImage.cpp 14>Done building project "TestDarknet.vcxproj" -- FAILED. 15>C:\Synet\src\Synet\Converters\InferenceEngineV10.h(299,35): warning C4018: '<': signed/unsigned mismatch (compiling source file ..\..\src\Test\TestInferenceEngine.cpp) 15>Done building project "TestInferenceEngine.vcxproj" -- FAILED. 20>C:\Synet\src\Synet\Utils\Activation.h(175,11): error C2039: 'SimdSynetSwish32f': is not a member of '
global namespace'' (compiling source file ....\src\Test\TestOnnx.cpp)
20>C:\Synet\src\Synet\Utils\Activation.h(175,1): error C3861: 'SimdSynetSwish32f': identifier not found (compiling source file ....\src\Test\TestOnnx.cpp)
20>Done building project "TestOnnx.vcxproj" -- FAILED.
========== Build: 13 succeeded, 7 failed, 0 up-to-date, 0 skipped ==========`
Hello,
How do grayscale frames (instead of bgra) affect performance and detection accuracy of Synet algorithms?
Thank you
Hello, is it possible to run a quantized YOLOv8 network without creating a custom layer?
Ошибка при конвертировании text-detection-0002.bin/text-detection-0002.xml
Cant't convert layer : type = Interp, name = pixel_cls/resize_images/ResizeBilinear, id = 122 !
VS 2019 x64 Debug
face_detection конвертируется нормально
I trained a custom model with yolov5 and export to onnx format, then convert to synet format.
The following code detected nothing, but the onnx model works fine with detect.py of yolov5.
Net net;
net.Load("test.xml", "test.bin");
net.Reshape(1920, 1920, 1);
Shape shape = net.NchwShape();
View original;
original.Load("test0.png");
View resized(shape[3], shape[2], original.format);
Simd::Resize(original, resized, ::SimdResizeMethodArea);
net.SetInput(resized, 0.0f, 255.0f);
net.Forward();
Regions objects = net.GetRegions(original.width, original.height, 0.5f, 0.5f);
uint32_t white = 0xFFFFFFFF;
for (size_t i = 0; i < objects.size(); ++i)
{
const Region & object = objects[i];
ptrdiff_t l = ptrdiff_t(object.x - object.w / 2);
ptrdiff_t t = ptrdiff_t(object.y - object.h / 2);
ptrdiff_t r = ptrdiff_t(object.x + object.w / 2);
ptrdiff_t b = ptrdiff_t(object.y + object.h / 2);
Simd::DrawRectangle(original, l, t, r, b, white);
}
original.Save("test0r.ppm");
return 0;
Hello, I would love to use your library to run onnx models without GPU. Unfortunatly I have no idea where to start. How do I go from a compiled onnx model to making predictions with a converted Synet model
Hello Igor!
What do you think about making a brief instruction on how to annotate and train models to be compatible with Synet?
Which software to use for annotation, which for training, general recommendations/requirements, etc.
It would be awesome to have that.
Thank you!
When using your build script after cloning, every target but test_performance difference builds. test_performance difference fails to build with error /usr/bin/ld: libSimd.so: undefined reference to
pthread_create' also there is no documentation on what that target does
Hello,
I tried to use the following model:
Expected output:
Name: age_conv3, shape: 1, 1, 1, 1 - Estimated age divided by 100.
Name: prob, shape: 1, 2, 1, 1 - Softmax output across 2 type classes [0 - female, 1 - male].
Received output example:
age_conv3 = [0.349784911, 0.328657120, 0.373537451, 0.369158268] (4 values instead of one)
prob = [0.0271999519, 0.972800076, 0.0115807839, 0.988419175, 0.0321026519, 0.967897415, 0.0114557156, 0.988544285] (8 values instead of 2)
The question is, why do I receive such output? Maybe I use wrong output processing for StubLayer?
Thank you
Can't convert layer : id = 210 , name = Gather_199549/Cast_122100_const , type = Const , version = opset1 !
Can't convert IE model v10!
Unknown element_type = i32 !
Can't convert layer : id = 383 , name = up_sampling2d/Shape/GatherNCHWtoNHWC/Cast_123490_const , type = Const , version = opset1 !
Can't convert IE model v10!
Basically I can convert only the oldest, early 2019 models.
Hello Igor!
Can you suggest what kind of tracker is best to use with your detector?
Or maybe there is some kind of generic tracker incorporated with SIMD library?
Thank you!
I am wondering which configuration is build when using your build script. Is there a way to change the configuration?
When I try to load any model on Intel Celeron J4125 (x64) the function SynetConvolution32fNhwcDirect::OldReorderWeight()
hangs on an infinite loop (looks like a.macroD
parameter always equals to 0).
I try to convert my own custom onnx model but I get an error.
Convert network from Onnx to Synet : Can't found layer onnx !
Can't found layer onnx !
Can't found layer onnx !
Can't found layer onnx !
Can't convert node[4]: type: Slice, name: Slice_4 ( images onnx::Slice_127 onnx::Slice_128 onnx::Slice_126 onnx::Slice_129 ) -> ( onnx::Slice_130 ) { } !
Conversion finished with errors!
Do you have any idea why I get this error and maybe how I could fix it?
Hello,
I get the following during the compilation under VS2019:
1>------ Rebuild All started: Project: Synet, Configuration: Release x64 ------
1>Synet.cpp
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,30): error C2039: 'SimdTensorDataType': is not a member of '`global namespace''
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,56): error C3646: 'Convert': unknown override specifier
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,57): error C2275: 'Synet::TensorType': illegal use of this type as an expression
1>C:\Synet-master\src\Synet\Params.h(237): message : see declaration of 'Synet::TensorType'
1>C:\Synet-master\src\Synet\Utils\Permute.h(88,68): error C2146: syntax error: missing ')' before identifier 'type'
1>C:\Synet-master\src\Synet\Utils\Permute.h(89,9): error C2334: unexpected token(s) preceding '{'; skipping apparent function body
1>Done building project "Synet.vcxproj" -- FAILED.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========
ggml.ai can quantize a model to int4/8, and can seed up the inference of a model.
Hi. There is a problem: On visual studio 2019 and 2015 (32 bit, not tested on others, 64 bit works) it is impossible to convert and load synet network. Here is screenshot of converting error:
The error while trying load is similar.
In memory windows there is something strange: the pointer to virtual table begins with 4 byte offset. But after that there is a trash, and only after - there is a first "params" data. So, I think it is the compiler bug, but not sure. On gcc everything works fine. Can you test your framework on visual studio to investigate this bug?
Hello!
Is working with INT8 models currently supported?
In files perf.sh, test.sh, check.sh the quantization check is disabled, as well as test_003i and test_009i.
If I try to enable them manually, it start require file quant.xml, but it is missing from the project.
As the title.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.