ivilson / yolov7net Goto Github PK

Hey great work adding some initial support for the yolov8 series!

I started messing around with your latest nuget release and noticed that the prediction labels are null unless I specified true for the NumSharp parameter.

But setting param to true does produce a different problem which is that it only ever returns 1 prediction/label (the first one, the same 1st prediction that would show without the NumSharp option)

I can provide more details if it helps, but here's what I'm seeing when I mess around in powershell:

Add-Type -AssemblyName "$DEPS_ROOT\Microsoft.ML.OnnxRuntime.Managed.1.14.0\lib\net6.0\Microsoft.ML.OnnxRuntime.dll"
Add-Type -AssemblyName "$DEPS_ROOT\NumSharp.0.30.0\lib\netstandard2.0\NumSharp.dll"
Add-Type -AssemblyName "$DEPS_ROOT\IVilson.AI.Yolov7net.1.0.3\lib\net6.0\IVilson.AI.Yolov7net.dll"

$y8client = [IVilson.AI.Yolov7net.Yolov8]::new($MODEL_PATH)
$y8client.SetupYoloDefaultLabels()

$img = [System.Drawing.Image]::FromFile($IMG_PATH)

$y8client.Predict($img) 

<# Returns with null Label property ->

Label Rectangle                                                 Score
----- ---------                                                 -----
      {X=726.8968,Y=1456.7361,Width=953.0971,Height=2573.4204}   0.93
      {X=1223.8837,Y=1089.3862,Width=884.0172,Height=2938.6584}  0.91
#>


$y8client.Predict($img, 0, 0, $true)

<# When NumSharp param is true, returns with only first prediction ->

Label               Rectangle                                               Score
-----               ---------                                               -----
Yolov7net.YoloLabel {X=923.1725,Y=1456.7361,Width=714.823,Height=2573.4204}  0.93
#>

I'll try poking around at the new updates in the source code to see when I can figure out but just wanted to see if you might know what's happening here, thanks again!

Microsoft.ML.OnnxRuntime.OnnxRuntimeException:“[ErrorCode:Fail] Load model from ./Assets/yolov8m.onnx failed:D:\a_work\1\s\engine\lotus\onnxruntime\core/graph/model_load_utils.h:57 onnxruntime::model_load_utils::ValidateOpsetForDomain ONNX Runtime only guarantees support for models stamped with official released onnx opset versions. Opset 17 is under development and support for this is limited. The operator schemas and or other functionality may change before next ONNX release and in this case ONNX Runtime will not guarantee backward compatibility. Current official support for domain ai.onnx is till opset 15.
”

/

please help me

用你的模型测试没有问题，但是自己训练模型就会提示超出索引，lable是与训练的lable保持一致的。python调用是可以的，可以请教下需要注意什么问题吗

If i choose useCuda = True

Microsoft.ML.OnnxRuntime 1.11.0:

I thought I needed Microsoft.ML.OnnxRuntime.Gpu
Microsoft.ML.OnnxRuntime.Gpu 1.11.0:

Microsoft.ML.OnnxRuntime.Gpu 1.13.1:

I tried on different devices Windows 10/11
Please, help.

Hello

While running my program I constanty have this error :
Exception thrown: 'System.EntryPointNotFoundException' in Microsoft.ML.OnnxRuntime.dll .
Has anyone encountered this and found a solution?

Greetings

Hello,
YoloV8 by any chance?

Utils.ExtractPixels is very slow. On my machine it is 300-500ms. Nested Parallel processing is unnecessary, it makes the function only slower. If I remove the Parallel loops, the result is 70ms.. which is still quite a lot. (Tensor indexer is very slow, use the tensor.Buffer)

In your code you already assume that the bitmapData is ARGB, 4 byte per pixel, so using the Stride is unnecerrasy, since (from the documetation):

The stride is the width of a single row of pixels (a scan line), rounded up to a four-byte boundary.

And in this case the width of a single row is always multiple of 4.

I rewrote the function, this is only 3ms and not an "unsafe" code:
int pixelCount = width * height;
var spanR = tensor.Buffer.Span;
var spanG = spanR.Slice(pixelCount);
var spanB = spanG.Slice(pixelCount);

    int sidx = 0;
    int didx = 0;
    for (int i = 0; i < pixelCount; i++)
    {
        spanR[didx] = data[sidx + 2] / 255.0F;
        spanG[didx] = data[sidx + 1] / 255.0F;
        spanB[didx] = data[sidx] / 255.0F;
        didx++;
        sidx += 4;
    }

Maybe you can make it even faster by using unsafe code.

This is just an idea how you could make it faster. If you expect higher models in the future (like 8K * 8K), you can keep the outer parallel loop or make "my" single loop parallel... but a nested parallel loop is overkill.... And for 640x640 pixels the parallel loop is unnecessary.

Hi!
Can I use multiple GPUs?
Do I need to do some configuration?

if we separate the loading mode and predict the image, the prediction time is faster ?
Thanks so much ivilson !

Hello, when using useCuda = true the Yolo7 does use the GPU.
However there is no performance gain at all. The result is same as when using CPU.

What could be the reason?

I am using Microsoft.ML.OnnxRuntime.Gpu version 1.6.0
which works with Nvidia Cuda 10.2 and my card GT 710 2GB.

useCuda = false (CPU 40%, GPU 0%)
useCuda = true (CPU 10%, GPU 50%)

Thanks!

作者可以给个邮箱或者其他联系方式吗, 我这里有张图片通过添加这个库引用识别出来的Y坐标为负值, 给您源码看下

我看默认是yolov7-tiny.onnx yolov7-pytorch版本并没有yolov7-tiny版本，darknet确有一个，只不过现在找不到将weights转换onnx代码

using var image = Image.FromFile("Assets/demo.jpg")
可不可不從檔案讀取Image來源,而直接取自某一PictureBox或System.Drawing.Bitmap物件內容?

Hello! your library works nicely. However I am facing some issues when looping over images. Say 20 or more.

_yolov8 = new Yolov8("./yolov8n.onnx", false);
_yolov8.SetupYoloDefaultLabels();

var imgFiles = info.GetFiles().OrderBy(p => p.CreationTime).ToArray();

foreach (var imgFile in imgFiles)
{
using var inputImage = Image.FromFile(imgFile.FullName);
var items = _yolov8.Predict(inputImage).Where(i => i.Score >= 0.70).ToList();

 // more code goes here....

}

The application is a windows form, .net 6.0. It runs okay when in visual studio debugger.
However when running the same compiled EXE alone, out of the visual studio environment.

The applications gets hung "not responding"

Are you aware of any such issues? or differentiation between visual studio debug mode run / and normal exe run?

Thanks!

The predicted scores are same for every single predicted labels.

ive had great results on both yolov7net and ONNX-YOLOv7-Object-Detection using models from here but...

i trained a custom model and if i use it in ONNX-YOLOv7-Object-Detection i get results, however if i use Yolov7net i got no results. Im still very new at training custom models but when im seeing results on one library and not on another it makes me question on whether im doing something wrong or if theres something wrong with yolov7net...

Note:

1. The same image was put into both programs, but they got resized after becuase i had to screen capture them to post here.
2. the image from ONNX-YOLOv7-Object-Detection says "person" because I didn't know how to change it (I don't really know python).
3. also when i mean inconsistent i mean that i dont get any results from yolov7net, (multiple images/videos tested)

results from using ONNX-YOLOv7-Object-Detection:

results from using Yolov7net:

code from yolov7net:

using System.Drawing;
using Yolov7net;
using OpenCvSharp;
using OpenCvSharp.Extensions;

// init Yolov7 with onnx (include nms results)file path
using var yolo = new Yolov7("C:\\Users\\imnot\\Downloads\\yolov7-main\\yolov7-main\\runs\\train\\yolov7-custom2\\weights\\best.onnx", true);
// setup labels of onnx model 
yolo.SetupYoloDefaultLabels();
using var image = (Bitmap)Image.FromFile("C:\\Users\\imnot\\Desktop\\Test.jpg");
var predictions = yolo.Predict(image, 0.01f, 0.5f);

// draw box
Mat img = BitmapConverter.ToMat(image);

foreach (var prediction in predictions) // iterate predictions to draw results
{
    int x = (int)prediction.Rectangle.X;
    int y = (int)prediction.Rectangle.Y;
    int width = (int)prediction.Rectangle.Width;
    int height = (int)prediction.Rectangle.Height;

    Cv2.Rectangle(img, new OpenCvSharp.Rect(x, y, width, height), new Scalar(0, 255, 0), 3);
    Cv2.Rectangle(img, new OpenCvSharp.Rect(x, y - 42, 90, 40), new Scalar(0, 0, 0), -1);
    Cv2.PutText(img, prediction.Label.Name, new OpenCvSharp.Point(x, y - 10), HersheyFonts.HersheySimplex, 1.5, new Scalar(36, 255, 12), 4);

    Console.WriteLine(prediction.Label.Name);
}

var newImg = BitmapConverter.ToBitmap(img);
newImg.Save("C:\\Users\\imnot\\Desktop\\newimage.jpg", System.Drawing.Imaging.ImageFormat.Jpeg);

code from ONNX-YOLOv7-Object-Detection:

import cv2
from imread_from_url import imread_from_url

from yolov7 import YOLOv7

# Initialize yolov7 object detector
model_path = "C:/Users/imnot/Downloads/yolov7-main/yolov7-main/runs/train/yolov7-custom2/weights/best.onnx"
yolov7_detector = YOLOv7(model_path, conf_thres=0.02, iou_thres=0.3)

# Read image
img_url = "https://i.imgur.com/P8MBg2M.jpg"
img = imread_from_url(img_url)

# Detect Objects
boxes, scores, class_ids = yolov7_detector(img)

# Draw detections
combined_img = yolov7_detector.draw_detections(img)
cv2.namedWindow("Detected Objects", cv2.WINDOW_NORMAL)
cv2.imshow("Detected Objects", combined_img)
cv2.imwrite("doc/img/detected_objects.jpg", combined_img)
cv2.waitKey(0)

Please consider removing the numpy dependency, since now it is not faster and only adds an extra dependency (and size) to this package.

使用自己的模型报错信息： lable与训练的lable是一致的 yolov5 类文件

System.IndexOutOfRangeException
HResult=0x80131508
Message=Index was outside the bounds of the array.
Source=System.Private.CoreLib
StackTrace:
在 System.ThrowHelper.ThrowIndexOutOfRangeException()
在 System.Span1.get_Item(Int32 index) 在 Microsoft.ML.OnnxRuntime.Tensors.DenseTensor1.GetValue(Int32 index)
在 Microsoft.ML.OnnxRuntime.Tensors.Tensor1.get_Item(ReadOnlySpan1 indices)
在 Microsoft.ML.OnnxRuntime.Tensors.Tensor1.get_Item(Int32[] indices) 在 Yolov7net.Yolov5.<>c__DisplayClass9_0.<ParseDetect>b__0(Int32 i) 在 System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)

使用自己的模型报错信息： lable与训练的lable是一致的 yolov7 类文件
System.IndexOutOfRangeException
HResult=0x80131508
Message=Index was outside the bounds of the array.
Source=System.Private.CoreLib
StackTrace:
在 System.ThrowHelper.ThrowIndexOutOfRangeException()
在 System.Span1.get_Item(Int32 index) 在 Microsoft.ML.OnnxRuntime.Tensors.DenseTensor1.GetValue(Int32 index)
在 Microsoft.ML.OnnxRuntime.Tensors.Tensor1.get_Item(ReadOnlySpan1 indices)
在 Microsoft.ML.OnnxRuntime.Tensors.Tensor1.get_Item(Int32[] indices) 在 Yolov7net.Yolov7.<>c__DisplayClass6_0.<ParseDetect>b__0(Int32 i) 在 System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)