nickswardh / yolodotnet Goto Github PK

The details is:

at YoloDotNet.VideoHandler.VideoHandler.ExtractMetaData()
at YoloDotNet.VideoHandler.VideoHandler.ProcessVideoPipeline(VideoAction step)
at YoloDotNet.VideoHandler.VideoHandler.OnProcessCompleteEvent(Object sender, EventArgs e)
at YoloDotNet.VideoHandler.ProcessHandler.HandleExitEvent(Object sender, EventArgs e)
at YoloDotNet.VideoHandler.ProcessHandler.b__6_2(Object sender, EventArgs e)
at System.Diagnostics.Process.RaiseOnExited()
at System.Diagnostics.Process.WaitForExit()
at YoloDotNet.VideoHandler.ProcessHandler.RunProcess(Executable executable, String arguments)
at YoloDotNet.VideoHandler.VideoHandler.Execute(String cmd, Executable exe)
at YoloDotNet.VideoHandler.VideoHandler.GetVideoInfo()
at YoloDotNet.VideoHandler.VideoHandler.ProcessVideoPipeline(VideoAction step)
at YoloDotNet.VideoHandler.VideoHandler.OnProcessCompleteEvent(Object sender, EventArgs e)
at YoloDotNet.VideoHandler.ProcessHandler.HandleExitEvent(Object sender, EventArgs e)
at YoloDotNet.VideoHandler.ProcessHandler.b__6_2(Object sender, EventArgs e)
at System.Diagnostics.Process.RaiseOnExited()
at System.Diagnostics.Process.WaitForExit()
at YoloDotNet.VideoHandler.ProcessHandler.RunProcess(Executable executable, String arguments)
at YoloDotNet.VideoHandler.VideoHandler.Execute(String cmd, Executable exe)
at YoloDotNet.VideoHandler.VideoHandler.PreProcess()
at YoloDotNet.VideoHandler.VideoHandler.ProcessVideoPipeline(VideoAction step)
at YoloDotNet.Data.YoloBase.RunVideo[T](VideoOptions options, Double confidence, Double iouThreshold, ModelType expectedModel)
at YoloDotNet.Yolo.RunObjectDetection(VideoOptions options, Double confidence, Double iou)
at Program.<

Unhandled exception. System.Exception: There are more labels than available colors.
at YoloDotNet.Extensions.OnnxPropertiesExtension.MapLabelsAndColors(String onnxLabelData, ModelType modelType)
at YoloDotNet.Extensions.OnnxPropertiesExtension.GetOnnxProperties(InferenceSession session)
at YoloDotNet.Data.YoloBase..ctor(String onnxModel, Boolean useCuda, Int32 gpuId)
at YoloDotNet.Yolo..ctor(String onnxModel, Boolean cuda, Int32 gpuId)

Hello,

When using instance segmentation the post processing is taking quite a lot of time and I was wondering if there might be way to optimize it. I found which line is taking the most time, but have not found a way to optimize it.
Maybe somebody else has a good idea.

var value = Enumerable.Range(0, output.Channels).Sum(i => tensor1[0, i, y, x] * maskWeights[i]);

Originally posted by @niclasastrom in #9 (comment)

Performance using the GPU is worse than using the CPU. I have an RTX 4070, running Windows 11 Pro. The latest OS- and NVidia driver updates are installed.

I expected higher throughput when using the GPU, but I could be wrong. What performance can I expect, CPU vs GPU?

For example, the classification test took 130ms on the CPU and 572ms on the GPU. Do you know if this is expected?

I added a couple of lines to measure compute time:

var stopWatch=new Stopwatch();
stopWatch.Start();
List<Classification> results = yolo.RunClassification(image, 3); // Get top 5 classifications. Default = 1
stopWatch.Stop();
Console.WriteLine("Elapsed time: "+stopWatch.ElapsedMilliseconds);

Thanks for your input. If this follow-up question doesn't fit the topic, please forgive me and I will try to file my question somewhere else.

There doesn't seem to be any IoU parameter from what I can see when I looked at the source code, I see that RemoveOverlappingBoxes uses a default value of 0.45f I think it would be useful if this was a parameter.

And maybe another IoU threshold parameter specifically if the overlapping boxes are only the same class, but I don't know if this is preferred. But I think at least one is very much needed.

Hi !

I just asking if this package can do image classification, and how ?

Thanks ^^

Having the label index will be useful as I sometimes use it for further modification of my results.

Hi
I am using visual studio 2022 and the project is in .net8. The exception comes when loading the onnx model. I exported the model from .pt to .onnx in pytorch.

I am using nuget.

The exception is : 'The given key 'date' was not present in the dictionary.'

Please help

Hey great project, I've been tweaking it to use DirectML as the onnx execution provider because I'm one of the poor suckers who doesn't have a Nvidia GPU and it works very well as an abstraction on most GPUs.

If you're interested, I'd be happy to submit my patch thru a PR, although I couldn't seem to come up with a reasonable way to avoid breaking changes to the main Yolo constructor's call signature since it's currently designed for simple disabling of GPU use but let me know if you have any suggestions/preferences in that regard.

Cheers

Hello,

I am currently using the YoloDotNet NuGet package to test the performance of YOLO models, I'm doing this testing for my degree thesis. However, I have encountered an issue where the GPU performance is significantly worse than the CPU performance.

Environment:

YoloDotNet version: v2.0
GPU: 4090
CUDA/cuDNN version: cuda 11.8 and cudnn 8.9.7
.NET version: 8

Steps to Reproduce:

var sw = new Stopwatch();
for (var i = 0; i < 500; i++)
{
    var file = $@"C:\Users\Utente\Documents\assets\images\input\frame_{i}.jpg";

    using var image = SKImage.FromEncodedData(file);
    sw.Restart();
    var results = yolo.RunObjectDetection(image, confidence: 0.25, iou: 0.7);
    sw.Stop();
    image.Draw(results);

    image.Save(file.Replace("input", $"output_{yolo_version}{version}_{target}").Replace(".jpg", $"_detect_{yolo_version}{version}_{target}.jpg"),
        SKEncodedImageFormat.Jpeg);
    times.Add(sw.Elapsed.TotalMilliseconds);
    Console.WriteLine($"Time taken for image {i}: {sw.Elapsed.TotalMilliseconds:F2} ms");

This is the way I'm taking the time measure for the detections

Expected Behavior is that the inference using the GPU should be faster than inference using the CPU.
But the performance are not improving using the GPU.

To load the model i use this setup in the GPU case

yolo = new Yolo(new YoloOptions
{
    OnnxModel = @$"C:\Users\Utente\Documents\assets\model\yolov{yolo_version}{version}_{target}.onnx",
    ModelType = ModelType.ObjectDetection,  // Model type
    Cuda = true,                           // Use CPU or CUDA for GPU accelerated inference. Default = true
    GpuId = 0,                               // Select Gpu by id. Default = 0
    PrimeGpu = true,                       // Pre-allocate GPU before first. Default = false
});
Console.WriteLine(yolo.OnnxModel.ModelType);
Console.WriteLine($"Using GPU for version {yolo_version}{version}");

Performance Metrics:

GPU Inference Time:
Total time taken for version m: 21124.52 ms
Average time per image for version m: 42.25 ms

CPU Inference Time:
Total time taken for version m: 18869.73 ms
Average time per image for version m: 37.74 ms

@NickSwardh I would appreciate any assistance or guidance in resolving this issue. Please let me know if you need any further information.

Thank you.

Hello,

in your code, you are using imagesharp directly but i need to perform some custom operations on my video feed for advance detection and drawing. Can you provide a sample code for that, i want to:

1. filter based on label (so only cars or bikes are boxed in the feed)
2. get instance segmentation and recolor objects based on filtering

thanks

Hi, sorry to bother for what can I imagine a basic question but after installing cuda and cuDNN as suggested in the README I get this message in the console:

Could not locate zlibwapi.dll. Please make sure it is in your library path!

If I set the flag to use the CPU everything works.

@NickSwardh if you have some advice it will be really helpful.

Btw you have done a really nice implementation. It has simplified the work for my thesis a lot!

Thankyou so much for your work!
The code works fine just a small issue in your code in YoloDotNet/Extensions/ImageExtensions.cs line 33
var tensor = new DenseTensor(new[] { inputBatchSize, inputChannels, height, width });

should be replaced with
var tensor = new DenseTensor(new[] { inputBatchSize, inputChannels, width, height });

This fixes the dimension error (else dim error will occur for images not having same width and height).

as title, i love this project

System.NotSupportedException:“Onnx model not supported!”

Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:RuntimeException] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\cyberpark\Documents\YoloDotNet-master\ConsoleDemo\bin\Debug\net8.0\runtimes\win-x64\native\onnxruntime_providers_cuda.dll"

_session = useCuda
? new InferenceSession(onnxModel, SessionOptions.MakeSessionOptionWithCudaProvider(gpuId))
: new InferenceSession(onnxModel);

OnnxModel = _session.GetOnnxProperties();

_parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount };
_useCuda = useCuda;

in this case i got this problem after going to video progress section, can you assist?

Hi,

Do you have a version for Yolo nas?

Thx