Comments (19)
OrtApi last entry from 1.18 is not protected by the usual
static_assert
@Hopobcn @sophies927 A fix is here #20671
from onnxruntime.
@JulienTheron can you provide us a repro test case or point us to a public model that can be used to investigate further? Are you also updating TensorRT version from 8.6 to 10.0 along with the ORT version update? +@chilo-ms
Yes I've used the default TensorRT version for each ORT release, so 8.6 for 1.17.3 and 10 for 1.18.0.
I will see if I can find a public model to repro the issue with.
from onnxruntime.
Is there GPU python package for CUDA 12?
There will be for the release. We had to make some updates to make the CUDA 12 package publication a bit easier (see this PR), which is why the packages weren't included in the first round release candidates.
from onnxruntime.
Here's a follow-up to the TensorRT issue I mentioned.
I edited
Line 41 in 4573740
onnx_tensorrt;https://github.com/onnx/onnx-tensorrt/archive/bacfaaa951653cd4e72efe727a543567cb38f7de.zip;26434329612e804164ab7baa6ae629ada56c1b26
, then built with TensorRT 8.6.
The issue is now gone. I haven't quite figured why ScatterND is not placed on the TensorRT EP with TRT 10, but I guess the issue has nothing to do with the ONNX Runtime?
from onnxruntime.
Is there GPU python package for CUDA 12?
from onnxruntime.
OrtApi last entry from 1.18 is not protected by the usual static_assert
https://github.com/microsoft/onnxruntime/blob/rel-1.18.0/onnxruntime/core/session/onnxruntime_c_api.cc#L2729
https://github.com/microsoft/onnxruntime/blob/rel-1.18.0/onnxruntime/core/session/onnxruntime_c_api.cc#L2758C32-L2758C54
from onnxruntime.
OrtApi last entry from 1.18 is not protected by the usual
static_assert
Thanks for catching that!
@yihonglyu would you mind looking into this?
from onnxruntime.
Hi, I just found out that ScatterND nodes are now placed on CPU. They were placed on GPU before.
See below:
2024-05-13 15:19:36.112 [16248] DEBUG Node placements
2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [CPUExecutionProvider]. Number of nodes: 6
2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND)
2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_1)
2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_2)
2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_3)
2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_4)
2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_5)
2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [TensorrtExecutionProvider]. Number of nodes: 23
With ORT 1.17.3:
2024-05-13 15:38:07.967 [ 576] DEBUG Node placements
2024-05-13 15:38:07.967 [ 576] DEBUG All nodes placed on [TensorrtExecutionProvider]. Number of nodes: 1
Is it because of the upgrade to TensorRT 10?
from onnxruntime.
Hi, I just found out that ScatterND nodes are now placed on CPU. They were placed on GPU before.
See below:
2024-05-13 15:19:36.112 [16248] DEBUG Node placements 2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [CPUExecutionProvider]. Number of nodes: 6 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_1) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_2) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_3) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_4) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_5) 2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [TensorrtExecutionProvider]. Number of nodes: 23
With ORT 1.17.3:
2024-05-13 15:38:07.967 [ 576] DEBUG Node placements 2024-05-13 15:38:07.967 [ 576] DEBUG All nodes placed on [TensorrtExecutionProvider]. Number of nodes: 1
Is it because of the upgrade to TensorRT 10?
from onnxruntime.
Hi, I just found out that ScatterND nodes are now placed on CPU. They were placed on GPU before.
See below:
2024-05-13 15:19:36.112 [16248] DEBUG Node placements 2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [CPUExecutionProvider]. Number of nodes: 6 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_1) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_2) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_3) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_4) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_5) 2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [TensorrtExecutionProvider]. Number of nodes: 23
With ORT 1.17.3:
2024-05-13 15:38:07.967 [ 576] DEBUG Node placements 2024-05-13 15:38:07.967 [ 576] DEBUG All nodes placed on [TensorrtExecutionProvider]. Number of nodes: 1
Is it because of the upgrade to TensorRT 10?
from onnxruntime.
Hi, I just found out that ScatterND nodes are now placed on CPU. They were placed on GPU before.
See below:
2024-05-13 15:19:36.112 [16248] DEBUG Node placements 2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [CPUExecutionProvider]. Number of nodes: 6 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_1) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_2) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_3) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_4) 2024-05-13 15:19:36.112 [16248] DEBUG ScatterND (/ScatterND_5) 2024-05-13 15:19:36.112 [16248] DEBUG Node(s) placed on [TensorrtExecutionProvider]. Number of nodes: 23
With ORT 1.17.3:
2024-05-13 15:38:07.967 [ 576] DEBUG Node placements 2024-05-13 15:38:07.967 [ 576] DEBUG All nodes placed on [TensorrtExecutionProvider]. Number of nodes: 1
Is it because of the upgrade to TensorRT 10?
@JulienTheron can you provide us a repro test case or point us to a public model that can be used to investigate further? Are you also updating TensorRT version from 8.6 to 10.0 along with the ORT version update? +@chilo-ms
from onnxruntime.
Here is a version of our model that we can share:
model.zip
To repro, just create a TensorRT session like this:
const Ort::Env env(ORT_LOGGING_LEVEL_VERBOSE);
Ort::SessionOptions options;
OrtTensorRTProviderOptionsV2* trtOptions;
Ort::GetApi().CreateTensorRTProviderOptions(&trtOptions);
options.AppendExecutionProvider_TensorRT_V2(*trtOptions);
Ort::GetApi().ReleaseTensorRTProviderOptions(trtOptions);
Ort::Session session(env, L"model.onnx", options);
Thanks.
from onnxruntime.
from onnxruntime.
from onnxruntime.
Thanks for calling this out! Appears to be an issue with the model input caused by an ONNX update, rather than an issue with ORT 1.18.
from onnxruntime.
@sophies927, Regarding to #20737, shall we achieve older pypi package before release (also publish new packages to aiinfra.pkgs.visualstudio.com as a backup)? It seems that every new release will cause an older version be deleted.
from onnxruntime.
@sophies927, Regarding to #20737, shall we achieve older pypi package before release (also publish new packages to aiinfra.pkgs.visualstudio.com as a backup)? It seems that every new release will cause an older version be deleted.
Yes, I think that should definitely be the plan moving forward
from onnxruntime.
The ORT 1.18.0 release is now available here: https://github.com/microsoft/onnxruntime/releases
from onnxruntime.
@sophies927 Thank you for the release! Unfortunately, Python's onnxruntime-gpu
package built for CUDA 12 is missing here https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12 . Could you please upload it? Thank you very much.
from onnxruntime.
Related Issues (20)
- [Build] wheels of 1.17/1.18 not found installing with uv HOT 3
- [Mobile] android prod crash: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR) HOT 4
- [Feature Request] prebuilt package with cudnn 9 support HOT 6
- When will onnxruntime make it available to get nodeunit inputs correctly for QLinearConcat HOT 5
- [Build] support for CPython 3.13.0b1 HOT 2
- [Feature Request] why is `FunctionProto` missing from the file "onnxruntime/core/providers/shared_library/provider_wrappedtypes.h"? HOT 3
- [Build] Dockerfile.tensorrt out of date HOT 3
- [Web] I canβt use onnruntime-web to load a onnx model in a react web HOT 6
- Unauthorized for `onnxruntime-cuda-12/pypi/simple/`
- [Documentation] The documentation for early versions is missing HOT 4
- pip install failure for onnxruntime==1.17.3 HOT 1
- Index put loop model regression with ort==1.18 HOT 1
- Non-zero status code returned while running Add node. Name:'Add_221' HOT 1
- can't install on Python HOT 1
- Error in quantize vicuna-7b model from fp16 to int8 HOT 5
- Issue of UNet Model Inference with [model.onnx + model.onnx_data] in ONNX Runtime C++ for Stable Diffusion HOT 9
- Gemm fp8 run error HOT 4
- Where is onnxruntime-openvino==1.18.0 HOT 3
- [.NET] Update tensor implementations to new Tensor<T> type
- [Web] WebGPU and WASM Backends Unavailable within Service Worker HOT 49
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime.