Comments (4)
COREML_FLAG_USE_CPU_ONLY results in CoreML executing the same nodes using its reference CPU implementation. We set this as the MLModelConfiguration.computeUnits. The rest of the ORT CoreML EP code runs exactly the same. That would strongly suggest an issue with the internal CoreML handling of a large input when running on GPU/NPU.
COREML_FLAG_ONLY_ALLOW_STATIC_INPUT_SHAPES is applied on a per-node basis. Parts of the model may have fixed shapes leading to CoreML executing them. If you set the session logging severity to VERBOSE it will print out details of which nodes are/aren't assigned to CoreML. That would at least narrow down which CoreML operator could be going wrong.
from onnxruntime.
This appears to be a CoreML NeuralNetwork specific problem. There are only a few Div and Sub nodes assigned to CoreML as the rest have dynamic input shapes. Most of those produce the expected output.
There are 2 Div nodes (Div_185 and Div_143) that end up doing 2 / (2048 - 1) (one for the height and one for the width). For some reason the NeuralNetwork Div is somewhat inaccurate for this floating point operation.
Python as a reference (double precision):
2.0 / 2047.0 = 0.0009770395701025891
EP | Value name | Value |
---|---|---|
CPU EP | Mul_340 | 0.00097703957 |
CoreML NeuralNetwork | Mul_340 | 0.00097751617 |
CoreML ML Program | Mul_340 | 0.00097703957 |
That difference must become significant across all the other downstream operations in the model, leading to the output discrepancies. I would guess it comes down to floating point inaccuracies from 2 divided by a large number as to why smaller numbers for the height or width don't trigger the issue.
from onnxruntime.
FWIW it's possible to get a good result from NeuralNetwork but the model would need to be updated and you might need some experimentation to figure out what works best.
If I scale down the input size value (the 2047
in this case) first, do the Div, and scale back up it's happy. Guessing it's due to the difference in floating point representation due to the range between '2' and '2047'.
e.g. scaling the 2047 by 1000 (arbitrarily chosen) would be a = 2047 / 1000
, b = 2 / a
, c = b * 1000
from onnxruntime.
This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.
from onnxruntime.
Related Issues (20)
- [Web] Allow passing Uint8ClampedArray to ort.Tensor() HOT 1
- [Build] Android build issue on windows HOT 2
- CoreML - Writing CoreML Model on every inference session creation HOT 2
- [Build] Getting issue with ONNX Runtime library name HOT 2
- 【CUDA does not work】onnxruntime-gpu==1.19.0 HOT 4
- Can anyone successfully use onnx and yolo5? HOT 1
- The data I output using YOLO5 is incorrect, why is that? Has anyone succeeded?
- Why ONNX Runtime Dlls are not Signed by Microsoft
- Breaks RootNamespace
- [Web] requested dist/*.mjs files for cdnjs HOT 2
- AppendExecutionProvider_DML error HOT 2
- [Mobile] Pre-built 1.19.0 lib is missing for onnxruntime-android HOT 2
- run_async
- [Build] cuda dockerfile build error HOT 3
- [Mobile] QNN-EP graph preparation failed HOT 3
- DirectML failed with invalid command
- [Web] [SD 1.5][SD Turbo][Whipser Base] Browser page out of memory - WebGPU and WebNN GPU EP HOT 3
- [Web] Previously used .wasm binaries missing in 1.19.0 (using web bundler with CopyPlugin) HOT 2
- [Mobile][Android][Bug] SIGSEGV if ortSession.close() is called before ortSession.run() finish HOT 4
- [Bug] [onnxruntime-node] Error: no available backend found. ERR: [wasm] backend not found. HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime.