Comments (4)
Sweet! I assumed that PR was already part of the stable release. It's looking much better with ort-nightly
, thank you
Using PyTorch 2.3.1, Transformers 4.41.2, and ORT 1.19.0
Optimizing with ViT config:
{'EmbedLayerNormalization': 0, 'Attention': 0, 'MultiHeadAttention': 0, 'Gelu': 0, 'FastGelu': 0, 'BiasGelu': 0, 'GemmFastGelu': 0, 'LayerNormalization': 3, 'SimplifiedLayerNormalization': 0, 'SkipLayerNormalization': 23, 'SkipSimplifiedLayerNormalization': 0, 'RotaryEmbedding': 0, 'QOrderedAttention': 0, 'QOrderedGelu': 0, 'QOrderedLayerNormalization': 0, 'QOrderedMatMul': 0}
Optimizing with CLIP config:
{'Attention': 12, 'LayerNormalization': 3, 'QuickGelu': 12, 'SkipLayerNormalization': 23}
from onnxruntime.
Can you specify transformers version as well? They made a lot of changes recently.
from onnxruntime.
Can you specify transformers version as well? They made a lot of changes recently.
Hi,
In [6]: import transformers
In [7]: transformers.__version__
Out[7]: '4.41.2'
I also tried with 4.28.1
that (I think) was released at around the time CLIP fusion support was added to onnxruntime
but got the same results:
Using PyTorch 2.3.1, Transformers 4.28.1, and ORT 1.18.0
Optimizing with ViT config:
{'EmbedLayerNormalization': 0, 'Attention': 0, 'MultiHeadAttention': 0, 'Gelu': 0, 'FastGelu': 0, 'BiasGelu': 0, 'GemmFastGelu': 0, 'LayerNormalization': 3, 'SimplifiedLayerNormalization': 0, 'SkipLayerNormalization': 23, 'SkipSimplifiedLayerNormalization': 0, 'RotaryEmbedding': 0, 'QOrderedAttention': 0, 'QOrderedGelu': 0, 'QOrderedLayerNormalization': 0, 'QOrderedMatMul': 0}
Optimizing with CLIP config:
{'Attention': 0, 'LayerNormalization': 3, 'SkipLayerNormalization': 23}
I'll run the test again with PyTorch 1.13.1 later
Edit: my mistake, 4.28.1 is much older than either of the two PRs that added CLIP attention fusion
from onnxruntime.
Can you try your code with the nightly ORT package instead of the stable ORT 1.18.0 package? New fusions for CLIP were recently added in this PR and aren't in the stable ORT package currently. Alternatively, you can try the "from source" instructions in the PR's description.
from onnxruntime.
Related Issues (20)
- [Build] error cross compiling HOT 4
- Onnxruntime LoadLibrary failed with error 126 HOT 5
- [Feature Request] Introduce get_available_initializers HOT 4
- [Feature Request] onnxruntime-node support for FreeBSD. HOT 1
- [TEST ISSUE] Using this issue to test logic app - please ignore HOT 1
- [Documentation] hopefully final logic app test - please ignore HOT 1
- [Build] detect nothing.i use opencv4.9 onnxruntime 1.16.1, it detect nothing HOT 3
- Error converting Microsoft Phi3 model to ONNX using Python and Transformers HOT 6
- Model saved by ORT as external data format will not be aligned for mapfile support
- CUDA_PATH is set but CUDA wasnt able to be loaded HOT 2
- MLAS failing with "Could not find an implementation for QLinearMatMul"
- [Build] Docerfile.cuda docker image build error HOT 1
- Pushing Rust bindings forward HOT 6
- [Web] Quantized model decreases in size, but takes same amount of inference time as non-quantized model HOT 3
- [Mobile] build error: unknown type name 'bfloat16_t'; did you mean 'float16_t' HOT 2
- [Performance] First inference with CUDAExecutionProvider is slow HOT 1
- [Build] Fail to build onnxruntime-web NPM package HOT 2
- Runtime issue with onnxruntime-win-x64-1.18.1 HOT 5
- [Build] reduce the size of the shared library HOT 1
- [TEST ISSUE - please ignore] cuda HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime.