[Performance] Severe performance penalty with transformer model and DirectML about onnxruntime HOT 4 OPEN

@smk2007

from onnxruntime.

I made some extra experiments with dynamic axes. I could confirm that removing any dynamic axes provides a small speed-up (few % with DirectML), but once again it does not fill the gap between Windows+DirectML and Linux+CUDA.

from onnxruntime.

Any reason not to compare Windows CUDA with Linux CUDA?

from onnxruntime.

Any reason not to compare Windows CUDA with Linux CUDA?

The reason is that I don't have a have a reliable way to build onnxruntime with CUDA on Windows, cudart/cudnn distribution for Windows has been sketchy at best in recent times (at least up to 11.7 which is what I currently use). So, I cannot do the measurements on the same infrastructure I use for the other cases. A hacky test however suggests that Windows+CUDA suffers from a similar performance penalty than Windows+DirectML.

from onnxruntime.