Import Whisper large v3 into Spark NLP about spark-nlp HOT 5 CLOSED

Hmm this is a very curious error, I don't think anything is hard coded that would cause this. I will update this thread, once I have found the underlying issue!

from spark-nlp.

It seems ONNX models above 2G will be exported into *model.onnx and *model.onnx_data files. Similar to other models we have, we should also adapt Whisper to use that feature since it can go above 2G and not assume there will be 1 file. (we support model.onnx_data)

from spark-nlp.

from sparknlp.annotator import *

whisper = (
WhisperForCTC.loadSavedModel(f"/content/export_onnx/openai/whisper-large", spark)
.setInputCols("audio_assembler")
.setOutputCol("text")

IllegalArgumentException: Size cannot be greater than Integer max value: 2547138560

from spark-nlp.

Hi @DevinTDHa
Is this a limitation on Java or we hard-coded a dimension somewhere?

from spark-nlp.

Hi @minhduc210196,

I am trying to reproduce it, but I don't get quite the same error. It also fails on my side, because the large model is split into a graph file (*model.onnx) and a file with the weights (*model.onnx_data).

If it differs from the default notebook, could you share your environment, so I could try to reproduce it? Namely the version of transformers and optimum. Perhaps you could share a file with the output of pip freeze > pip_env.txt. Thanks!

PS: I was able to reproduce it, by renaming the encoder.onnx_data file to encoder.onnx. Is this what you did?

from spark-nlp.