xenova / transformers.js Goto Github PK

I'm trying to run a slightly larger model (https://huggingface.co/facebook/bart-large-cnn). It's a Bart model, and I converted it to .onnx successfully using your script. Size comes out to ~1gb.

I get this error when running the summarization and text2text generation pipelines:

Error: invalid input 'encoder_attention_mask'
    at eval (ort-web.min.js?d8cf:6:446899)
    at Array.forEach (<anonymous>)
    at e.OnnxruntimeWebAssemblySessionHandler.run (ort-web.min.js?d8cf:6:446819)
    at InferenceSession.run (inference-session-impl.js?f23d:91:1)
    at sessionRun (models.js?a626:34:1)
    at seq2seq_forward (models.js?a626:111:1)
    at async Function.forward (models.js?a626:971:1)
    at async seq2seqRunBeam (models.js?a626:168:1)
    at async Function.runBeam (models.js?a626:964:1)
    at async Function.generate (models.js?a626:562:1)

Describe the bug
A clear and concise description of what the bug is.

Converter doesn't convert Whisper model to ONNX.
Converter doesn't work for non *.en mpdels

How to reproduce
Steps or a minimal working example to reproduce the behavior
python ./scripts/convert.py --model_id openai/whisper-tiny --from_hub --quantize --task speech2seq-lm-with-past

result:
Merging decoders
Traceback (most recent call last):
File "D:\Users\Dimq1\source\OpenAI\transformers.js\scripts\convert.py", line 301, in
main()
File "D:\Users\Dimq1\source\OpenAI\transformers.js\scripts\convert.py", line 293, in main
merge_decoders(
File "C:\Users\Dimq1\AppData\Local\Programs\Python\Python310\lib\site-packages\optimum\onnx\graph_transformations.py", line 135, in merge_decoders
_unify_onnx_outputs(decoder, decoder_with_past)
File "C:\Users\Dimq1\AppData\Local\Programs\Python\Python310\lib\site-packages\optimum\onnx\transformations_utils.py", line 147, in _unify_onnx_outputs
_check_num_outputs(model1, model2)
File "C:\Users\Dimq1\AppData\Local\Programs\Python\Python310\lib\site-packages\optimum\onnx\transformations_utils.py", line 136, in _check_num_outputs
raise ValueError(
ValueError: Two model protos need to have the same outputs. But one has 18 outputs while the other has 10 outputs.
PS D:\Users\Dimq1\source\OpenAI\transformers.js>

Expected behavior
A clear and concise description of what you expected to happen.

Logs/screenshots
If applicable, add logs/screenshots to help explain your problem.

Environment

• Transformers.js version:
• Browser (if applicable):
• Operating system (if applicable):
• Other:

Additional context
Add any other context about the problem here.

Hi @xenova,

Thanks again for all the hard work and this amazing library.
I am trying to make it work on Next.js. Everything works fine but when I build (static site generation) and I run the website, I get weird errors. I think they might be related to webpack.

This is the error:
Uncaught SyntaxError: Label 'r' has already been declared

I am using next.js 13.2.4

I have seen you made some commits to fix Jimp on Next.js do you have an example how to use it?

Just wanted to mention that I tried pipeline('summarization') without a second parameter and it didn't work (default is sshleifer/distilbart-cnn-12-6)

it did work when I provided the model that is specified in the test - sshleifer/distilbart-cnn-6-6.

I saw that 12-6 is an existing model on huggingface, is there a process that makes a model compatible? I'm sorry if these newbie questions are answered somewhere? I saw the section on how to convert a model to onnx... was it done for all the models manually?

Another questions is how to integrate a model that works slightly differently - for example this model that is oriented for Q&A?

Thanks

Describe the bug
A clear and concise description of what the bug is.

Uncaught (in promise) Error: failed to call OrtRun(). error code = 6.
    at e.run (ort-web.min.js:6:454860)
    at e.run (ort-web.min.js:6:444208)
    at e.OnnxruntimeWebAssemblySessionHandler.run (ort-web.min.js:6:447139)
    at o.run (inference-session-impl.js:91:44)
    at x (models.js:52:32)
    at A (models.js:147:34)
    at Function.forward (models.js:936:22)
    at O (models.js:202:29)
    at Function.runBeam (models.js:927:22)
    at Function.generate (models.js:558:41)

How to reproduce
Steps or a minimal working example to reproduce the behavior

Expected behavior
A clear and concise description of what you expected to happen.

Try on this audio file in Chrome for macOS

Logs/screenshots
If applicable, add logs/screenshots to help explain your problem.

Environment

• Transformers.js version: latest from npm
• Browser (if applicable): Chrome
• Operating system (if applicable): MacOS
• Other:

Additional context
Add any other context about the problem here.

I am running the flan-t5 model for text2text-generation (e.g. await pipeline("text2text-generation", "flan-t5-base");) in a service worker. Seeing this error:

TypeError: Cannot convert undefined to a BigInt
    at BigInt (<anonymous>)
    at eval (models.js?a626:289:1)
    at Array.map (<anonymous>)
    at Function.toI64Tensor (models.js?a626:289:1)
    at Function.prepare_inputs (models.js?a626:303:1)
    at Function.forward (models.js?a626:557:1)
    at Function.runBeam (models.js?a626:544:1)
    at Function.generate_single (models.js?a626:366:1)
    at eval (models.js?a626:326:1)
    at Array.map (<anonymous>)

Dear @xenova ,

thank you for this repo! It is pretty sad that Microsoft did no similar for their onnx web on their own and i am grateful that you've done it.

I tried to make the simplest demo at all in JSfiddle (and codepen).
After adding the jsdelivr source, just added the following js:

import { pipeline } from "/@xenova/transformers";

// Allocate a pipeline for sentiment-analysis
let pipe = await pipeline('embeddings', 'sentence-transformers/all-MiniLM-L6-v2');

let out = await pipe('I love transformers!');
console.log(out);

but get no output in the console.

Expected: is the embedding of the text.

Adding the trailing "/" was needed, due to error:
"Uncaught TypeError: Failed to resolve module specifier \"@xenova/transformers\". Relative references must start with either \"/\", \"./\", or \"../\"."
Could that be the source of the problem?

JSFiddle is here.

thank you for your help!

While converting the bloomz model, I am getting the 'invalid syntax' error. Is conversion limited to only predefined model types?
If not, please provide the syntax for converting the above model with quantization.

(I will run the inference in nodejs and not in browser, so memory will not be an issue in inference.)

When using the pipeline from transformers.js library for inference, the output answer span and score are very different from the results obtained by using transformers Pipeline. Specifically, when inputting the same question and passage, the answer span output by transformers.js library is not in the passage, and has a low score; while the answer span output by transformers Pipeline is in the passage, and has a high score. This indicates that there is a problem with the question answering model from transformers.js library.

After testing, I found that under the condition of using the same model and input, the output results of fill mask and question answering are very different from transformers pipeline.

Hi @xenova, I just noticed you uploaded a quantized Blenderbot to Hugging Face. Do you have instructions on how to use it? The original model doesn't have any...

Thanks!

.

I ran the following locally python ./scripts/convert.py --model_id openai/whisper-tiny.en --from_hub --quantize --task speech2seq-lm-with-past

Which worked mostly fine. However, the resulting directory containing converted model had a couple of files missing, namely
preprocessor_config.json and tokenizer.json. After grabbing them manually from HF https://huggingface.co/openai/whisper-tiny.en/tree/main, things seem to work fine

I'm requesting to add the pipeline "zero-shot-classification" for label extraction from a text based on a pretrained model and hypotheses.
To view usage examples: https://huggingface.co/facebook/bart-large-mnli

This is a very requested feature as can be measured by the number of downloads of the linked model in the last month (> 2.5M)
And thank you for the extremely useful work you are doing!

The site source code should be removed from the library.
There are several solutions:

• Make this repo a monorepo: I would use Lerna to keep and maintain the examples and the library all together with many packages:
  • transformers
  • demo
• Split out the demo in another repo

Hi, thank you for your works

I'm a nodejs user and i read that there is no model cache implementation right now, and you are working on it.

Do you have an idea of when you will be able to push a release with a cache implementation ?

Just asking because i was at the point to code it on my side

I would love to see a discussion on using the web-app URL-interface, as a way into using the various AI models and getting some form of semantically structured JSON back. This would make integrating the library a lot easier I think.

In my mind I see the tasks as a function-name name and the model context data as arguments on that function. There can also be global arguments, which are applied to all tasks, eg. the format type of the output (and perhaps the language).

I'm currently using the demo app with some URL parameters: https://conze.pt/app/ai/?task=summarization&arg1=Lightning%20is%20a%20naturally%20occurring&l=en

Maybe a good first step would be to have a set of JS objects describing the:

• Available tasks,
• Supported models for each task,
• The model arguments (defaults, optional. required). Later these could be more formalized using some schema.org schema (or similar), eg: AchieveAction

const tasks = {

  'summarization':

    'description' : 'text summarization',
    'type' : 'text-generation',

    'models': {

      't5-small' : {
        'options' : {
          'output_formats' : [ 'text, 'json' ],
          'output_name' : 'summary',
          'max_new_tokens': 50,
          'num_beams': 1,
          'temperature': 1,
          'top_k': 20, 
          'do_sample': true,
          'summary_text' : 'text',
         }
       },

      't5-base' : ...,
      't5-v1_1-small' : ...,
      't5-v1_1-base' : ...,
      'facebook/bart-large-cnn' : ,
      'sshleifer/distilbart-cnn-6-6' : ...,
      'sshleifer/distilbart-cnn-12-6', ...,
    }

  // ...another task

}

TODO: Maybe add some "required / optional" and data-type declarations on the model options too.

Once we have this we can create a URL parameter structure to call a task with its options and get some plaintext or JSON back.

And perhaps (later) look into creating an OpenAPI schema (never did that myself): https://editor.swagger.io

I'm not yet sure what the best way would be to use larger amounts of text for models, I'm currently using HTTP GET, but a POST might be better suited for this. The drawback is the linking needs to become indirect then.

Hello team,

How do I simply output tokens / embeddings from a model like "bert-base-multilingual-cased" using this library?

Thanks.

Normal VIT seems to struggle in my test, maybe these are better for object-detection?

• DETR https://huggingface.co/docs/transformers/model_doc/detr
• OWL-VIT https://huggingface.co/docs/transformers/model_doc/owlvit

These are models that support a wider range of languages.

• mT5 https://huggingface.co/docs/transformers/model_doc/mt5
• mBART https://huggingface.co/docs/transformers/model_doc/mbart
• NLLB https://huggingface.co/docs/transformers/model_doc/nllb

Btw, very cool project, thanks!

The automatic-speech-recognition currently does not have the option to return timestamps for the text. Is it possible to enable this behavior?

In the AutomaticSpeechRecognitionPipeline class the callback_function is used in two places:

1. To return the chucks
2. In the model generation

As a result, 1 returns a different data format than 2. This wasn't very clear to me. Does it not make sense to have a separate callback for these?

Given the following script:

const { pipeline, env } = require("@xenova/transformers");
env.onnx.wasm.numThreads = 1;


(async() => {
  let embedder = await pipeline('embeddings', 'sentence-transformers/all-MiniLM-L6-v2');
  let sentences = [
      'The quick brown fox jumps over the lazy dog.'
  ];
  let output = (await embedder(sentences)).tolist();
  console.log(output);
})();

This will execute without error when running the following from the shell

$ node test.js

But if the same script is executed using jest, I receive the following error:

$ npx jest test.js

TypeError: A float32 tensor's data must be type of function Float32Array() { [native code] }
    at new h (node_modules/onnxruntime-common/dist/webpack:/onnxruntime-common/lib/tensor-impl.ts:111:17)
    at m.run (node_modules/onnxruntime-common/dist/webpack:/onnxruntime-common/lib/inference-session-impl.ts:112:28)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at sessionRun (node_modules/@xenova/transformers/src/models.js:52:18)
    at Function._call (node_modules/@xenova/transformers/src/models.js:365:16)
    at Function._call (node_modules/@xenova/transformers/src/pipelines.js:69:23)
    at Function._call (node_modules/@xenova/transformers/src/pipelines.js:351:33)
    at test.js:10:17

This appears to be related to the choice of the ONNX runtime. If this line: https://github.com/xenova/transformers.js/blob/main/src/backends/onnx.js#L10 is changed to onnxruntime-web (instead of onnxruntime-node). Executing under jest will now succeed, so there appears to be some issue with jest + onnxruntime-node. In terms of resolution, one option would be to detect within backends/onnx.js whether execution is being done under jest, which can be done by checking process.env.JEST_WORKER_ID (this will be populated when running under jest). In terms of root cause, I'm not sure where the actual bug is (jest or onnxruntime-node), but it would make the most sense for it to be resolved there if its possible to determine which package is responsible.

Steps to reproduce:

• create new directory
• copy test script to file named test.js
• npm install @xenova/transformers jest
• node test.js
• npx jest test.js

Currently, the default model is gpt2 for text generation. It isn't good and I could not configure it to work correctly.

I used distilgpt2 and it works great out of the box. I want to create a PR and change it. @xenova What do you think?

Running the following:

global.self = global;

const { pipeline, env } = require("@xenova/transformers");
env.onnx.wasm.numThreads = 1;
(async()=> {
        let embedder = await pipeline('embeddings', 'sentence-transformers/all-MiniLM-L6-v2')
        let sentences = [
            'The quick brown fox jumps over the lazy dog.'
        ]
        let output = await embedder(sentences)
        console.log(output[0][0]);
})();

and this (installable via pip install sentence-transformers):

from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("all-MiniLM-L6-v2")
query = model.encode("The quick brown fox jumps over the lazy dog.", convert_to_tensor=True)
print(query[0])

results in different values (-0.07890480756759644 and 0.0439). Any idea why the embedding values are different? I also noticed that the cosine similarity values for the same two sentences between Python and transformers.js was also quite a bit different (which is likely caused by the difference in embedding values).

Getting this error when running the embeddings pipeline:

Error: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=true' and 'truncation=true' to have batched tensors with the same length.
    at Function._call (tokenizers.js?dd68:928:1)
    at Function.closure [as tokenizer] (utils.js?aa0e:362:44)
    at Function._call (pipelines.js?5d72:46:1)
    at Function._call (pipelines.js?5d72:263:1)
    at closure (utils.js?aa0e:362:44)
    at _callee$ (utils.ts?2b07:20:28)
    at tryCatch (runtime.js?7efe:45:16)
    at Generator.invoke [as _invoke] (runtime.js?7efe:274:1)
    at prototype.<computed> [as next] (runtime.js?7efe:97:1)
    at asyncGeneratorStep (utils.ts?2b07:1:53)

I read the source code and saw that padding and truncation are set to true by default, so not sure how this is happening. Any ideas?

Describe the bug
A clear and concise description of what the bug is.

Helsinki multilingual models (Helsinki-NLP/opus-mt-en-mul and Helsinki-NLP/opus-mt-mul-en) require a specific token in the shape of >>id<< as per the models documentation. However when using those tokens will cause the models.js file to throw an error:

models.js:73 An error occurred during model execution: "Error: failed to call OrtRun(). error code = 6.".

How to reproduce
Steps or a minimal working example to reproduce the behavior

const pipe = await pipeline('translation','Helsinki-NLP/opus-mt-en-mul')
const result = await pipe(">>jpn<< I love transformer.js, it's a wonderful library");

Expected behavior
A clear and concise description of what you expected to happen.

I expect to get the translated text

Logs/screenshots
If applicable, add logs/screenshots to help explain your problem.

Environment

• Transformers.js version: 1.4.0
• Browser (if applicable): Arc 0.96.0
• Operating system (if applicable): macOS Ventura 13.2
• Other:

Additional context
Add any other context about the problem here.

It should be noted that the models do not error without the >>id<< token and run just fine without them (although the translation is obviously wrong because the source/target language wasn't specified)

Name of the feature
Speech to text using SpeechT5, which was recently added to Transformers.

• Blog post: https://huggingface.co/blog/speecht5
• Spaces demo: https://huggingface.co/spaces/Matthijs/speecht5-tts-demo
• Models: https://huggingface.co/mechanicalsea/speecht5-tts
• Github repo: https://github.com/microsoft/SpeechT5/
• Paper: https://arxiv.org/abs/2110.07205
• Speaker embedding creation: https://huggingface.co/mechanicalsea/speecht5-vc/blob/main/manifest/utils/prep_cmu_arctic_spkemb.py

Reason for request
The brower's default TTS API is quite bad if you want to create an experience that works nicely across all browsers. Firefox's voices in particular are extremely robotic. Some applications require that the voice is consistent, and of a particular style/tone/etc. SpeechT5 allows you to create 512-dim speaker embeddings so you can use an arbitrary voice style.

Additional context

• The model runs in realtime on the CPU (Pytorch), so with WebGPU we should easily have realtime generation on the web.
• According the the above-linked models repo, the models are 600M (T5) and 300M (Hi-Fi-GAN), but I've just tried running it locally with the new docker integration on Hugging Face and it downloads a 585M model and a 50M model. So I'm not sure what's going on with the GAN size difference. Maybe they have quantized the GAN, but not T5? Hoping tha the T5 model can be quantized because that would move it from "reasonable" to "good" territory in terms of size. I'm assuming that it's currently in 16 bit format.

Example clip from the Spaces demo (this embedding is pretty monotone):

On longer audio files (e.g. > 1 minute or so), I get this error:

Uncaught (in promise) Error: There was an error while processing timestamps, we haven't found a timestamp as last token.
    at Function._decode_asr (tokenizers.js:1497:23)
    at Function._call (pipelines.js:489:56)

My worker code is as this:

importScripts('https://cdn.jsdelivr.net/npm/@xenova/transformers/dist/transformers.min.js');
async function speech_to_text(data) {
	let pipe = await pipeline('automatic-speech-recognition','openai/whisper-tiny.en');
    return await pipe(data.audio, {
		top_k: 0,
		do_sample: false,
		chunk_length_s: 30,
		stride_length_s: 5,
		return_timestamps: true,
		force_full_sequences: false,
    })
}

Is there anything I can do to prevent this error?

Any plans to add webgpu support ?

I tested with React but got error
Code:

import { pipeline } from "@xenova/transformers";
....
const classifier = await pipeline("translation");

Error:

tokenizers.js:1080 Uncaught (in promise) TypeError: text.split is not a function
    at Function._encode_text (tokenizers.js:1080:1)
    at Function.encode (tokenizers.js:1105:1)
    at tokenizers.js:970:1
    at Array.map (<anonymous>)
    at Function._call (tokenizers.js:970:1)
    at Function.closure [as tokenizer] (utils.js:372:1)
    at Function._call (pipelines.js:214:1)
    at closure (utils.js:372:1)
    at basicStateReducer (react-dom.development.js:16540:1)
    at updateReducer (react-dom.development.js:16664:1)

I am trying to duplicate your example service worker code here in order to run the models outside of the main thread: https://github.com/xenova/transformers.js/blob/main/assets/js/worker.js

However, my codebase is written in Typescript rather than Javascript, and I get the error Cannot find name 'env' because you set the path to the wasm files like so: env.onnx.wasm.wasmPaths = DIST_DIR;.

Where is the env object initialized?

Hello!
Thanks for making this great repository!
I was trying to implement machine translation and found that any language other than en, de, and ro does not work. Might it be possible to implement a more multilingual t5 model, such as Google's multilingual t5?
Thank you!

Is there a reason why usage of several generation config params is commented out in the _get_logits_processor function on line 370 of models.js?

_get_logits_processor(
        generation_config,
        input_ids_seq_length,
        // encoder_input_ids, TODO
        // prefix_allowed_tokens_fn, TODO
        logits_processor = null
    ) {
        const processors = new LogitsProcessorList();

        // if (generation_config.diversity_penalty !== null && generation_config.diversity_penalty > 0.0) {
        //     processors.push(new HammingDiversityLogitsProcessor(
        //         generation_config.diversity_penalty,
        //         generation_config.num_beams,
        //         generation_config.num_beam_groups
        //     ));
        // }

I've tested on Node 18 and latest Bun, both don't work.

it would be great to have a rephrase task. Mainly to use for rephrasing ChatGPT answers :)

Sorry for this noob question, can somebody give me a kind of guideline to be able to convert and use

https://huggingface.co/Helsinki-NLP/opus-mt-ru-en/tree/main

thank you

Would it be possible to add GPT-Neo as an available model? I believe it is pretty similar to GPT-2, as they both use GPT2Tokenizer, so I don't think it should be too difficult to implement. It would also be pretty cool to have a model that knows things like quantum entanglement since it was trained on The Pile.

Describe the bug
A clear and concise description of what the bug is.

How to reproduce
Steps or a minimal working example to reproduce the behavior
Audio with Hindi having transcribing stuck on X %

Expected behavior
A clear and concise description of what you expected to happen.

It should generate subtitles

Logs/screenshots
If applicable, add logs/screenshots to help explain your problem.
Screenshot attached, app name is screenrun.app

Environment

• Transformers.js version:
• Browser (if applicable):
• Operating system (if applicable):
• Other:

Additional context
Add any other context about the problem here.

To allow proper collaboration and healthiness we need to reduce some current technical debt. Something like CML. Disclaimer: I did that

• linter
• unit tests
• Github actions checks
• Github actions releases

Hello,
This project is really cool. I was wondering if it's possible to use the BlenderBot model.
My current code gives me an error:

let pipe = await pipeline('text-generation', model = 'facebook/blenderbot_small-90M')
let out = await pipe("Prompt");
console.log(out);

GET https://huggingface.co/Xenova/transformers.js/resolve/main/quantized/facebook/blenderbot_small-90M/causal-lm-with-past/tokenizer.json 404
Uncaught (in promise) Error: File not found. Could not locate "https://huggingface.co/Xenova/transformers.js/resolve/main/quantized/facebook/blenderbot_small-90M/causal-lm-with-past/tokenizer.json".

I saw that this was added in your Hugging Face repository, but I was wondering what the correct way to implement this is.
Thank you!

I got this error when trying:

TypeError [ERR_WORKER_PATH]: The worker script or module filename must be an absolute path or a relative path starting with './' or '../'. Received "blob:nodedata:....

Hi,

I love this library and I use it a lot, thank you for the effort.
Is there a plan to add types to this package? I t could help us understand how to use and configure the different pipelines.
Is there other documentation? Since the HuggingFace one is more general and less specific about this lib.

When trying to use the flan-t5-base model in place of the default model text-completion, the pipeline fails:

Error: File not found. Could not locate "/models/onnx/quantized/google/flan-t5-base/causal-lm-with-past/tokenizer_config.json".

I am running the flan-t5 model for text2text-generation (e.g. await pipeline("text2text-generation", "flan-t5-base");) in a service worker. Inference runs without an issue for the first several runs, but eventually I get this error:

RangeError: offset is out of bounds
    at Uint8Array.set (<anonymous>)
    at e.createSessionAllocate (webpack-internal:///../../node_modules/onnxruntime-web/dist/ort-web.min.js:7:450663)
    at Object.e.createSession (webpack-internal:///../../node_modules/onnxruntime-web/dist/ort-web.min.js:7:451415)
    at e.createSession (webpack-internal:///../../node_modules/onnxruntime-web/dist/ort-web.min.js:7:443678)
    at e.OnnxruntimeWebAssemblySessionHandler.loadModel (webpack-internal:///../../node_modules/onnxruntime-web/dist/ort-web.min.js:7:446572)
    at Object.createSessionHandler (webpack-internal:///../../node_modules/onnxruntime-web/dist/ort-web.min.js:7:156408)
    at Function.create (webpack-internal:///../../node_modules/onnxruntime-common/dist/lib/inference-session-impl.js:176:39)
    at async constructSession (webpack-internal:///../../node_modules/@xenova/transformers/src/models.js:20:19)
    at async Promise.all (index 2)
    at async Function.from_pretrained (webpack-internal:///../../node_modules/@xenova/transformers/src/models.js:102:75)

I ran with the same input params 5 times in a row and still get the error, so it doesn't seem like it's an issue with an invalid input value.

When running the script:

python ./scripts/convert.py --model_id philschmid/flan-t5-base-samsum --from_hub --quantize --task seq2seq-lm

I get the following error:

TypeError: quantize_dynamic() got an unexpected keyword argument 'activation_type'

It works fine without the --quantize argument.

Currently the library only operates with the WASM backend even in nodejs. However the better approach should be use node bindings in node and add the webgl EP to automatically fallback to wasm if webgl is not supported by the model.

Currenly in code:

let session = await InferenceSession.create(buffer, {
  // executionProviders: ["webgl"]
  executionProviders: ["wasm"]
});

Hack to at least improve in the web

let session;

try {
    session = await InferenceSession.create(buffer, {
        executionProviders: ['webgl', 'wasm']
    });
} catch (err) {
    session = await InferenceSession.create(buffer, {
        executionProviders: ['wasm']
    });
}

I can prepare a PR to improve the execution in node and web 😃

When running the following:
python3.9 ./transformers.js/scripts/convert.py --model_id google/flan-ul2 --from_hub --quantize --task seq2seq-lm-with-past

I receive the following:

Downloading (…)lve/main/config.json: 100%|█████████████████████████████████████████████████████████████| 784/784 [00:00<00:00, 86.3kB/s]
Downloading (…)model.bin.index.json: 100%|█████████████████████████████████████████████████████████| 67.5k/67.5k [00:00<00:00, 3.05MB/s]
Downloading (…)l-00001-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 4.69G/4.69G [00:54<00:00, 85.5MB/s]
Downloading (…)l-00002-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 4.97G/4.97G [01:02<00:00, 79.1MB/s]
Downloading (…)l-00003-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 4.97G/4.97G [01:05<00:00, 75.9MB/s]
Downloading (…)l-00004-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 4.96G/4.96G [01:08<00:00, 72.5MB/s]
Downloading (…)l-00005-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 5.00G/5.00G [01:02<00:00, 79.5MB/s]
Downloading (…)l-00006-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 4.93G/4.93G [01:32<00:00, 53.3MB/s]
Downloading (…)l-00007-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 5.00G/5.00G [01:39<00:00, 50.4MB/s]
Downloading (…)l-00008-of-00008.bin: 100%|█████████████████████████████████████████████████████████| 4.93G/4.93G [02:10<00:00, 37.9MB/s]
Killed 

Subsequent runs of the script will simply output Killed after about 15s.

Is there something I can do to fix this? I am running ubuntu 18.04 on x86 arch.

I'm trying to use this library to run whisper in a browser environment, outside of a webworker, using NextJS. However, because of this line, I get an error.

Here's how I triggered it:

      const pipe = await pipeline(
        "automatic-speech-recognition",
        "openai/whisper-base"
      );
      const out = await pipe(
        "https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/mlk.flac"
      );

Code sandbox

I am new to both transformers.js and whisper, so I am sorry for a lame question in advance.

I am trying to make return_timestamps parameter work...

I managed to customize script.js from transformer.js demo locally and added data.generation.return_timestamps = "char"; around line ~447 inside GENERATE_BUTTON click handler in order to pass the parameter. With that change in place I am seeing timestamp appears as chunks (result var in worker.js):

{
    "text": " And so my fellow Americans ask not what your country can do for you ask what you can do for your country.",
    "chunks": [
        {
            "timestamp": [0,8],
            "text": " And so my fellow Americans ask not what your country can do for you"
        },
        {
            "timestamp": [8,11],
            "text": " ask what you can do for your country."
        }
    ]
}

however the chunks are not "char level" granular as expected following the return_timestamps doc.

I am looking for ideas how to achieve char/word level timestamp granularity with transform.js and whisper. Do some models/tools need to be updated and/or rebuild?

When trying to run the models from a mobile browser it fails.

Mobile browsers have limited storage for webpages (let's assume 100MB since it depends on the browser and smartphone).
When using smaller models like whisper-tiny (STT) or distilbert-base-uncased (QA) it works.

Any ideas on how can we solve it?

I was thinking about pruning big models and having a less than 100MB version for them like Tensorflow Lite. What do you think?