webmachinelearning / webnn-samples Goto Github PK

Please follow https://github.com/webmachinelearning/webnn-polyfill/tree/master/.github

We need:

1. update the version of tools in actions
2. add dependabot so it can automatically update the versions

Add a live example based on the code example in WebNN spec

Repro Steps:

1. Open Developer Tools -> Console
2. Visit https://webmachinelearning.github.io/webnn-samples/nnotepad/
3. Click "Show generated code" or "Show documentation" button
4. Click "Close" button

Actual Result:
e.target.close is not a function

The TinyYOLO is one of the first wave models.

The NSNet is one of the first wave models.

Repro Steps:

1. Launch latest Chrome Canary on Windows 11
   C:\Users\AppData\Local\Google\Chrome SxS\Application\chrome.exe" --enable-features=WebMachineLearningNeuralNetwork

2. Visit https://webmachinelearning.github.io/webnn-samples/style_transfer/index.html

3. Choose webnn-cpu backend

Actual Result:

    DOMException: Failed to execute 'build' on 'MLGraphBuilder': XNNPACK only supports constant padding mode.

Environment:

Chrome Canary: 125.0.6381.0

The bias is 1D tensor that isn't supported in WebNN, please reshape it to 2D like native implementation

https://github.com/webmachinelearning/webnn-samples/blob/master/image_classification/mobilenet_nchw.js#L44
const bias = await buildConstantByNpy(this.builder_, biasName);

This is required by webmachinelearning/webnn#226

To implement this pipeline, the semantic segmentation sample probably could execute following steps:

1. Use mediacapture-transform to capture the VideoFrame
2. Import the VideoFrame into a WebGPU texture (GPUExternalTexture)
3. Execute WebGPU shader to do pre-processing and convert the GPUExternalTexture to a GPUBuffer
4. Compute the MLGraph with converted GPUBuffer and the output is another GPUBuffer
5. Execute WebGPU shader to do post-processing on the output GPUBuffer and render the result to a canvas.
6. Create a VideoFrame from the CanvasImageSource and enqueue to the mediacapture-transform API's controller.

The MobileNet is one of the first wave models.

- [ ] Image classification (ResNet50v2 NHWC model)

• Fast style transfer

Refer to latest Spec change webmachinelearning/webnn#355

Align with webmachinelearning/webnn#206:
ClampOptions {
float minValue;
float maxValue;
};

Use the fused activation of conv2d and batchNormalization that was introduced by webmachinelearning/webnn#188

Related webnn-polyfill PR: webmachinelearning/webnn-polyfill#98

Hi,

I am looking for Face detection and recognition samples migrated to new repository, but i don't see any specific samples there, and i tried samples from old repository ([https://github.com/intel/webml-polyfill [github.com]) but those are crashing in the live camera case. we are looking for latest libraries which support FR/FD functionality to use in our project. so anything available immediately is of great help.

Regards
Roshan

https://github.com/webmachinelearning/webnn-samples/actions/runs/873331801

@Honry , please help fix this issue. Thanks!

Implement the changes introduced by webmachinelearning/webnn#174

The changes include:

1. Change MLGraphBuilder.build and MLGraph.compute to sync API.
2. Change MLGraph.compute to require pre-allocated output buffers.
3. Simplify the the input / output resources binding by MLResource. Leave MLInput only for dynamic input shape case.

The webnn-polyfill PR webmachinelearning/webnn-polyfill#85 is ready to merge.

Please adapt to that change before we merge that PR which will break the samples. Thanks!

This sample would be according to WebNN semantic segmentation use case.

The WebNN reshape spec is updated to use null to indicate the dimension that needs to be calculated: https://www.w3.org/TR/webnn/#api-mlgraphbuilder-reshape

We need to fix samples which use reshape with old -1.

webmachinelearning/webnn#108

The ResNet is one of the first wave models.

It would be good that users can run samples with different device by configuring the device preference.

https://webmachinelearning.github.io/webnn/#enumdef-mldevicepreference

For example, allow to select 'cpu', 'gpu' or 'default' for image classification sample and others.

The SqueezeNet is one of the first wave models.

Test: https://webmachinelearning.github.io/webnn-samples/style_transfer/
Error:

Both YOLO and SSD models are referred in WebNN object detection use case. Now the object detection sample supports TinyYOLOV2. It would be good to support SSD as well.

Repro Steps:

1. Launch latest Chrome Canary on Windows 11
   C:\Users\AppData\Local\Google\Chrome SxS\Application\chrome.exe" --enable-features=WebMachineLearningNeuralNetwork
2. Visit https://webmachinelearning.github.io/webnn-samples/lenet/index.html
3. Choose backend webnn-cpu and click Clear button.
4. Click Predict button.

Actual Result:

    When the number in the picture are cleared, it can still be predicted.
Environment:

Chrome Canary: 125.0.6381.0

Tried the sample here: https://webmachinelearning.github.io/webnn-samples/lenet/index.html and got the incorrect prediction for the handwritten number 7

Repro Steps:

1. Launch latest Chrome Canary on Windows 11 without WebNN flag
   "C:\Users\AppData\Local\Google\Chrome SxS\Application\chrome.exe"

2. Visit https://webmachinelearning.github.io/webnn-samples/nsnet2/index.html

3. Choose wasm backend

Actual Result:

    Error: Kernel 'Complex' not registered for backend 'wasm'.

Environment:

Chrome Canary: 125.0.6381.0

webmachinelearning/webnn#149 introduces

• MLContext
• MLGraphBuilder
• MLGraph

The corresponding webnn-polyfill issue webmachinelearning/webnn-polyfill#42 and PR webmachinelearning/webnn-polyfill#46

Repro Steps:

1. Launch latest Chrome Canary on Windows 11
   C:\Users\AppData\Local\Google\Chrome SxS\Application\chrome.exe" --enable-features=WebMachineLearningNeuralNetwork

2. Visit https://webmachinelearning.github.io/webnn-samples/nsnet2/index.html

   or https://webmachinelearning.github.io/webnn-samples/rnnoise/index.html

3. Choose webnn-gpu backend

Actual Result:

    DOMException: Failed to execute 'build' on 'MLGraphBuilder': DirectML: This operator (gru) is not supported.

Environment:

Chrome Canary: 125.0.6381.0

Host the LeNet example at https://webmachinelearning.github.io/webnn-samples/lenet/

Chromium prototype now supports gru op in TFLite backend for CPU. The nsnet2 and rnnoise samples should be able to work on WebNN CPU backend.

The RNNoise is one of the first wave models.

Steps to reproduce:

1. Load https://webmachinelearning.github.io/webnn-samples/code/?example=mul_add.js

2. Click Run

3. Console log prints:

Could not get context for WebGL version 2
Output shape: 2,2
Output value: 1,1,1,1

4. Click Run again
5. Console log prints:

Output shape: 2,2
Output value: 1,1,1,1

Expected log messages in 3 and 5 to be identical, including the WebGL error message.

Mozilla/5.0 (iPhone; CPU iPhone OS 14_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Mobile/15E148 Safari/604.1

Normally, when serializing an expression into code, an array [...] is serialized as a constant MLOperand. Since some methods take sequences, there's a WebNNUtil.isNonOperandArg() helper that answers "for method X argument Y, should it be a constant MLOperand?". If this returns true, then a method arg is serialized as a JS array instead.

But this is not applied recursively. This is sufficient for this case:

T = [[1,2],[3,4]]
concat([T, T], 1)

... where concat() wants an array of tensors. But not for:

T = [[1,2],[3,4]]
reshape(data, [1, 2, 2, 1])

... where reshape wants an array of numbers (not scalar constants).

I design to support https://webmachinelearning.github.io/webnn-samples/lenet/?numRuns={n} and update UI by adding 'numRuns' input, show median Execution Time on UI when 'numRuns' is greater than 1.

@huningxin Any suggestion, thanks.

Hi!
Thanks for making this available!

I get an error on the fast style transfer example saying:
"the value length 874800 is invalid, size of [1,3,33,33] 3267 is expected." and it never starts. This is on Chrome (also Canary) and Firefox, Mac OSX and POP_OS.

In the console it also shows: "Error with Permissions-Policy header: Unrecognized feature: 'interest-cohort'" but this may be unrelated.

Thanks,
Dirk

To align with latest spec: webmachinelearning/webnn#323

Enable the CI for lint as the first step.

I tested running the sample examples locally but the following errors are coming both in MacOS and in Windows :

1. Local VS Code Console Error :
   

2. Browser Error :
   

Both of them are for the missing kernels. Do we have to download the weights separately for each of the projects?

Hi,
Can we add fp16 version of the models? (have all inputs and weights be fp16).
On Mac, CoreML only use NPU when the data is fp16. It would be valuable to have webnn samples with fp16 precision to compare the performance.

Repro Steps:

1. Launch latest Chrome Canary on Windows 11
   C:\Users\AppData\Local\Google\Chrome SxS\Application\chrome.exe" --enable-features=WebMachineLearningNeuralNetwork

2. Visit https://webmachinelearning.github.io/webnn-samples/nsnet2/index.html

   or https://webmachinelearning.github.io/webnn-samples/rnnoise/index.html

3. Choose webnn-cpu backend

Actual Result:

    DOMException: Failed to execute 'build' on 'MLGraphBuilder': XNNPACK can't support broadcasting for matrix multiply.

Environment:

Chrome Canary: 125.0.6381.0

Repro:

1. Load https://webmachinelearning.github.io/webnn-samples/nnotepad/index.html
   1, In text field, select everything after a = 123, press backspace/delete

Expected:

• results area updates with scalar 123

Actual:

• results area still shows 2 tensors (result of split() example)

Notes:

• The code adds an "input"event listener to the #input element, which calls refresh(). The Monaco editor must not be firing this for deletion events? Is there a better way to listen to the Monaco editor?

In 2020/10/01 WebML CG call, @pyu10055 suggested

<ping_yu> It would be good to integrate this (code examples) with the spec doc

Currently the NNotepad code uses the Monaco editor with default (?) syntax highlighting, which is okay but not perfect.

Given that NNotepad has a well defined set of tokens and a grammar for parsing, and Monaco has a way to specify custom languages, we should be able to do better.

(Not urgent, this is just a "to-do" item.)

Now that the Chromium prototype supports passing "npu" in the context creation options we need samples that exercise this code path.

Hi, is it possible to get python implementation of NesNet2.

I run it in local and it will show "invalid signature"
my device is Win 11 22H2 and enable WebNN

Refer to WebNN Super Resolution use case: https://webmachinelearning.github.io/webnn/#usecase-super-resolution

Maybe this one in the ONNX Runtime project could be a start point model.