microsoft / customvision-tfjs Goto Github PK

View Code? Open in Web Editor NEW

28.0 12.0 20.0 354 KB

NPM package for TensorFlow.js models exported from Custom Vision Service

Home Page: https://customvision.ai

License: MIT License

JavaScript 96.45% HTML 3.55%

customvision-tfjs's Introduction

customvision-tfjs

NPM package for TensorFlow.js models exported from Custom Vision Service

This package is for web browsers. If you are looking for a library to run on Node.js environment, please use customvision-tfjs-node.

Install

npm install @microsoft/customvision-tfjs

Or, if you would like to use CDN,

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js"></script>
<script src="https://unpkg.com/@microsoft/[email protected]"></script>

Usage

<img id="image" src="test_image.jpg" />

Classification

import * as cvstfjs from '@microsoft/customvision-tfjs';

let model = new cvstfjs.ClassificationModel();
await model.loadModelAsync('model.json');
const image = document.getElementById('image');
const result = await model.executeAsync(image);

The result is a 1D-array of probabilities.

Object Detection

import * as cvstfjs from '@microsoft/customvision-tfjs';

let model = new cvstfjs.ObjectDetectionModel();
await model.loadModelAsync('model.json');
const image = document.getElementById('image');
const result = await model.executeAsync(image);

The result has 3 arrays.

[
	[[0.1, 0.3, 0.4, 0.3], [0.2, 0.4, 0.8, 0.9]], // bounding boxes (x1, y1, x2, y2)
	[0.2, 0.3], // probabilities
	[1, 4] // class ids
]

References

sample scripts

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

customvision-tfjs's People

Contributors

Stargazers

Watchers

Forkers

stevestrongapp turgunyusuf gsbecerrag bhaskers-blu-org2 soulpancake000 ankuraxz taffywrinkle claudiusgonzo drmanhatin yonet sojet standardgalactic fishcharlie ninexys sergiogonbar test-mass-forker-org-1 abubakrilaitan simplesoftmx muskanf

customvision-tfjs's Issues

model not found

I am provided path to my model but its not recognising

Doesn't work with new version of Node JS

Nothing works with new version of Node JS.

I fixed few errors to run the code , then finally got Javascript “Not a Constructor” in below line:

let model = new cvstfjs.ObjectDetectionModel();

Kindly check.

Memory leak in creation of tensor

Hi,

There's a memory leak in this library. Happens after a few dozens of tensors have been created. I can show you on teams if you like, just send me a message.

My (working) implementation of tensorflowjs is accessible at https://github.com/drmanhatin/CustomVisionRealTime/blob/main/nodejsdemo/public/main.js.

Most important part:

   let t4d = tf.tidy(() => {
          const image = tf.browser.fromPixels(canvas).resizeBilinear([320, 320])
          return tf.tensor4d(image.dataSync(), [1, 320, 320, 3])
  })

Wrapping with tidy ensures automatic cleanup.

Should loadModelAsync use tf.loadLayersModel instead of tf.LoadGraphModel()?

I am experiencing a small issued with a model, when getting load I get: Uncaught (in promise) TypeError: Cannot read property 'producer' of undefined

Custom Vision TFJS Models are now incompatible and throw errors

Literally everything is broken.. try it out, the tfjs models being exported by custom vision are completely different than what your code is expecting. Specifically the postprocessing method is trying to index into the model, but the model's structure is different and so it tries to get index 0 of null at some point.

Object Detection not working in Browser

I trained an Object detection model using Customvision.ai web portal and exported the model to Tensorflow.js.

I am following the code on the Github Repository https://github.com/microsoft/customvision-tfjs to use it in Browser but it's not working.

Can anyone help me with how to use it Model in Browser ?

Code that i am using :

<html>

<head>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/dist/tf.min.js"></script>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <script src="https://unpkg.com/@microsoft/customvision-tfjs"></script>
</head>

<body>
<img id="iamge "src="box.jpg" style="display: none;"/>
</body>
<script>
    import * as cvstfjs from '@microsoft/customvision-tfjs';

    let model = new ObjectDetectionModel();
    await model.loadModelAsync('http://localhost/od/model.json');
    const image = document.getElementById('image');
    const result = await model.executeAsync(image);
    alert(result)
</script>

</html>

Option To Load the complete Model

Currently We are Loading the Model all time to get the results. Using this for every frame of video is not good option.

Provide a option to Load the Model on Page Load.

Then use the Model to make real time predictions like Object Tracking in a Video.

my model only detect 10 objects, how can i detect more object

i want to detect all objects, like image, i have 28 objects, but now only can detect 10 objects
how can i detect all objects, thank you~

my code:

import * as cvstfjs from '@microsoft/customvision-tfjs';

let model = new cvstfjs.ObjectDetectionModel();
await model.loadModelAsync('http://127.0.0.1:7160/public/customvision-modal/model.json');

const image = document.getElementById('img');
const predictions = await model.executeAsync(image);

This model execution did not contain any nodes with control flow or dynamic output shapes

I use customvision-tfjs to do object detection, the log reports a warning:
[WARNING] :This model execution did not contain any nodes with control flow or dynamic output shapes. You can use model.execute() instead
I don't know the meaning and if model.execute() has a higher performance ?

Typescript: Could not find a declaration file for module '@microsoft/customvision-tfjs'.

Currently, there are no Typescript Declaration for this library

It would be very helpful to have