facebookresearch / playtorch Goto Github PK

PlayTorch is a framework for rapidly creating mobile AI experiences.

License: MIT License

Java 6.76% JavaScript 3.16% TypeScript 23.98% Swift 3.98% Ruby 0.19% Objective-C 1.43% Python 0.20% CSS 1.04% C 9.13% Objective-C++ 0.61% CMake 0.27% C++ 13.25% MDX 36.00%

playtorch's Introduction

This project has been archived and is no longer actively maintained.

Build your AI powered mobile prototypes in minutes: Get Started · Tutorials · API

PlayTorch is a set of tools to build AI-powered experiences for mobile.

This monorepo includes the PlayTorch React Native package to run on-device inference with PyTorch Mobile and a React Native template with examples ready to be deployed on mobile devices.

Requirements
Building your first PlayTorch app
How to Contribute
Code of Conduct
License

📋 Requirements

PlayTorch apps may target Android 10.0 (API 21) and iOS 12.0 or newer. You may use Windows, macOS, or Linux as your development operating system.

🎉 Building your first PlayTorch app

Follow the Getting Started guide. We also have a few tutorials for you to keep going after getting started:

📖 Documentation

The full documentation for PlayTorch can be found on our website.

👏 How to Contribute

The main purpose of this repository is to continue evolving PlayTorch. We want to make contributing to this project as easy and transparent as possible, and we are grateful to the community for contributing bug fixes and improvements. Read below to learn how you can take part in improving PlayTorch.

Code of Conduct

Facebook has adopted a Code of Conduct that we expect project participants to adhere to. Please read the full text so that you can understand what actions will and will not be tolerated.

Contributing Guide

Read our Contributing Guide to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes to PlayTorch.

License

PlayTorch is MIT licensed, as found in the LICENSE file.

playtorch's People

Contributors

Stargazers

Watchers

Forkers

yifree hugogresse lipovsek yippeesoft adilbenmoussa alleniver dconstan huangweiboy2 aaronespasa karthikrangasai vinayasathyanarayana tercellzp vmogroup ermankuruoglu sxela mdwelsh ntkzwane srinivest classicvalues leruohq eagol nadleeh00 evemyadzepike aadityaverma sujithln kp-forks jhyuuu ansonsyfang liuyinglao devinan bhadreshpsavani clarksandholtz bousejin dbarbedillo cjfghk5697 raedle zenetio snapbuy justinhaaheim tuskaw thornbirdzhang 9bow maximvlah truong11062002 reedless thepegasusai mldmix noernova jasminesaavedra28 supagonava deyh2020 ngovandau louderthanthunderx1 ikoevska nshutijean gg-big-org nadiamukarram ebgraham vra iainberliner suraj520 jokerhp6789 vladpaunescu mrousavy francismarcus shahidulislamshawon blazjurisic lesliedou shovelingpig phamquyhai xujianming2017 williamngan moises2406 web3aivc mingisa sorokinvld arushikesarwani94 pervrosen duckheada teutades lavrans gk37497 nnnn24 bycedric deepq-1 flyskywhy remod web-logs2 runningmaverick philiip evdcush nhype zixdev phyo6hu hsneed69 exoexpert kbfina-2 nductung

playtorch's Issues

Image Size Error

Version

"react-native-pytorch-core": "^0.1.3"

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

const image = await ImageUtil.fromBundle(require('./models/90uwPhoto.jpg'));
const result = await MobileModel.execute(model, {
image,
});
console.log('Model Results: ' + JSON.stringify(result));

Expected Results

Loads image without error

Code example, screenshot, or link to repository

No response

react-native-pytorch-core build failed on iOs simulator

Version

0.1.3

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

M1 Pro
follow https://pytorch.org/live/docs/tutorials/add-package/
When I use yarn ios, got this error

In .../Pods/LibTorch-Lite/install/lib/libtorch.a(empty.cpp.o), building for iOS Simulator, but linking in object file built for iOS, file '.../ios/Pods/LibTorch-Lite/install/lib/libtorch.a' for architecture arm64

but I can run it on my iPhone 13 with Xcode

Expected Results

package.json

{
 "name": "pytrn2",
 "version": "0.0.1",
 "private": true,
 "scripts": {
   "android": "react-native run-android",
   "ios": "react-native run-ios",
   "start": "react-native start",
   "test": "jest",
   "lint": "eslint ."
 },
 "dependencies": {
   "react": "17.0.2",
   "react-native": "0.68.2",
   "react-native-pytorch-core": "^0.1.3"
 },
 "devDependencies": {
   "@babel/core": "^7.18.0",
   "@babel/runtime": "^7.18.0",
   "@react-native-community/eslint-config": "^3.0.2",
   "babel-jest": "^28.1.0",
   "eslint": "^8.16.0",
   "jest": "^28.1.0",
   "metro-react-native-babel-preset": "^0.70.3",
   "react-test-renderer": "17.0.2"
 },
 "jest": {
   "preset": "react-native"
 }
}

Code example, screenshot, or link to repository

No response

How to use it in 3D-CNN?

Area Select

Other

Description

Thanks for ur contributions and I have some questions about this project:

Q1: Does it works in 3D-CNN?

Q2: Is there a demo for 3D-CNN?

Q3: Does it limit temporal if I use my model (My model needs tensor like [1, 3, 16, 112, 112])

I will appreciate it that you can reply me ^_^

Example with Lightning Flash

Area Select

Other

Description

Dear team from Live,

Thanks for this awesome work,
I believe it would be great to have more examples across more tasks and I believe there could be lot of synergy with Lightning Flash.
Here is repo: https://github.com/PyTorchLightning/lightning-flash

Best regards,
T.C

Auto close with white screen on android

Version

0.1.3

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

No bug shows up by following manual set up tutorial. But when I open the app, the app close itself with an instant white screen.
Two tutorial get the same result.
https://pytorch.org/live/docs/tutorials/get-started-manually/
https://pytorch.org/live/docs/tutorials/image-classification/

I think my device connection is OK, beacuse react native tutorial works fine.
Systems: Ubuntu 20 + python 3.9.7

Expected Results

Get a functional application, not just an app icon.

Code example, screenshot, or link to repository

npx react-native start shows something weired once I try to open app on my android device:

react-native-pytorch-core camera is showing blank screen on Windows environment React Native App

Description

I'm following this tutorial to create a React Native App that can read ptl files for image classification: https://pytorch.org/live/docs/tutorials/image-classification/

I'm already aware that, there more steps than normal to make this work in Windows, atleast according with https://pytorch.org/live/docs/tutorials/get-started-manually/ tutorial;

That should be simple, just open the camera at the very least, but the camera won't work, and doesn't show any kind of error, it's just a blank space instead of the camera.

There is no one using this code, so i need some help;

Images

Website

https://pytorch.org/live/docs/tutorials/image-classification/

Credit (optional)

No response

Source Code (optional)

No response

Unexpected token =

Version

0.1.3

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

torch live didn't work at all.
I tried npx torchlive-cli setup-dev, and nothing happened.

(android) dave@Acer:~/android$ npx torchlive-cli setup-dev
npx: installed 72 in 5.065s
  _                 _     _ _           
 | |_ ___  _ __ ___| |__ | (_)_   _____ 
 | __/ _ \| '__/ __| '_ \| | \ \ / / _ \
 | || (_) | | | (__| | | | | |\ V /  __/
  \__\___/|_|  \___|_| |_|_|_| \_/ \___|
                                        
torchlive version 0.1.3

Next, I used npx torchlive-cli init MyFirstProject. New errors showed up:

npx: installed 72 in 5.365s
  _                 _     _ _           
 | |_ ___  _ __ ___| |__ | (_)_   _____ 
 | __/ _ \| '__/ __| '_ \| | \ \ / / _ \
 | || (_) | | | (__| | | | | |\ V /  __/
  \__\___/|_|  \___|_| |_|_|_| \_/ \___|
                                        
torchlive version 0.1.3
✖ project MyFirstProject (initializing, this can take a few minutes.)
  › Init template react-native-template-pytorch-live

🚨 💥 🚨 💥 🚨

npx: installed 640 in 21.067s
Unexpected token =


💥 Failed to initialize MyFirstProject project. Please run 'npx react-native init <project-name> --template react-native-template-pytorch-live'
for more debug info.

false

If you still run into issues, please refer to https://github.com/pytorch/live/issues for more info.

Then I tried npx react-native init MyFirstProject --template react-native-template-pytorch-live. Errors Unexpected token = occured.

(android) dave@Acer:~/android$ npx react-native init MyFirstProject --template react-native-template-pytorch-live
npx: installed 640 in 25.67s
Unexpected token =

Moreover, npm_config_registry=https://registry.npmjs.org npx torchlive-cli init MyFirstProject also got Unexpetcted token = error.
Some more information by following #21:

(android) dave@Acer:~/android$ npm list -g react-native
/usr/local/lib
└── (empty)
(android) dave@Acer:~/android$ npm list -g --depth=0
/usr/local/lib
└── (empty)
(android) dave@Acer:~/android$ yarn global list
00h00m00s 0/0: : ERROR: [Errno 2] No such file or directory: 'global'

Systems

My system is Ubuntu 21 + python 3.9.7.

Attempt on Windows 10

Well, I also tried it on win10 + python 3.9.9. Erros seem to live with me lol:

(android) PS D:\Complier\android> npx torchlive-cli setup-dev
npx: installed 72 in 23.329s
  _                 _     _ _
 | |_ ___  _ __ ___| |__ | (_)_   _____
 | __/ _ \| '__/ __| '_ \| | \ \ / / _ \
 | || (_) | | | (__| | | | | |\ V /  __/
  \__\___/|_|  \___|_| |_|_|_| \_/ \___|

torchlive version 0.1.3

(android) PS D:\Complier\android> npx torchlive-cli init MyFirstProject
npx: installed 72 in 10.347s
  _                 _     _ _
 | |_ ___  _ __ ___| |__ | (_)_   _____
 | __/ _ \| '__/ __| '_ \| | \ \ / / _ \
 | || (_) | | | (__| | | | | |\ V /  __/
  \__\___/|_|  \___|_| |_|_|_| \_/ \___|

torchlive version 0.1.3
(node:3308) UnhandledPromiseRejectionWarning: Error: platform win32 unsupported
    at Object.getSDKPath (C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\android\AndroidSDK.js:65:15)
    at Object.getEnv (C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\utils\SystemUtils.js:63:39)
    at InitTask.isValid (C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\cli-commands\Init.js:69:72)
    at C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\utils\TaskUtils.js:64:77
    at Array.filter (<anonymous>)
    at Object.<anonymous> (C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\utils\TaskUtils.js:64:40)
    at step (C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\utils\TaskUtils.js:41:23)
    at Object.next (C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\utils\TaskUtils.js:22:53)
    at C:\Users\MNE079037\AppData\Roaming\npm-cache\_npx\3308\node_modules\torchlive-cli\build\utils\TaskUtils.js:16:71
    at new Promise (<anonymous>)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:3308) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:3308) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Expect

I want to deploy my deep learning models on android device. I discover that torch live is fancy and can help me realize these features. However, the configuration got some bugs difficult to fix. Deploying deep learning models only on Ubuntu is enough. Thanks for your possible advice to solve these problems.
I saw Check out the manual installation process if your operating system is Windows or Linux or if the PyTorch Live command line tool torchlive-cli fails for you! So hope there will have fast tutorials for Linux and windows.

Semantic Segmentation on RTX 3060ti is very slow

Description

I'm using RTX 3060ti for semantic Segmentation , it takes long time to segment out object ( inference on video)
2 minutes video covering in 10 minutes.

How can I speed up my model ?

Images

Website

https://github.com/maudzung/TTNet-Real-time-Analysis-System-for-Table-Tennis-Pytorch

Credit (optional)

No response

Source Code (optional)

No response

the app crashed in the release mode

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Hi, the app is crashing on my phone after release mode, but it is fine in debug mode.

Env:Android 11,Moto G100
Logs:
12-09 16:27:02.492 19401 10332 E AndroidRuntime: FATAL EXCEPTION: pool-2-thread-1
12-09 16:27:02.492 19401 10332 E AndroidRuntime: Process: org.pytorch.live.example, PID: 19401
12-09 16:27:02.492 19401 10332 E AndroidRuntime: com.facebook.jni.CppException: open file failed because of errno 2 on fopen: No such file or directory, file path: /models_resnet18
12-09 16:27:02.492 19401 10332 E AndroidRuntime: Exception raised from RAIIFile at ../caffe2/serialize/file_adapter.cc:27 (most recent call first):
12-09 16:27:02.492 19401 10332 E AndroidRuntime: (no backtrace available)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at org.pytorch.LiteNativePeer.initHybrid(Native Method)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at org.pytorch.LiteNativePeer.(LiteNativePeer.java:28)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at org.pytorch.LiteModuleLoader.load(LiteModuleLoader.java:19)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at org.pytorch.rn.core.ml.MobileModelModule.fetchCacheAndLoadModel(MobileModelModule.java:165)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at org.pytorch.rn.core.ml.MobileModelModule.lambda$preload$0$MobileModelModule(MobileModelModule.java:70)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at org.pytorch.rn.core.ml.-$$Lambda$MobileModelModule
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
12-09 16:27:02.492 19401 10332 E AndroidRuntime: at java.lang.Thread.run(Thread.java:923)

Expected Results

No response

Code example, screenshot, or link to repository

No response

error during cli setup

Version

No response

Problem Area

CLI

Steps to Reproduce

Error project "MyFirstProject" is created but we ran into the following i…
‎ ‎ ‎ ‎
ERROR: Could not find a version that satisfies the requirement torch==1.1…
ERROR: No matching distribution found for torch==1.10.0
‎ ‎ ‎ ‎
please rerun
‎ ‎ ‎ ‎ • "python make_models.py" in the MyFirstProject/models/ directory…
‎ ‎ ‎ ‎ • "pod install" in the MyFirstProje

ct/ios/ directory
if the issue persists, please report at https://github.com/pytorch/live

Expected Results

I was hoping to be able to install all necessary requirements via cli command prompt

Code example, screenshot, or link to repository

No response

Cannot run tutorial project in Get Started Manually on Windows

Version

0.13.0

Problem Area

Website/Documentation

Steps to Reproduce

First, try to run the project on an Android device:

npx react-native init MyFirstProject --template react-native-template-pytorch-live
cd MyFirstProject
npx react-native run-android
Then I find the app installed successfully on the device, but fails while bundling.
The output goes like: (the following error message is obtained after manually running python make_models.py)

error: Error: Unable to resolve module ../../models/detr_resnet50.ptl from C:\Users\t4rf9\PyTorchLiveProject\src\examples\ObjectDetectionExample.tsx:

None of these files exist:
  * detr_resnet50.ptl
  * models\detr_resnet50.ptl\index(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
  25 | const modelInfo: ModelInfo = {
  26 |   name: 'DETR',
> 27 |   model: require('../../models/detr_resnet50.ptl'),
     |                   ^
  28 | };
  29 |
  30 | const objectColors = [
    at ModuleResolver.resolveDependency (C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\node-haste\DependencyGraph\ModuleResolution.js:211:15)
    at DependencyGraph.resolveDependency (C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\node-haste\DependencyGraph.js:413:43)
    at Object.resolve (C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\lib\transformHelpers.js:317:42)
    at resolve (C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:629:33)
    at C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:645:26
    at Array.reduce (<anonymous>)
    at resolveDependencies (C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:644:33)
    at C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:329:33
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (C:\Users\t4rf9\PyTorchLiveProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:137:24)

Then, in an attempt to fix this, I run python make_models.py within models/，but fails within export_image_segmentation_models(). Other 3 export_xxx_models() functions returns normally.

By the way, the torch, torchvision, torchaudio and scipy version listed in requirements.txt is unaccessible via pip, and is only availble via conda. If the versions of the packages don't match those in requirements.txt, the error in https://github.com/pytorch/live/issues/51 will happen. However, the required versions of packages still give an error.

The error message goes:

(PyTorchLive) PS C:\Users\t4rf9\PyTorchLiveProject\models> python .\make_models.py
torch version 1.10.0
Exporting image segmentation models
Traceback (most recent call last):
  File "C:\Users\t4rf9\PyTorchLiveProject\models\make_models.py", line 124, in <module>
    main()
  File "C:\Users\t4rf9\PyTorchLiveProject\models\make_models.py", line 118, in main
    export_image_segmentation_models()
  File "C:\Users\t4rf9\PyTorchLiveProject\models\make_models.py", line 57, in export_image_segmentation_models
    model = torch.hub.load(
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torch\hub.py", line 399, in load
    model = _load_local(repo_or_dir, model, *args, **kwargs)
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torch\hub.py", line 428, in _load_local
    model = entry(*args, **kwargs)
  File "C:\Users\t4rf9/.cache\torch\hub\facebookresearch_detr_main\hubconf.py", line 32, in detr_resnet50
    model = _make_detr("resnet50", dilation=False, num_classes=num_classes)
  File "C:\Users\t4rf9/.cache\torch\hub\facebookresearch_detr_main\hubconf.py", line 15, in _make_detr
    backbone = Backbone(backbone_name, train_backbone=True, return_interm_layers=mask, dilation=dilation)
  File "C:\Users\t4rf9/.cache\torch\hub\facebookresearch_detr_main\models\backbone.py", line 89, in __init__
    backbone = getattr(torchvision.models, name)(
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torchvision\models\resnet.py", line 300, in resnet50
    return _resnet('resnet50', Bottleneck, [3, 4, 6, 3], pretrained, progress,
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torchvision\models\resnet.py", line 262, in _resnet
    state_dict = load_state_dict_from_url(model_urls[arch],
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torch\hub.py", line 590, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location)
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torch\serialization.py", line 600, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "C:\tools\miniconda3\envs\PyTorchLive\lib\site-packages\torch\serialization.py", line 242, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

A glance at google results tells me that this might be caused by the difference of PyTorch versions between the pretrained model saver and the loader, but I am not sure about this.

I advice the project owner to:

fix the "failed finding central directory" bug;
add python make_models.py into the instructions, or make it automatically run;
update the python packages used to the latest versions.

Expected Results

Following the instructions in the tutorial, the example project can be successfully run.

Code example, screenshot, or link to repository

No response

torchlive-cli setup-dev script fails to install openjdk@8 ("openjdk@8: no bottle available!")

Version

0.1.2

Problem Area

CLI

Steps to Reproduce

Open terminal in macOS (OS version 12.1, M1 chip)
Run command: npx torchlive-cli setup-dev
Script fails in the OpenJDK installation step: "Error: openjdk@8: no bottle available!"

Expected Results

Expected the JavaSDK installation to succeed.

What is a working way to install the Java SDK?
Would it be possible to use a newer Java SDK instead of openjdk@8, like openjdk@17?

Code example, screenshot, or link to repository

No response

model spec for image2image models

Area Select

react-native-pytorch-core (core package)

Description

What should I do to make a correct spec for image2image model? (unet, for example)
The "pack" spec part seems to be okay, as for "unpack" - I can't find anything that would fit my use case in your model spec API here - https://pytorch.org/live/docs/api/model-spec/
I need to convert a predicted tensor back to an image with reverse normalization.

Speech-To-Text- / Audio-Module-Demo

Area Select

Website/Documentation

Description

Hey I'm trying to use PyTorch live for a Speech-To-Text feature. It would be great if there were a tutorial for this use case.

I read the documentation for the audio module. but I thing the documentation isn't up to date. the module provides only a record function in version "react-native-pytorch-core": "0.1.3", not start and stop-recording as described in the article.

I tried a small playground but the following code leads to an error:

const audio = await AudioUtil.record(2)
console.log('audio', audio)
const {result} = await MobileModel.execute(model_wav2vec, { audio })
console.log('result', result)

there is also a great tutorial for android but none for PyTorch live (react native)

yolov5s.torchscript.ptl

Version

1.1.0

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

run model yolov5s.torchscript.ptl

Expected Results

get results (bbox, class)

Code example, screenshot, or link to repository

Error Possible Unhandled Promise Rejection (id: 0):
Error: End of input at character 0 of promiseMethodWrapper

Did I follow this instruction to convert .pt to .ptl or am I doing something wrong?
https://github.com/pytorch/android-demo-app/tree/master/ObjectDetection
I ran android project pytorch mobile model worked there.

As I understand it, you need live.spec.json but where to get it for yolov5?

It's him ? I used it but it still gives an error or I need to project it?
{'config.txt': '{"shape": [1, 3, 640, 640], "stride": 32, "names": ["person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"]}'}

here is the link to the issue

First Project fails on Windows

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I am trying to install PyTorchLive on Windows 10 machine.

Already have Python installed.
I followed steps in https://pytorch.org/live/docs/tutorials/get-started-manually/ to install reactnative development environment
Am able to successfully run the react-native sample application.
npx react-native init MyFirstProject --template react-native-template-pytorch-live give the following error:
Welcome to React Native!
Learn once, write anywhere

√ Downloading template
√ Copying template
√ Processing template
√ Executing post init script
× Installing dependencies

error Error: Command failed: npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/react
npm ERR!   react@"17.0.1" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"^16.0" from @react-native-community/[email protected]
npm ERR! node_modules/@react-native-community/masked-view
npm ERR!   @react-native-community/masked-view@"^0.1.10" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Expected Results

No response

Code example, screenshot, or link to repository

No response

Image unwrap issue leading to Pytorch Live demo not working on iPad Pro

Version

main

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Deploy demo to iPad.
Run any example with images (for example: run Photo Example, and click on any image like Panda).

React native will say "unhandled exception."

Expected Results

Unwrap should not cause any issues for iPad version.

Code example, screenshot, or link to repository

Tracked down the failure to:
guard let image = castedImage else { throw ImageModuleError.castingObject }
@ https://github.com/pytorch/live/blob/2282cd52946f9133551e152200cd06202be54653/react-native-pytorch-core/ios/Image/ImageModule.swift#L126

EDIT: I am happy to do the heavy lifting debugging on my end since I have an iPad Pro accessible, though I am unfamiliar with React Native + pytorch stack. Any clues will be helpful.

react-native-pytorch-core:externalNativeBuildDebug failed on android build above React-Native 66

Version

0.1.3

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

npx react-native init AwesomeTSProject --template react-native-template-typescript
cd AwesomeTSProject
yarn add react-native-pytorch-core
follow tutorial
cd android && ./gradlew assembleDebug

I attached the patch file for CMakeLists.txt, pls use that for android build above RN66.
BUT then, we can't build on android under RN65.
So, please make condition branch like this for the users

Expected Results

BUILD SUCCESSFUL

but

> Task :react-native-pytorch-core:externalNativeBuildDebug FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':react-native-pytorch-core:externalNativeBuildDebug'.
> Build command failed.

Code example, screenshot, or link to repository

there is same issue here, and already closed.

patch : react-native-pytorch-core+0.1.3.patch.zip

How to detect objects Real Time

Description

Pytorch is amazing,thank you for the work you are doing.
I'm trying Object detection example from tutorial , Can you give an example of real time use "onFrame" of CAMERA?

Website

https://pytorch.org/live/docs/tutorials/get-started/

Error: fft: ATen not compiled with FFT support

Version

0.1

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I have a model trained by myself in python(Unix). but it can't be running in react size(Macos).

output:
`Possible Unhandled Promise Rejection (id: 2):
Error: fft: ATen not compiled with FFT support

Debug info for handle(s): -1, was not found.

Exception raised from _fft_r2c_mkl at ../aten/src/ATen/native/mkl/SpectralOps.cpp:569 (most recent call first):
(no backtrace available)
Error: fft: ATen not compiled with FFT support

Debug info for handle(s): -1, was not found.

Expected Results

No response

Code example, screenshot, or link to repository

No response

Cannot run the example template

Version

0.1.3

Problem Area

Pytorch Live Template

Steps to Reproduce

I downloaded the example app with npx react-native init pytorchlive --template react-native-template-pytorch-live

When trying to run on my android device, I was getting errors saying it could not find the ptl files. After some research, I need to run models\make_models.py? Then I get this error:

C:\Users\jslok\Projects\pytorchlive\models>py make_models.py torch version 1.11.0+cpu Exporting image classification models Exporting model resnet18 Model resnet18 successfully exported Exporting model mobilenet_v3_small Traceback (most recent call last): File "C:\Users\jslok\Projects\pytorchlive\models\make_models.py", line 124, in <module> main() File "C:\Users\jslok\Projects\pytorchlive\models\make_models.py", line 117, in main export_image_classification_models() File "C:\Users\jslok\Projects\pytorchlive\models\make_models.py", line 49, in export_image_classification_models bundle_live_spec_and_export_model(name, script_model) File "C:\Users\jslok\Projects\pytorchlive\models\make_models.py", line 26, in bundle_live_spec_and_export_model optimized_model = optimize_for_mobile(model) File "C:\Python310\lib\site-packages\torch\utils\mobile_optimizer.py", line 62, in optimize_for_mobile optimized_cpp_module = torch._C._jit_pass_optimize_for_mobile( RuntimeError: Expected a value of type 'NoneType' for field 'bias', but found 'Tensor'
I am on a windows machine.

Expected Results

Template app should run.

Code example, screenshot, or link to repository

No response

Custom trained DETR

Version

latest

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Hi, I have a problem to create working live.spec.json file for my custom detr. The detr itself works fine when inferenced from python

steps to reproduce:

retrain detr
change .spec according the tutorial
section to modify ...

"unpack": {
    "type": "bounding_boxes",
    "key": "boundingBoxes",
    "dtype": "float",
    "probabilityThreshold": "$probabilityThreshold",
    "classes": [
      "test class"
    ]
  }

rebuild model for mobile

Problem to solve:
Error: length=1; index=-1

Expected Results

No response

Code example, screenshot, or link to repository

Detr was trained from this repository ... https://github.com/woctezuma/detr.git . My custom detr has 1 class with the class indexed from 0.

Could you please tell me what is the correct spec ?

add fromNativeJSRef method to ImageUtil

Area Select

react-native-pytorch-core (core package)

Description

I've played around with Image2Image models, and I had to add the fromNativeJSRef method to ImageUtil, in order to wrap my inference result, as my model returned a NativeJSRef object.
Trying to import wrapRef directly into my app.ts file resulted in import errors, which the typescript transpiler doesn't catch.

Maybe there is a more elegant solution, like wrapping the output directly in the MobileModelModule

Lite Interpreter version number does not match

Version

0.1.3

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Executed the code at:

https://pytorch.org/live/docs/tutorials/prepare-custom-model/

with pytorch: 1.12.0.dev20220425+cu113

and then I put the model in models folder (from demo image classification) with live.spec.json file:

at last I create and update the classes json file.

When the model is loading I have the following message:

when testing with the current other existing models, everything went fine, the only issue is with my custom model

Expected Results

No response

Code example, screenshot, or link to repository

No response

Link Broken

Tutorial Select

Get Started

Feedback

"Follow the manual installation instructions to get started on Windows and Linux" link is broken.

Error: Unable to resolve module ../models/mobilenet_v3_small.ptl from /Users/ziyuanzhou/Desktop/signdectection/signdetection/src/screens/CameraScreenTest.js:

Version

0.1.2

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I followed the tutorial of preparing custom model, and received this error message while executing. The full message is like this:
Error: Unable to resolve module ../models/mobilenet_v3_small.ptl from /Users/ziyuanzhou/Desktop/signdectection/signdetection/src/screens/CameraScreenTest.js:

None of these files exist:

src/models/mobilenet_v3_small.ptl(.native|.ios.js|.native.js|.js|.ios.json|.native.json|.json|.ios.ts|.native.ts|.ts|.ios.tsx|.native.tsx|.tsx)
src/models/mobilenet_v3_small.ptl/index(.native|.ios.js|.native.js|.js|.ios.json|.native.json|.json|.ios.ts|.native.ts|.ts|.ios.tsx|.native.tsx|.tsx)
10 |
11 | export default class CameraScreenTest extends Component{

12 | model = require('../models/mobilenet_v3_small.ptl');
| ^
13 | constructor(props){
14 | super(props);
15 | this.state = {
at ModuleResolver.resolveDependency (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/node-haste/DependencyGraph/ModuleResolution.js:107:15)
at DependencyGraph.resolveDependency (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/node-haste/DependencyGraph.js:288:43)
at Object.resolve (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/lib/transformHelpers.js:129:24)
at resolve (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/DeltaBundler/traverseDependencies.js:396:33)
at /Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/DeltaBundler/traverseDependencies.js:412:26
at Array.reduce ()
at resolveDependencies (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/DeltaBundler/traverseDependencies.js:411:33)
at processModule (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/DeltaBundler/traverseDependencies.js:140:31)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async addDependency (/Users/ziyuanzhou/Desktop/signdectection/signdetection/node_modules/metro/src/DeltaBundler/traverseDependencies.js:230:18)

I tried to solve the this problem by adding the encoding option in make_model.py.
spec = Path("live.spec.json").read_text(encoding= "UTF-8")
But it didn't help me solve the problem.

Expected Results

No response

Code example, screenshot, or link to repository

No response

For pyrorch live react live, how does it work? Does it use torch script or ONNX?

Tutorial Select

Server Model

Feedback

Thank you for answering.
I am planning to use a custom model, such as a model that is created by me or others. Is it possible?

Besides the question, I guess the tutorial should explain more about that.

Question: Does the 'unpack' accept the object types in spec.json

Tutorial Select

Prepare Custom Model

Feedback

I am trying to write the spec.json for yolo_v5. The output of the model is "models.common.Detections" object, and I didn't see this type of output mentioned in the tutorial. Currently, it seems like I have to let my project able to obtain the object type of prediction result first, then trying to access the specific data (like bounding box) in the .js file.

Is there a way to support custom tokenizer?

Area Select

react-native-pytorch-core (core package)

Description

The current tokenizers only support Bert and GPT-2. But if we want to run other language models, for instance RoberTa, we won't be able to import the RoberTa tokenizer given the current setup.

Error with pod install

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I got this problem with install the library in my project. CocoaPods couldn't find compatible versions for react-native-pytroch-core. I ran the "npm install react-native-pytorch-core" and "pod install", and received the following error message:
[!] CocoaPods could not find compatible versions for pod "react-native-pytorch-core":
In Podfile:
react-native-pytorch-core (from ../node_modules/react-native-pytorch-core)

Specs satisfying the react-native-pytorch-core (from ../node_modules/react-native-pytorch-core) dependency were found, but they required a higher minimum deployment target.

Anyone know where the problem is?

Expected Results

No response

Code example, screenshot, or link to repository

No response

Crash on Android

Version

0.1.3

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I used the following commands to build an apk ,and the building process was successfully finished.
but after I installed the apk on my Android phone with android version 10，it crashed immediately every time you open it .

npx torchlive-cli init myproject
cd myproject/android
./gradlew assembleRelease

Expected Results

Run

Code example, screenshot, or link to repository

No response

Support for Expo and Web

Area Select

react-native-pytorch-core (core package)

Description

First of all, I want to say that this is one of the best projects I've come across during the holidays. This exactly fits my use case of rapid ML application prototyping. can't wait for the stable release! :)

Unfortunately, I'm running a mac m1 so I faced some complications in the automated cli setup. I tried to hack together a solution with expo cli and it worked for Android, although now I can't remember what I did exactly :(
This brings me to my questions/ feature requests.

Will the expo cli be officially supported? It'll probably ease the setup process.
I don't remember how, but I was able to get it working in expo cli by installing it via expo install react-native-pytorch-core and ran the app using yarn android and it worked! I did some troubleshooting by googling the errors but I don't remember it now. Currently, when I try to build the app after importing and using say Camera module from PyTorch core, I get this error:

Earlier I was able to solve it. I know that expo isn't officially supported so this isn't really a bug but It'd be great if it could be supported.

2. Will the pytorch react native core components work with Web?
When I was able to run the camera module on the android emulator successfully, I tried to bundle the app for web. But I was getting errors like "export 'requireNativeComponent' was not found in 'react-native-web/dist/index' and the app didn't load on web. So, I was wondering if this library is only meant for mobile app development or is it intended to be the pytorch port for react-native that supports all platforms.
TLDR: Can I use react-native-pytorch-core components on web and mobile alike?

Template not installing with init command

Version

v0.1.2

Problem Area

CLI

Steps to Reproduce

Following the tutorial, no errors on the setup, but when initializing a new project, the template doesn't get downloaded.

npx torchlive-cli setup-dev
npx torchlive-cli init MyFirstProject

error An unexpected error occurred: "https://registry.yarnpkg.com/react-native-template-react-native-template-pytorch-live: Not found".

Expected Results

I think it should download from "https://registry.yarnpkg.com/react-native-template-pytorch-live", not "https://registry.yarnpkg.com/react-native-template-react-native-template-pytorch-live". I don't know why it tries to install from that URL.

Code example, screenshot, or link to repository

No response

error:Error:Unable to resolve module ../../models/detr_resnet50.ptl from F:\PyTorchDemo\MyFirstProject\src\examples\ObjectDetectionExample.tsx

Tutorial Select

Get Started Manually

Feedback

I installed the relevant software according to tutorial instructions, and after executing the relevant steps, the Node window showed this problem. I am very confused, and I hope to get your answer.

Problem description:

None of these files exist:

detr_resnet50.ptl
models\detr_resnet50.ptl\index(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
25 | const modelInfo: ModelInfo = {
26 | name: 'DETR',

27 | model: require('../../models/detr_resnet50.ptl'),
| ^
28 | };
29 |
30 | const objectColors = [
at ModuleResolver.resolveDependency (F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\node-haste\DependencyGraph\ModuleResolution.js:211:15)
at DependencyGraph.resolveDependency (F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\node-haste\DependencyGraph.js:413:43)
at Object.resolve (F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\lib\transformHelpers.js:317:42)
at resolve (F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:629:33)
at F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:645:26
at Array.reduce ()
at resolveDependencies (F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:644:33)
at F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:329:33
at Generator.next ()
at asyncGeneratorStep (F:\PyTorchDemo\MyFirstProject\node_modules\metro\src\DeltaBundler\traverseDependencies.js:137:24)

Error: Could not convert downloaded file into Torch Module

Version

0.1.2

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I am trying to run my custom model in my project, and the model file couldn't convert into Torch Module. I used and modified the export.py from the Yolo_v5 GitHub, to convert my model file type from .pt to .ptl. So far, I didn't found too much info about this error. I checked the model.live.spec.json file, and found that it is different than the example in the tutorial.

Here is the context of the json file for my model.ptl:
{'model/live.spec.json': '{"shape": [1, 3, 640, 640], "stride": 32, "names": ["Parking Sign"]}'}

Here is the example in the tutorial:
{
"pack": {
"type": "tensor_from_image",
"image": "image",
"transforms": [
{
"type": "image_to_image",
"name": "center_crop"
},
{
"type": "image_to_image",
"name": "scale",
"width": 224,
"height": 224
},
{
"type": "image_to_tensor",
"name": "rgb_norm",
"mean": [0.485, 0.456, 0.406],
"std": [0.229, 0.224, 0.225]
}
]
},
"unpack": {
"type": "argmax",
"dtype": "float",
"key": "maxIdx",
"valueKey": "confidence"
}
}

I guess that I suppose to have "pack" and "unpack" in my json file (like in the example). If anyone know or met this error before, please let me know how to fix it. Thanks!

Expected Results

No response

Code example, screenshot, or link to repository

No response

libreactnativejni.so is missing when adding the live package

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Test it on Android
Do all the steps from here https://pytorch.org/live/docs/tutorials/add-package/
Use the Android Studio, run clean build, rebuild, and run
You will see the following Errors

Expected Results

Build command failed.
Error while executing process D:\ADK\cmake\3.10.2.4988404\bin\ninja.exe with arguments {-C D:\XXXXXXXXXXX\Main\Project\Project\XXXXXXX\src\mobile\node_modules\react-native-pytorch-core\android.cxx\cmake\debug\armeabi-v7a torchlive}
ninja: Entering directory `D:\XXXXXXXXXXX\Project\Project\XXXXXXX\src\mobile\node_modules\react-native-pytorch-core\android.cxx\cmake\debug\armeabi-v7a'

ninja: error: '../../../../build/react-native-0.64.3.aar/jni/armeabi-v7a/libreactnativejni.so', needed by '../../../../build/intermediates/cmake/debug/obj/armeabi-v7a/libtorchlive.so', missing and no known rule to make it

If you run the cmake on clion, you will see this warning. I am not sure it is the reason.

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
FBJNI_LIBRARY
linked by target "torchlive" in directory /cygdrive/d/XXXXXXXXXXX Main/Project/Project/XXXXXXX/src/mobile/node_modules/react-native-pytorch-core/android
PYTORCH_LIBRARY
linked by target "torchlive" in directory /cygdrive/d/XXXXXXXXXXX Main/Project/Project/XXXXXXX/src/mobile/node_modules/react-native-pytorch-core/android
REACT_NATIVE_JNI_LIB
linked by target "torchlive" in directory /cygdrive/d/XXXXXXXXXXX Main/Project/Project/XXXXXXX/src/mobile/node_modules/react-native-pytorch-core/android
REACT_NATIVE_UTILS_LIB
linked by target "torchlive" in directory /cygdrive/d/XXXXXXXXXXX Main/Project/Project/XXXXXXX/src/mobile/node_modules/react-native-pytorch-core/android

Code example, screenshot, or link to repository

No response

Emulator is not installed correctly on the M1 MacBook using the setup-dev command

Version

0.1.2

Problem Area

CLI

Steps to Reproduce

This issue only occurs on the M1 MacBook.

To reproduce this issue:

If you have the Android Emulator already installed in your computer, uninstall it.
Execute the command npx torchlive-cli setup-dev to install the dependencies. You will see that the Android Emulator has been installed correctly:
Execute the command emulator -V and you will see that it won't output the version:

Expected Results

There's an easy fix to this issue. Just going to Android Studio, deleting the Android Emulator (in Appearances & Behavior > System Settings > Android SDK > SDK Tools) and installing it again. Then, it will work correctly.

By doing this, we can execute the command emulator -V again and we will see that it works as expected:

However, the Android Emulator should be installed correctly on the M1 MacBooks when executing the setup-dev command.

Code example, screenshot, or link to repository

No response

app crashes on android

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I am trying to run Image Classification model on Android. I had followed manual instructions for setup on linux machine. Everything is fine till the last step of installation and app gets succesfully installed on my android phone running android v9(Pie). But app crashes as soon as it is launched at start.

Expected Results

No response

Code example, screenshot, or link to repository

No response

Problems in make_models.py

Version

"react-native-pytorch-core": "0.1.2",

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

npx torchlive-cli init MyFirstProject

Expected Results

Create a project without an error

Code example, screenshot, or link to repository

I get this error when I create new project.

› Error project "SmartCalc" is created but we ran into the following issues at generatin…
‎ ‎ ‎ ‎
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /Users/nec…
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
h.request(req.get_method(), req.selector, req.data, headers,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py",…
self._send_request(method, url, body, headers, encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py",…
self.endheaders(body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py",…
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py",…
self.send(msg)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py",…
self.connect()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py",…
self.sock = self._context.wrap_socket(self.sock,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 50…
return self.sslsocket_class._create(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 10…
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 13…
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify fail…
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/make_models.py", line 1…
main()
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/make_models.py", line 1…
export_image_classification_models()
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/make_models.py", line 3…
"resnet18": torchvision.models.resnet18(pretrained=True),
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/venv/lib/python3.9/site…
return _resnet('resnet18', BasicBlock, [2, 2, 2, 2], pretrained, progress,
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/venv/lib/python3.9/site…
state_dict = load_state_dict_from_url(model_urls[arch],
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/venv/lib/python3.9/site…
download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "/Users/necmettinsargin/Documents/GitHub/SmartCalc/models/venv/lib/python3.9/site…
u = urlopen(req)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
return self.do_open(http.client.HTTPSConnection, req,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.p…
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate ver…
‎ ‎ ‎ ‎
please rerun
‎ ‎ ‎ ‎ • "python make_models.py" in the SmartCalc/models/ directory, and
‎ ‎ ‎ ‎ • "pod install" in the SmartCalc/ios/ directory
if the issue persists, please report at https://github.com/pytorch/live/issues.

And When I run python make_models.py it says

File "make_models.py", line 17
print(f"torch version {torch.version}")
^
SyntaxError: invalid syntax

manual set-up error: Command failed: /tmp/rncli-init-template-lRLLcg/node_modules/react-native-template-pytorch-live/script.js

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

OS: Ubuntu 20.04.3 LTS
python3 -V: Python 3.8.5

List of all installed node.js modules with npm -g ls --depth=0:

/usr/local/lib
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

command: npx react-native init MyFirstProject --template react-native-template-pytorch-live

Expected Results

               ######                ######               
             ###     ####        ####     ###             
            ##          ###    ###          ##            
            ##             ####             ##            
            ##             ####             ##            
            ##           ##    ##           ##            
            ##         ###      ###         ##            
             ##  ########################  ##             
          ######    ###            ###    ######          
      ###     ##    ##              ##    ##     ###      
   ###         ## ###      ####      ### ##         ###   
  ##           ####      ########      ####           ##  
 ##             ###     ##########     ###             ## 
  ##           ####      ########      ####           ##  
   ###         ## ###      ####      ### ##         ###   
      ###     ##    ##              ##    ##     ###      
          ######    ###            ###    ######          
             ##  ########################  ##             
            ##         ###      ###         ##            
            ##           ##    ##           ##            
            ##             ####             ##            
            ##             ####             ##            
            ##          ###    ###          ##            
             ###     ####        ####     ###             
               ######                ######               
                                                          

                  Welcome to React Native!                
                 Learn once, write anywhere               

✔ Downloading template
✔ Copying template
✔ Processing template
⠙ Executing post init script node:child_process:903
    throw err;
    ^

Error: Command failed: cd ./models && python3 -m venv ./venv 2> ./error.log             && source ./venv/bin/activate 2> ./error.log             && pip install --upgrade pip 2> ./error.log             && pip install -r requirements.txt 2> ./error.log             && python -W ignore make_models.py 2> ./error.log
    at checkExecSyncError (node:child_process:826:11)
    at execSync (node:child_process:900:15)
    at Object.<anonymous> (/tmp/rncli-init-template-lRLLcg/node_modules/react-native-template-pytorch-live/script.js:16:5)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47 {
  status: 127,
  signal: null,
  output: [ null, Buffer(0) [Uint8Array] [], Buffer(0) [Uint8Array] [] ],
  pid: 12763,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(0) [Uint8Array] []
}
✖ Executing post init script 
error Error: Command failed: /tmp/rncli-init-template-lRLLcg/node_modules/react-native-template-pytorch-live/script.js

if needed, I can supply the outcome when run with --verbose

Code example, screenshot, or link to repository

No response

iOS does not support all types in model spec

Version

No response

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

Write a model spec JSON with an "unpack" type of "tuple" or "list", as described in the documentation here:
https://pytorch.org/live/docs/api/model-spec/#unpack-output-post-processing
Attempt to execute this model on iOS.

Expected Results

The model should execute and return the tuple or list as a return value.

Instead, the model returns an InvalidUnpackType error, which seems to be returned by:

https://github.com/pytorch/live/blob/main/react-native-pytorch-core/ios/ML/Processing/BaseIValuePacker.swift#L79

At minimum the documentation should probably point out that these types are not currently supported on iOS.

Code example, screenshot, or link to repository

I've got an error while creating new TorchLive Project

Tutorial Select

Get Started Manually

Feedback

C:\WINDOWS\system32>npx torchlive-cli init MyFirstProject

| |_ ___ _ __ | | | ()_ _____
| / _ | '/ | ' | | \ \ / / _
| || (_) | | | (| | | | | |\ V / /
___/|_| _|| |||| _/ __|

torchlive version 0.1.1
C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\android\AndroidSDK.js:65
throw new Error("platform " + SystemUtils_1.platform + " unsupported");
^

Error: platform win32 unsupported
at Object.getSDKPath (C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\android\AndroidSDK.js:65:15)
at Object.getEnv (C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\utils\SystemUtils.js:63:39)
at InitTask.isValid (C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\cli-commands\Init.js:69:72)
at C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\utils\TaskUtils.js:63:77
at Array.filter ()
at Object. (C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\utils\TaskUtils.js:63:40)
at step (C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\utils\TaskUtils.js:41:23)
at Object.next (C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\utils\TaskUtils.js:22:53)
at C:\Users\krtmad\AppData\Local\npm-cache_npx\e9a46805b0361e9c\node_modules\torchlive-cli\build\utils\TaskUtils.js:16:71
at new Promise ()

C:\WINDOWS\system32>

ImageUtil.toFile always return null

Version

0.0.0-04292022-1206-56d9e1d56

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

I am trying to save the image to the file and save in the photo library, but the result from ImageUtil.toFile is null. Here is the output from the ImageUtil.toFile:{"_U": 0, "_V": 0, "_W": null, "_X": null}.

Expected Results

No response

Code example, screenshot, or link to repository

};

UnhandledPromiseRejection @ npx torchlive-cli run-android in "get started"

Tutorial Select

Get Started

Feedback

when I run these,I got an error

(venv) (base) abemasatoshinoMacBook-Air:~ patriot$ npx torchlive-cli init PL_project
  _                 _     _ _           
 | |_ ___  _ __ ___| |__ | (_)_   _____ 
 | __/ _ \| '__/ __| '_ \| | \ \ / / _ \
 | || (_) | | | (__| | | | | |\ V /  __/
  \__\___/|_|  \___|_| |_|_|_| \_/ \___|
                                        
torchlive version 0.1.3
✔ project PL_project
  › Initialized project PL_project
    ‎ ‎ ‎ ‎
    Run instructions for Android:
    ‎ ‎ ‎ ‎ • Have an Android emulator running (quickest way to get started), or a device connected.
    ‎ ‎ ‎ ‎ • cd PL_project && npx torchlive-cli run-android
    ‎ ‎ ‎ ‎
    Run instructions for iOS:
    ‎ ‎ ‎ ‎ • cd PL_project && npx torchlive-cli run-ios
    ‎ ‎ ‎ ‎ - or -
    ‎ ‎ ‎ ‎ • Open PL_project/ios/PL_project.xcworkspace in Xcode or run "xed -b ios"
    ‎ ‎ ‎ ‎ • Hit the Run but

and

(venv) (base) abemasatoshinoMacBook-Air:PL_project patriot$ npx torchlive-cli run-android
INFO    | configAndStartRenderer: setting vsync to 60 hz
INFO    | added library /Users/patriot/Library/Android/sdk/emulator/lib64/vulkan/libvulkan.dylib
INFO    | Started GRPC server at 127.0.0.1:8554, security: Local
INFO    | Advertising in: /Users/patriot/Library/Caches/TemporaryItems/avd/running/pid_27722.ini
yarn‎ run‎ v1.22.17
$‎ react-native‎ run-android
info‎ Running‎ jetifier‎ to‎ migrate‎ libraries‎ to‎ AndroidX.‎ You‎ can‎ disable‎ it‎ using‎ "--no-jetifier"‎ flag.
Jetifier‎ found‎ 1182‎ file(s)‎ to‎ forward-jetify.‎ Using‎ 4‎ workers...
info‎ Starting‎ JS‎ server...
info‎ Installing‎ the‎ app...

FAILURE:‎ Build‎ failed‎ with‎ an‎ exception.

*‎ What‎ went‎ wrong:
A‎ problem‎ occurred‎ configuring‎ project‎ ':react-native-pytorch-core'.
>‎ NDK‎ at‎ /Users/patriot/Library/Android/sdk/ndk/20.1.5948944‎ did‎ not‎ have‎ a‎ source.properties‎ file

*‎ Try:
Run‎ with‎ --stacktrace‎ option‎ to‎ get‎ the‎ stack‎ trace.‎ Run‎ with‎ --info‎ or‎ --debug‎ option‎ to‎ get‎ more‎ log‎ output.‎ Run‎ with‎ --scan‎ to‎ get‎ full‎ insights.

*‎ Get‎ more‎ help‎ at‎ https://help.gradle.org

BUILD‎ FAILED‎ in‎ 24s
Error:‎ Command‎ failed:‎ ./gradlew‎ app:installDebug‎ -PreactNativeDevServerPort=8081

FAILURE:‎ Build‎ failed‎ with‎ an‎ exception.

*‎ What‎ went‎ wrong:
A‎ problem‎ occurred‎ configuring‎ project‎ ':react-native-pytorch-core'.
>‎ NDK‎ at‎ /Users/patriot/Library/Android/sdk/ndk/20.1.5948944‎ did‎ not‎ have‎ a‎ source.properties‎ file

*‎ Try:
Run‎ with‎ --stacktrace‎ option‎ to‎ get‎ the‎ stack‎ trace.‎ Run‎ with‎ --info‎ or‎ --debug‎ option‎ to‎ get‎ more‎ log‎ output.‎ Run‎ with‎ --scan‎ to‎ get‎ full‎ insights.

*‎ Get‎ more‎ help‎ at‎ https://help.gradle.org

BUILD‎ FAILED‎ in‎ 24s

‎ ‎ ‎ ‎ at‎ makeError‎ (/Users/patriot/PL_project/node_modules/execa/index.js:174:9)
‎ ‎ ‎ ‎ at‎ /Users/patriot/PL_project/node_modules/execa/index.js:278:16
‎ ‎ ‎ ‎ at‎ processTicksAndRejections‎ (node:internal/process/task_queues:96:5)
‎ ‎ ‎ ‎ at‎ async‎ runOnAllDevices‎ (/Users/patriot/PL_project/node_modules/@react-native-community/cli-platform-android/build/commands/runAndroid/runOnAllDevices.js:94:5)
‎ ‎ ‎ ‎ at‎ async‎ Command.handleAction‎ (/Users/patriot/PL_project/node_modules/@react-native-community/cli/build/index.js:186:9)
info‎ Run‎ CLI‎ with‎ --verbose‎ flag‎ for‎ more‎ details.
info‎ Visit‎ https://yarnpkg.com/en/docs/cli/run‎ for‎ documentation‎ about‎ this‎ command.
node:internal/process/promises:246
          triggerUncaughtException(err, true /* fromPromise */);
          ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "error Failed to install the app. Make sure you have the Android development environment set up: https://reactnative.dev/docs/environment-setup.
error Command failed with exit code 1.
".] {
  code: 'ERR_UNHANDLED_REJECTION'

Unhandled Promise Rejection from the tutorial

Tutorial Select

Question Answering

Feedback

Source Text:
I ordered a big Mac, French fries, and soda at McDonald's with order number 261 and sit at table 20. I made that order at 3 p.m.

Question:
What did I order?

Can't run the app on my phone or the

Tutorial Select

Get Started

Feedback

When I run
npx torchlive-cli init MyFirstProject
I get the following exception. This happens when I both try to run while having a phone connected or the emulator. I am using the latest version of android studio.

FAILURE:‎ Build‎ failed‎ with‎ an‎ exception.

*‎ What‎ went‎ wrong:
Execution‎ failed‎ for‎ task‎ ':app:installDebug'.

‎ java.util.concurrent.ExecutionException:‎ com.android.builder.testing.api.DeviceException:‎ com.android.ddmlib.InstallException:‎ INSTALL_FAILED_INVALID_APK:‎ Scanning‎ Failed.:‎ Package‎ /data/app/org.pytorch.live.example-oieYaFbXt6dMki1MKv4QNA==/base.apk‎ code‎ is‎ missing

*‎ Try:
Run‎ with‎ --stacktrace‎ option‎ to‎ get‎ the‎ stack‎ trace.‎ Run‎ with‎ --info‎ or‎ --debug‎ option‎ to‎ get‎ more‎ log‎ output.‎ Run‎ with‎ --scan‎ to‎ get‎ full‎ insights.

*‎ Get‎ more‎ help‎ at‎ https://help.gradle.org

BUILD‎ FAILED‎ in‎ 17s

‎ ‎ ‎ ‎ at‎ makeError‎ (/Users/anastasiosnoulas/MyFirstProject/node_modules/execa/index.js:174:9)
‎ ‎ ‎ ‎ at‎ /Users/anastasiosnoulas/MyFirstProject/node_modules/execa/index.js:278:16
‎ ‎ ‎ ‎ at‎ processTicksAndRejections‎ (internal/process/task_queues.js:93:5)
‎ ‎ ‎ ‎ at‎ async‎ runOnAllDevices‎ (/Users/anastasiosnoulas/MyFirstProject/node_modules/@react-native-community/cli-platform-android/build/commands/runAndroid/runOnAllDevices.js:94:5)
‎ ‎ ‎ ‎ at‎ async‎ Command.handleAction‎ (/Users/anastasiosnoulas/MyFirstProject/node_modules/react-native/node_modules/@react-native-community/cli/build/index.js:186:9)

wrong template name

Tutorial Select

Get Started Manually

Feedback

After running npx react-native init MyFirstProject --template react-native-template-pytorch-live on windows 10, everything goes well until this: info Fetching template react-native-template-react-native-template-pytorch-live...
which results in error An unexpected error occurred: "https://registry.yarnpkg.com/react-native-template-react-native-template-pytorch-live: Not found".
Looks like the package name is resolved incorrectly (installing react-native-template-pytorch-live works okay)

Is initial maxIndex = -1 in BoundingBoxesUnpacker.swift right? then, I'd like to know the reason.

Version

0.1.3, and also latest commits

Problem Area

react-native-pytorch-core (core package)

Steps to Reproduce

create object detection model .ptl file
put the model file at src/app/models/ and input the ModelInfo in models/index.tsx
build and run the project
Fatal error: Index out of range happens

I'm enjoying to do real-time object detection on ios device because of great PyTorch live that you've been making.
I'm a beginner at Swift, so I don't know what maxIndex value= -1 indicates.
After changed maxIndex value -1 ⇨ 0, this error never happen.
So, I thought the value = 0 is right one.

Could you tell me whether the maxIndex value = -1 is correct and then the reason Maxindex = -1 is set, thanks in advance.

Expected Results

"Fatal error: Index out of range" doesn't happen

Code example, screenshot, or link to repository

Model Spec for MobileNet Classes

Tutorial Select

Prepare Custom Model

Feedback

If we follow the tutorial verbatim and take the MobilenetV3 images directly from pretrained, how do we get the MobileModel function to return the top 5 classes and confidence scores?

https://pytorch.org/live/docs/api/model-spec/

PyTorch Live for Flutter

Area Select

Other

Description

Is there any plans for pytorch-live to have an implementation in Flutter (Dart) ?

facebookresearch / playtorch Goto Github PK

playtorch's Introduction

This project has been archived and is no longer actively maintained.

Build your AI powered mobile prototypes in minutes: Get Started · Tutorials · API

Contents

📋 Requirements

🎉 Building your first PlayTorch app

📖 Documentation

👏 How to Contribute

License

playtorch's People

Contributors

Stargazers

Watchers

Forkers

playtorch's Issues

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Area Select

Description

Area Select

Description

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Description

Images

Website

Credit (optional)

Source Code (optional)

Version

Problem Area

Steps to Reproduce

Systems

Attempt on Windows 10

Expect

Description

Images

Website

Credit (optional)

Source Code (optional)

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository

Area Select

Description

Area Select

Description

Version

Problem Area

Steps to Reproduce

Expected Results

Code example, screenshot, or link to repository