Comments (9)
First of all, thanks for the reply, I'll try to make it work as soon as I can, but right now I'm working on an emergency in another project, when I manage to work this out, I'll reply.
from playtorch.
@ademirmochi, what style is applied to the camera? It may be the camera component mounts but renders with
0
width and height. Please try giving the camera a fixed width/height or apply the following style.import {StyleSheet} from 'react-native'; // ... <Camera style={StyleSheet.absoluteFill} />
EDIT: The camera should render based on the example in your first comment, so there might be something else happening. Are there any errors in the terminal/log? If not, can you try Flipper and see if there is an error logged but swallowed by the console?
So I did it, it didn't help, but I did manage to go over this particular problem, now, I've created a custom ptl model from scratch, but the thing is, it was based in Yolo, not MobileNet, like in the tutorial, how can I read it properly, I mean, I did like in the tutorial, but, since it expects a MobileNet model, of course that generated an error, so basically my question is, and keep in mind I very recent with AI development with image recognition, so I know very little:
Does PyTorchLive for react native, or react-native-pytorch-core, offer support for ptl files generated from Yolo data, and if so, how can I do it?
To further explain my problem, pretty much, I have to config the image specs in the input and the output, and that is the problem, the image interface from react-native-pytorch-core, does not help me with that...
In this case, I'm expecting a image with this specs from the camera: [1,640,640,3], but when I tried to mess with the image resolution, by using the targetResolution, according with the tutorial for the Camera component, but it doesn't put in the resolution I want, it even raised it to 1024/768, why is this happening?
same for the output, that I should expect a float32[1,25200,9] matrix, so how can I do that with react-native-pytorch-core?
BTW this is the error after trying with the tutorial from Metro:
Possible Unhandled Promise Rejection (id: 1):
Error: End of input at character 0 of
promiseMethodWrapper@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:2552:45
_callee3$@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:110988:76
tryCatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24227:23
invoke@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24397:32
tryCatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24227:23
invoke@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24299:30
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24329:19
tryCallTwo@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:28913:9
doResolve@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:29077:25
Promise@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:28936:14
callInvokeWithMethodAndArg@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24328:33
enqueue@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24333:157
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24348:69
_callee3@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:110982:42
execute@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:110999:8
handleImage$@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:109911:128
tryCatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24227:23
invoke@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24397:32
tryCatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24227:23
invoke@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24299:30
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24329:19
tryCallTwo@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:28913:9
doResolve@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:29077:25
Promise@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:28936:14
callInvokeWithMethodAndArg@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24328:33
enqueue@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24333:157
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:24348:69
handleImage@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:109906:40
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:110155:39
invokeGuardedCallbackProd@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:3879:21
invokeGuardedCallback@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:3983:42
invokeGuardedCallbackAndCatchFirstError@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:3987:36
executeDispatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:4065:48
executeDispatchesInOrder@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:4085:26
executeDispatchesAndRelease@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5345:35
forEachAccumulated@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:4618:18
runEventsInBatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5369:27
runExtractedPluginEventsInBatch@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5449:25
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5424:42
batchedUpdates$1@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:17486:20
batchedUpdates@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5331:36
_receiveRootNodeIDEvent@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5423:23
receiveEvent@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:5453:32
__callFunction@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:3056:36
http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:2780:31
__guard@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:3007:15
callFunctionReturnFlushedQueue@http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.test_pytorch&modulesOnly=false&runModule=true:2779:21
callFunctionReturnFlushedQueue@[native code]
from playtorch.
So, first a update, i've started using a mac to eliminate any problems with the SO, next, about the last question, i simply messed with the targetResolution parameter in the Camera component, to 640/640, but on metro server console.log it appeared as 1024/768, just that, nothing more.
But from my understanding, Yolo is not supported, but i've tried to change the input of the image, to change the resolution, height, width, and the output object from the ptl model, but I have no access to the image frame taken from the camera, nor the parameters of said model undergoing your component, at least, that's what it seems, only the parameters the Camera component gives me to change it accordingly, such as targetResolution.
btw, i should mention that i'm testing with my own android cellphone, don't know if that is a issue.
In the end, what I need to know is, the react-native-pytorch-core dependency and its modules and components, does any of them offer me a solution to customize the input frame image taken from the camera, and the output model object given by the ptl model?
from playtorch.
Hi @ademirmochi thanks for using PTL and provide the feedback. We are happy to follow up on this -- if I am understanding correctly, you are developing (and building the app) with a Windows machine. May I ask where is the app ran? Is the camera NOT running on a physical device, simulator, or both? And are you building an Android App or iOS app? And are you trying to integrate the react-native-pytorch-core package to an existing app, which might require adjustment of the permission?
This would help us reproduce the issue and see how we can help! Thanks!
from playtorch.
Hi @ademirmochi thanks for using PTL and provide the feedback. We are happy to follow up on this -- if I am understanding correctly, you are developing (and building the app) with a Windows machine. May I ask where is the app ran? Is the camera NOT running on a physical device, simulator, or both? And are you building an Android App or iOS app? And are you trying to integrate the react-native-pytorch-core package to an existing app, which might require adjustment of the permission?
This would help us reproduce the issue and see how we can help! Thanks!
thanks for the reply, BOTH, I've used in a Windows machine, then, after seeing that there was a lot of problems with windows and/or Linux, I've tried with a m1 Mac machine, I'm trying to use pytorch-live-core sdk and integrating in an already existing app, and I've tried with both IOS and Android phones, and with the emulator as well, none worked, but since there is a camera involved, I'm trying more with my android device, so I can see the camera working, but it only shows a blank screen, it appears the text "Image classification" on top, like the tutorial, and even asks for permission, but the area where is supposed to be a camera rolling, there is just a blank screen.
BTW I'm trying to make it work regardless of the phone SO, so I'm trying for both Android and iOS system
from playtorch.
@ademirmochi, what style is applied to the camera? It may be the camera component mounts but renders with 0
width and height. Please try giving the camera a fixed width/height or apply the following style.
import {StyleSheet} from 'react-native';
// ...
<Camera style={StyleSheet.absoluteFill} />
EDIT: The camera should render based on the example in your first comment, so there might be something else happening. Are there any errors in the terminal/log? If not, can you try Flipper and see if there is an error logged but swallowed by the console?
from playtorch.
Hi @ademirmochi
To your questions
Does PyTorchLive for react native, or react-native-pytorch-core, offer support for ptl files generated from Yolo data, and if so, how can I do it?
So far YOLO is not supported by react-native-pytorch-core package. We are currently working on a Data Processing API to make the data preprocessing/postprocessing more flexible for users.
In this case, I'm expecting a image with this specs from the camera: [1,640,640,3], but when I tried to mess with the image resolution, by using the targetResolution, according with the tutorial for the Camera component, but it doesn't put in the resolution I want, it even raised it to 1024/768, why is this happening?
May I ask how you are configuring the camera? Though the main support would still focus on Mac OS, we are trying to gain more experience on PTL with Windows (and your experience are valuable! It would be great if you could provide codes/ steps to reproduce the issue.
Thanks!
from playtorch.
Hi @ademirmochi congrat on your new mac (but pls let us know if you still need support for windows!)
about the last question, i simply messed with the targetResolution parameter in the Camera component, to 640/640, but on metro server console.log it appeared as 1024/768, just that, nothing more.
for the set resolution, that's expected. As mentioned in (https://pytorch.org/live/docs/api/core/interfaces/cameraview.cameraprops/#targetresolution)
Camera target resolution. It is not guaranteed that the camera runs at the set target resolution, and it might pick the closest available resolution.
Android implementation are based on CameraX, and iOS's implementation are here (https://github.com/pytorch/live/blob/main/react-native-pytorch-core/ios/Camera/CameraView.swift#L77-L91). (so the output image resolution might be platform-specific as well)
thanks @raedle for providing the pointer!
In the end, what I need to know is, the react-native-pytorch-core dependency and its modules and components, does any of them offer me a solution to customize the input frame image taken from the camera, and the output model object given by the ptl model?
at this moment, the data pre/post process support is limited to the pack/unpack json (we are woking on a more flexible one)
(as an example: https://github.com/pytorch/live/blob/423caa394f43e0a615c287071494c94697eee853/react-native-pytorch-core/example/models/resnet18.ptl.live.spec.json#L11-L14).
(and FYI the Android Native implementaion: https://github.com/pytorch/live/blob/2282cd52946f9133551e152200cd06202be54653/react-native-pytorch-core/android/src/main/java/org/pytorch/rn/core/ml/processing/BaseIValuePacker.java#L661)
let me know does that answer your questions? and what else I can help!
from playtorch.
@ademirmochi I will close the issue for now. But feel free to re-open it / create another issue once you are back with us!
from playtorch.
Related Issues (20)
- PlayTorchJSIModule not found - ios & android m2 macbook pro HOT 2
- It is not possible to convert audio float32 tensor to playable uint8 audio expected by .audioFromBytes() HOT 1
- How to specify interpolation in transforms.resize?
- AudioUtil is not working on Android
- How to convert my yolov5s.pt to yolov5s.ptl HOT 1
- resize function gives error: 'Factory function resize expects 1 argument but 4 are given.'
- Cannot scan QR code from the PlayTorch app HOT 2
- playtorchjsimodule not found
- Mac M1 tutorial failure HOT 2
- unable to load local .ptl model HOT 2
- How to export a Huggingface transfromer model? HOT 3
- react_native_pytorch_core error during gradle build HOT 15
- image data fetch
- Is there a processing step im missing when using a custom trained yolo model?
- Is this project still maintained?
- Unable to work
- CMake Error
- Android build fails | Expo app using prebuild HOT 4
- 'react_native_pytorch_core-Swift.h' file not found during ios build HOT 1
- Add how to use EAS (Expo Application Service) build within this tutorial
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from playtorch.