Comments (12)
You could run the remainder of the script manually. It might still take a long time, though.
$ cd ${HOME}/src/tensorflow-2.3.0
$ bazel build --config=opt \
--config=cuda \
--config=noaws \
--local_cpu_resources=HOST_CPUS*0.25 \
--local_ram_resources=HOST_RAM*0.5 \
//tensorflow/tools/pip_package:build_pip_package
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package wheel/tensorflow_pkg
from jetson_nano.
Thanks @jkjung-avt, I try !
from jetson_nano.
Hi @jkjung-avt ,
After many hours the script failed again and always in the same point ,it's odd because the manual import of google.protobuf (exactly 3.8.0) works without problem.
Maybe some artifact from the previous script run was the cause so I restarted from scratch the procedure.
Maybe would be possible for you to upload somewhere the resulting wheel file TF 2.3.0 for Jetson ? Would you be so kind ? Ineed to verify if I can use a software based onTF gpu>=2.2
Thanks
germal
from jetson_nano.
Please check out the wheel files provided by NVIDIA: https://developer.download.nvidia.com/compute/redist/jp/v44/tensorflow/
And the official installation instructions are here: Installing TensorFlow For Jetson Platform
By the way, based on my experience, the "ModuleNotFoundError: No module named 'google.protobuf'" issue could usually be solved by removing "__init__,py" in the "google" folder. Refer to: jkjung-avt/tensorrt_demos#270
from jetson_nano.
Thank you ! I managed to install 2.3.0 with the wheel provided by Nvidia and the build script is ongoing on another machine.
Now I am facing an OOM error ,that I supposed depending from the model that I am loading .But also decreasing batch size to 1 the problem is still there.
Have you experienced this type of error in Tensorflow 2.x ?
Cheers
germal
from jetson_nano.
The "ModuleNotFoundError: No module named 'google.protobuf'" is not specific to tensorflow. I think it's due to conflicts in the "__init__.py" between different modules from google. It usually happens when you install "protobuf" along with some other modules by google.
If you search for the error message, you would find a lot of fellow users reporting the same issue. For example, https://stackoverflow.com/questions/31308812/no-module-named-google-protobuf/37249630
from jetson_nano.
Sorry for being imprecise , I was meaning Out of memory error.
I am loading tiny yolo weights and I find impossible that the nano couldn't load them, this is the message
020-11-08 16:11:40.418398: I tensorflow/core/common_runtime/bfc_allocator.cc:1040] total_region_allocated_bytes_: 37363712 memory_limit_: 37363712 available bytes: 0 curr_region_allocation_bytes_: 74727424
2020-11-08 16:11:40.418439: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Stats:
Limit: 37363712
InUse: 6309376
MaxInUse: 15738368
NumAllocs: 123
MaxAllocSize: 4718592
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2020-11-08 16:11:40.418489: W tensorflow/core/common_runtime/bfc_allocator.cc:439] _**_____*****************_________________________
2020-11-08 16:11:40.418567: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at random_op.cc:77 : Resource exhausted: OOM when allocating tensor with shape[3,3,512,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
I also tried to limit the Tensorflow memory with the following block without luck
from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession
config = ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.7
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)
Have you some suggestion on this topic ?
Thanks a lot
from jetson_nano.
Nope. I ran yolov4/yolov3 models with darknet on Jetson Nano. I also converted those yolov3/yolov4 models to TensorRT. They all worked without OOM problem.
Reference:
from jetson_nano.
Thank you @jkjung-avt ,it seems the problem come from the way TF use to access the memory,But for sure something is wrong in my configuration.
I too used yolov3 inference in another application using onlys darknet and performed like a charm !
from jetson_nano.
Hello @jkjung-avt how are you ?
Now I have resolved the memory issue limiting the fraction of memory that TF lock at its bootstrap.
But the yolov2 inference code that I am using ,that works well on the nvidia gpu on my laptop, doesn't detect anything on the nano , the message is bounding box found 0.
I installed the correct cuda 10.0 and cudnn 7 libraries and used all the available TF versions on the nvidia repository, and full or tiny weights but the result is the same , zero detections. I am using a realsense camera for realtime detection of persons.
From your experience on nano and yolov2 is there any parameter I can tune,considering the limited capacity of the nano wrt to realtime detection from camera ? I tried limiting the camera fps but without luck.
Do you know how could I trap a message related to the possible bottleneck in TF to process s stream too fast of data ?
Thanks a lo for you reply
Cheers
germal
from jetson_nano.
Please consider optimizing and running your model with TensorRT, instead of inferencing with TensorFlow directly. In general, the TensorRT engine runs faster and requires less memory to run. You could refer to my tensorrt_demos repository for examples.
from jetson_nano.
Hello @jkjung-avt thank you for your resources that are definitively a thesaurus.I will try tensort first with your guided examples because in the case of the yolov2 application I am using it is based on this yolov2 https://github.com/experiencor/keras-yolo2 that seems to me has not a straightforward conversion procedure because it is based on two set of weights the frontend and backend. .the problem is there, indeed it runs too slowly on Jetson and detection doesn't happen
from jetson_nano.
Related Issues (20)
- patch of tensorflow/opencv HOT 5
- Cannot build OpenCV-3.4.6 in Jetson Nano HOT 4
- time on install tensorflow 1.15 with jetpack v4.4 HOT 2
- not installed numpy on jetpack 4.4 on jetson nano2 HOT 2
- Could not find any cuda.h matching version '10.0' in any subdirectory HOT 4
- Did you try to install TF2.4 from source with GPU accel on Nano or NX? HOT 1
- tensorflow 2.0.0 build error HOT 4
- Error using the script HOT 9
- Error installing protobuf-3.8.0.sh HOT 1
- Issues during installing TF 1.15 HOT 2
- hello, i installed protobuf using your script, but i come to an issue, please help HOT 1
- testing out demo 5 YOLOv4
- Latest Tensor Flow version
- jetpack-4.3 upgrade pip3 command not found HOT 2
- build whl tensorflow 1.12 and not tensorflow-gpu-1.12 HOT 1
- build tensorflow 1.15 HOT 1
- install opencv_3.4.8 on jetson nano HOT 3
- build tensorflow on jetson nano HOT 3
- Missing protobuf HOT 16
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jetson_nano.