ojh6404 / tracking_ros Goto Github PK

ROS compatible package for object tracking based on SAM, Cutie, GroundingDINO, YOLO-World, VLPart and DEVA

CMake 1.95% Python 93.76% Dockerfile 3.52% Shell 0.77%

computer-vision object-detection pytorch ros-noetic segment-anything segmentation video-object-segmentation video-object-tracking video-segmentation object-tracking

tracking_ros's Introduction

tracking_ros

ROS1 package for detecting and tracking objects using SAM, Cutie, GroundingDINO and DEVA, inspired by detic_ros.

Usage

Tested : image of 480X640 30hz, 3090ti

Interactive prompt for generating mask and tracking object using SAM and Cutie.

sam_node publishes segmentation prompt which is used by cutie_node to track objects. It runs almost real-time (~30hz).

Detecting and tracking object using SAM, GroundingDINO and DEVA.

deva_ndoe queries objects GroundingDINO and SAM at some intervals, so it can track new object after tracking is started. It runs ~15hz and you can adjust cfg['detection_every'] for performance. See node_scripts/model_config.py

Setup

Prerequisite

This package is build upon

ROS1 (Noetic)
catkin virtualenv (python>=3.9 used for DEVA)
(Optional) docker and nvidia-container-toolkit (for environment safety)

Build package

on your workspace

If you want build this package directly on your workspace, please be aware of python environment dependencies (python3.9 and pytorch is needed to build package).

mkdir -p ~/ros/catkin_ws/src && cd ~/ros/catkin_ws/src
git clone https://github.com/ojh6404/tracking_ros.git
wstool init
wstool merge -t . tracking_ros/tracking_ros/rosinstall.noetic
wstool update -t . # jsk-ros-pkg/jsk_visualization for GUI
cd tracking_ros/tracking_ros && ./prepare.sh
cd ~/ros/catkin_ws && catkin b

using docker (Recommended)

Otherwise, you can build only tracking_ros_utils package for using intractive prompt gui

mkdir -p ~/ros/catkin_ws/src && cd ~/ros/catkin_ws/src
git clone https://github.com/ojh6404/tracking_ros.git
wstool init
wstool merge -t . tracking_ros/tracking_ros/rosinstall.noetic
wstool update -t . # jsk-ros-pkg/jsk_visualization for GUI
cd ~/ros/catkin_ws && catkin b tracking_ros_utils

and build whole package on docker environment.

source ~/ros/catkin_ws/devel/setup.bash
roscd tracking_ros_utils/../tracking_ros
docker build --build-arg CUDA_VERSION=11.3 -t tracking_ros . # default is 11.3, you can also build with 12.1

How to use

Please refer sample_track.launch and deva.launch

Tracking using SAM and Cutie with interactive gui prompt.

1. run directly

roslaunch tracking_ros sample_track.launch \
    input_image:=/kinect_head/rgb/image_rect_color \
    mode:=prompt \
    model_type:=vit_t \
    device:=cuda:0

2. using docker

You need to launch tracker and gui seperately cause docker doesn't have gui, so launch tracker by

./run_docker -host pr1040 -launch track.launch \
    input_image:=/kinect_head/rgb/image_rect_color \
    mode:=prompt \
    model_type:=vit_t \
    device:=cuda:0

where

-host : hostname like pr1040 or localhost
-launch : launch file name to run

and launch rqt gui on your gui machine by

roslaunch tracking_ros_utils sam_gui.launch

Detecting and tracking object.

roslaunch tracking_ros deva.launch \
    input_image:=/kinect_head/rgb/image_rect_color \
    model_type:=vit_t \
    device:=cuda:0

./run_docker -host pr1040 -launch deva.launch \
    input_image:=/kinect_head/rgb/image_rect_color \
    model_type:=vit_t \
    device:=cuda:0

and use dynamic reconfigure to set detection and object tracking by

rosrun dynamic_reconfigure dynparam set /deva_node classes "cloth; cup; bottle;"

TODO

add rostest and docker build test
add CoTracker and Track Any Point.

tracking_ros's People

Contributors

Stargazers

Watchers

Forkers

hiroishida kanazawanaoaki homeworldl tfgbestneal

tracking_ros's Issues

Model try to use gpu though device=cpu is specified

h-ishida@azarashi:~/misc/tracking_ros$ ./run_docker -host pr1040 -mount ./launch -name track.launch     input_image:=/kinect_head/rgb/image_rect_color  mode:=prompt  model_type:=vit_t device:=cpu

==========
== CUDA ==
==========

CUDA Version 12.1.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use the NVIDIA Container Toolkit to start this container with GPU support; see
   https://docs.nvidia.com/datacenter/cloud-native/ .

*************************
** DEPRECATION NOTICE! **
*************************
THIS IMAGE IS DEPRECATED and is scheduled for DELETION.
    https://gitlab.com/nvidia/container-images/cuda/blob/master/doc/support-policy.md

set ROS_IP and ROS_HOSTNAME to 133.11.216.116
set ROS_MASTER_URI to http://pr1040:11311
... logging to /home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/roslaunch-azarashi-102.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://133.11.216.116:43999/

SUMMARY
========

PARAMETERS
 * /cutie_node/device: cpu
 * /cutie_node/with_bbox: True
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /sam_node/device: cpu
 * /sam_node/mode: prompt
 * /sam_node/model_type: vit_t
 * /sam_node/num_slots: 10

NODES
  /
    cutie_node (tracking_ros/cutie_node.py)
    sam_node (tracking_ros/sam_node.py)

ROS_MASTER_URI=http://pr1040:11311

process[sam_node-1]: started with pid [110]
process[cutie_node-2]: started with pid [134]
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/matplotlib/projections/__init__.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
  warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/matplotlib/projections/__init__.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
  warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
shutdown request: [/sam_node] Reason: new node registered with same name
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/huggingface_hub/utils/_runtime.py:184: UserWarning: Pydantic is installed but cannot be imported. Please check your installation. `huggingface_hub` will default to not using Pydantic. Error message: '{e}'
  warnings.warn(
Downloading: "https://download.pytorch.org/models/resnet50-19c8e357.pth" to /home/user/.cache/torch/hub/checkpoints/resnet50-19c8e357.pth
 79%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉                                   | 77.6M/97.8M [00:00<00:00, 117MB/s]/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/mobile_sam/modeling/tiny_vit_sam.py:656: UserWarning: Overwriting tiny_vit_5m_224 in registry with mobile_sam.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/mobile_sam/modeling/tiny_vit_sam.py:656: UserWarning: Overwriting tiny_vit_11m_224 in registry with mobile_sam.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/mobile_sam/modeling/tiny_vit_sam.py:656: UserWarning: Overwriting tiny_vit_21m_224 in registry with mobile_sam.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/mobile_sam/modeling/tiny_vit_sam.py:656: UserWarning: Overwriting tiny_vit_21m_384 in registry with mobile_sam.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/user/tracking_ws/devel/.private/tracking_ros/share/tracking_ros/venv/lib/python3.9/site-packages/mobile_sam/modeling/tiny_vit_sam.py:656: UserWarning: Overwriting tiny_vit_21m_512 in registry with mobile_sam.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 97.8M/97.8M [00:00<00:00, 116MB/s]
Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /home/user/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 44.7M/44.7M [00:00<00:00, 115MB/s]
[sam_node-1] process has finished cleanly
log file: /home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/sam_node-1*.log
Traceback (most recent call last):
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/cutie_node.py", line 110, in <module>
    node = CutieNode()
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/cutie_node.py", line 21, in __init__
    self.predictor = self.cutie_config.get_predictor()
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/model_config.py", line 111, in get_predictor
    model_weights = torch.load(cfg.weights)
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 1014, in load
    return _load(opened_zipfile,
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 1422, in _load
    result = unpickler.load()
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 1392, in persistent_load
    typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 1366, in load_tensor
    wrap_storage=restore_location(storage, location),
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 381, in default_restore_location
    result = fn(storage, location)
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 274, in _cuda_deserialize
    device = validate_cuda_device(location)
  File "/home/user/.local/lib/python3.9/site-packages/torch/serialization.py", line 258, in validate_cuda_device
    raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
[cutie_node-2] process has died [pid 134, exit code 1, cmd /home/user/tracking_ws/devel/lib/tracking_ros/cutie_node.py ~input_image:=/kinect_head/rgb/image_rect_color ~input_segmentation:=/sam_node/output/segmentation __name:=cutie_node __log:=/home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/cutie_node-2.log].
log file: /home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/cutie_node-2*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done

The version of python packages inside the docker container is

user@5ca2f6ec7dd2:~$ pip3 list
Package                    Version             
-------------------------- --------------------
actionlib                  1.14.0              
angles                     1.9.13              
appdirs                    1.4.3               
atomicwrites               1.1.5               
attrs                      19.3.0              
autobahn                   17.10.1             
Automat                    0.8.0               
bcrypt                     3.1.7               
beautifulsoup4             4.8.2               
blinker                    1.4                 
blosc                      1.7.0               
bondpy                     1.8.6               
boto3                      1.9.253             
botocore                   1.16.19             
camera-calibration-parsers 1.12.0              
catkin                     0.8.10              
catkin-pkg                 1.0.0               
catkin-pkg-modules         1.0.0               
catkin-tools               0.9.2               
catkin-virtualenv          0.6.1               
cbor                       1.0.0               
certifi                    2019.11.28          
chardet                    3.0.4               
Click                      7.0                 
cloudpickle                1.3.0               
colorama                   0.4.3               
constantly                 15.1.0              
cryptography               2.8                 
cupshelpers                1.0                 
cv-bridge                  1.16.2              
cycler                     0.10.0              
Cython                     0.29.14             
dask                       2.8.1+dfsg          
dbus-python                1.2.16              
decorator                  4.4.2               
defer                      1.0.6               
defusedxml                 0.6.0               
Deprecated                 1.2.7               
diagnostic-updater         1.11.0              
distlib                    0.3.0               
distro                     1.4.0               
docutils                   0.16                
dynamic-reconfigure        1.7.3               
empy                       3.3.2               
entrypoints                0.3                 
et-xmlfile                 1.0.1               
filelock                   3.13.1              
fsspec                     0.6.1               
gdown                      4.7.3               
gencpp                     0.7.0               
geneus                     3.0.0               
genlisp                    0.4.18              
genmsg                     0.6.0               
gennodejs                  2.0.2               
genpy                      0.6.15              
h5py                       2.10.0              
html5lib                   1.0.1               
httplib2                   0.14.0              
hyperlink                  19.0.0              
idna                       2.8                 
image-geometry             1.16.2              
imageio                    2.4.1               
imagesift                  1.2.17              
importlib-metadata         1.5.0               
incremental                16.10.1             
interactive-markers        1.12.0              
jdcal                      1.0                 
jmespath                   0.9.4               
joblib                     0.14.0              
joint-state-publisher      1.15.1              
jsk-data                   2.2.12              
jsk-network-tools          2.2.12              
jsk-perception             1.2.17              
jsk-recognition-msgs       1.2.17              
jsk-recognition-utils      1.2.17              
jsk-rqt-plugins            2.1.8               
jsk-rviz-plugins           2.1.8               
jsk-tools                  2.2.12              
jsk-topic-tools            2.2.12              
keyring                    18.0.1              
kiwisolver                 1.0.1               
language-selector          0.1                 
laser-geometry             1.6.7               
launchpadlib               1.10.13             
lazr.restfulclient         0.14.2              
lazr.uri                   1.0.3               
locket                     0.2.0               
lxml                       4.5.0               
lz4                        3.0.2+dfsg          
macaroonbakery             1.3.1               
matplotlib                 3.1.2               
message-filters            1.16.0              
more-itertools             4.2.0               
moveit-core                1.1.13              
mpi4py                     3.0.3               
netifaces                  0.10.4              
networkx                   2.4                 
nose                       1.3.7               
numexpr                    2.7.1               
numpy                      1.17.4              
oauthlib                   3.1.0               
olefile                    0.46                
openni2-launch             1.6.1               
openpyxl                   3.0.3               
osrf-pycommon              2.0.2               
packaging                  20.3                
pandas                     0.25.3              
paramiko                   2.6.0               
partd                      1.0.0               
Pillow                     7.0.0               
pip                        20.0.2              
pluggy                     0.13.0              
progressbar                2.5                 
protobuf                   3.6.1               
psutil                     5.5.1               
py                         1.8.1               
py-ubjson                  0.14.0              
pyasn1                     0.4.2               
pyasn1-modules             0.2.1               
pycairo                    1.16.2              
pycryptodomex              3.6.1               
pycups                     1.9.73              
PyGithub                   1.43.7              
PyGObject                  3.36.0              
PyHamcrest                 1.9.0               
PyJWT                      1.7.1               
pymacaroons                0.13.0              
PyNaCl                     1.3.0               
pyOpenSSL                  19.0.0              
pyparsing                  2.4.6               
pypng                      0.0.20              
PyQRCode                   1.2.1               
PyQt5                      5.14.1              
pyRFC3339                  1.1                 
PySocks                    1.7.1               
pytest                     4.6.9               
python-apt                 2.0.1+ubuntu0.20.4.1
python-dateutil            2.7.3               
python-gnupg               0.4.5               
python-qt-binding          0.4.4               
python-snappy              0.5.3               
PyTrie                     0.2                 
pytz                       2019.3              
PyWavelets                 0.5.1               
PyYAML                     5.3.1               
pyzmq                      18.1.1              
qt-gui                     0.4.2               
qt-gui-cpp                 0.4.2               
qt-gui-py-common           0.4.2               
requests                   2.22.0              
requests-unixsocket        0.2.0               
resource-retriever         1.12.7              
roman                      2.0.0               
rosbag                     1.16.0              
rosboost-cfg               1.15.8              
rosclean                   1.15.8              
roscreate                  1.15.8              
rosdep                     0.22.2              
rosdep-modules             0.22.2              
rosdistro                  0.9.0               
rosdistro-modules          0.9.0               
rosgraph                   1.16.0              
rosinstall                 0.7.8               
roslaunch                  1.16.0              
roslib                     1.15.8              
roslz4                     1.16.0              
rosmake                    1.15.8              
rosmaster                  1.16.0              
rosmsg                     1.16.0              
rosnode                    1.16.0              
rosparam                   1.16.0              
rospkg                     1.5.0               
rospkg-modules             1.5.0               
rospy                      1.16.0              
rosservice                 1.16.0              
rostest                    1.16.0              
rostopic                   1.16.0              
rosunit                    1.15.8              
roswtf                     1.16.0              
rqt-bag                    0.5.1               
rqt-console                0.4.12              
rqt-gui                    0.5.3               
rqt-gui-py                 0.5.3               
rqt-image-view             0.4.17              
rqt-logger-level           0.4.12              
rqt-plot                   0.4.13              
rqt-py-common              0.5.3               
rqt-reconfigure            0.5.5               
rviz                       1.14.20             
s3transfer                 0.3.3               
scikit-image               0.16.2              
scikit-learn               0.22.2.post1        
scipy                      1.3.3               
SecretStorage              2.3.1               
sensor-msgs                1.13.1              
service-identity           18.1.0              
setuptools                 45.2.0              
Shapely                    1.7.0               
simplejson                 3.16.0              
sip                        4.19.21             
six                        1.14.0              
smclib                     1.8.6               
sound-play                 0.3.17              
soupsieve                  1.9.5               
srdfdom                    0.6.4               
systemd-python             234                 
tables                     3.6.1               
tabulate                   0.8.6               
texttable                  1.6.2               
tf                         1.13.2              
tf-conversions             1.13.2              
tf2-geometry-msgs          0.7.7               
tf2-kdl                    0.7.7               
tf2-py                     0.7.7               
tf2-ros                    0.7.7               
toolz                      0.9.0               
topic-tools                1.16.0              
tqdm                       4.66.1              
Twisted                    18.9.0              
txaio                      2.10.0              
u-msgpack-python           2.1                 
urdfdom-py                 0.4.6               
urllib3                    1.25.8              
vcstools                   0.1.42              
virtualenv                 20.0.17             
wadllib                    1.3.3               
wcwidth                    0.1.8               
webencodings               0.5.1               
wheel                      0.34.2              
wrapt                      1.11.2              
wsaccel                    0.6.2               
wstool                     0.1.18              
xacro                      1.14.16             
xlrd                       1.1.0               
xlwt                       1.3.0               
zipp                       1.0.0               
zope.interface             4.7.1

Make dockerbuild configurable for cuda and torch version

Dockerfile of this package is built upon cuda==12 and torch==2.1 but it can also run on cuda==11.* and torch>=1.12.
See #4 (comment).
I think it is better to make it configurable which version of cuda and torch to use.

There are several option like using docker compose or makefile.

RuntimeError: Unexpected error from cudaGetDeviceCount(). error occurred

As a quick report, the following error occurred when I tried to run the program using docker.

$ ./run_docker -host pr1040 -mount ./launch -name track.launch     input_image:=/kinect_head/rgb/image_rect_color     mode:=prompt     model_type:=vit_t     device:=cuda:1
---
Traceback (most recent call last):
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/sam_node.py", line 301, in <module>
    node = SAMNode()
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/sam_node.py", line 21, in __init__
    self.predictor = self.sam_config.get_predictor()
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/model_config.py", line 70, in get_predictor
    model.to(device=self.device).eval()
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  [Previous line repeated 2 more times]
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 833, in _apply
    param_applied = fn(param)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/home/user/.local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 298, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW
100%|██████████████████████████████████████████████████| 97.8M/97.8M [00:00<00:00, 115MB/s]
Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /home/user/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth
100%|██████████████████████████████████████████████████| 44.7M/44.7M [00:00<00:00, 114MB/s]
[sam_node-1] process has died [pid 110, exit code 1, cmd /home/user/tracking_ws/devel/lib/tracking_ros/sam_node.py ~input_image:=/kinect_head/rgb/image_rect_color ~input_point:=/sam_node/output/segmentation_image/screenpoint ~input_bbox:=/sam_node/output/segmentation_image/screenrectangle __name:=sam_node __log:=/home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/sam_node-1.log].
log file: /home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/sam_node-1*.log
Traceback (most recent call last):
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/cutie_node.py", line 110, in <module>
    node = CutieNode()
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/cutie_node.py", line 21, in __init__
    self.predictor = self.cutie_config.get_predictor()
  File "/home/user/tracking_ws/src/tracking_ros/node_scripts/model_config.py", line 110, in get_predictor
    cutie = CUTIE(cfg).to(self.device).eval()
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 833, in _apply
    param_applied = fn(param)
  File "/home/user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/home/user/.local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 298, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW
[cutie_node-2] process has died [pid 118, exit code 1, cmd /home/user/tracking_ws/devel/lib/tracking_ros/cutie_node.py ~input_image:=/kinect_head/rgb/image_rect_color ~input_segmentation:=/sam_node/output/segmentation __name:=cutie_node __log:=/home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/cutie_node-2.log].
log file: /home/user/.ros/log/20240114-153226_af282e38-b2a6-11ee-97d3-a571c3c52472/cutie_node-2*.log

I think it may be due to GPU architecture or nvida driver version and CUDA version correspondence.

My desktop PC (Error occurred)

$ lspci | grep -i nvidia
1c:00.0 VGA compatible controller: NVIDIA Corporation Device 2204 (rev a1)
1c:00.1 Audio device: NVIDIA Corporation Device 1aef (rev a1)
8d:00.0 VGA compatible controller: NVIDIA Corporation Device 2204 (rev a1)
8d:00.1 Audio device: NVIDIA Corporation Device 1aef (rev a1)

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  470.223.02  Sat Oct  7 15:39:11 UTC 2023
GCC version:  gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.2)

dlbox (Error occurred)

$ lspci | grep -i nvidia
1c:00.0 VGA compatible controller: NVIDIA Corporation Device 2204 (rev a1)
1c:00.1 Audio device: NVIDIA Corporation Device 1aef (rev a1)
8d:00.0 VGA compatible controller: NVIDIA Corporation Device 2204 (rev a1)
8d:00.1 Audio device: NVIDIA Corporation Device 1aef (rev a1)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.