Use it with Intel Realsense about mediapipe-touchdesigner HOT 3 CLOSED

Hey @niknaim thanks for the kind words!

Mediapipe only takes a 2D video image as an input and infers distance purely based on that, so you can't feed it a depth map directly.

That being said, you can probably look up the xy position of each point returned by MediaPipe and then overwrite the Z coordinates with the one you get from the RealSense camera to get more accurate depth positions.

from mediapipe-touchdesigner.

Thanks for your fast reply. how can I get the the xy position of all points? I only see the xy positions, with, height ans scale of the objects tracked by the object detection. Also I dont see any Z ccordinate, which I could overwrite.

from mediapipe-touchdesigner.

oh if you're object detection, you only get the xy coordinates of the box containing the object. You could still add Z positions to those values from your RealSense data and/or crop the depth data to the same box and take an average depth of the pixels in the box?

There's lots of options, but they're all beyond the scope of MediaPipe

from mediapipe-touchdesigner.