facetracking - faceMask vs transformmatrix about mediapipe-touchdesigner HOT 3 OPEN

I'm playing with mediapipe, amazing work !! Thanks to share !
In the face_tracking_examples we can clearly see that the torus is not following the Z- axis / distance/size of the head.
How could i do that ?

Thanks a lot.

from mediapipe-touchdesigner.

Hey @yrostudio thanks for the kind words and glad you're having fun with the plugin!

I found the same issues you did when I was playing with the plugin and the short answer is I don't know why it happens. It feels like it's something to do with the dimensions being different. The canonical face mask is supposedly roughly real-world-size, so has a height of 20 units ~ 20 cm. The face mask that we're animating/generating in the MediaPipe tox is based on screen normalised coordinates (0-1), so has a height of something like 0.8 units.

The transformation matrix we receive from MediaPipe is supposed to map from the canonical face model to screen coordinates, but I get the feeling there's a wrong scale being used somewhere (or I've applied it incorrectly in the face_mapping tox). I'm not quite sure what it is, but if you figure it out I'd love to know and we can roll it into the next release!

from mediapipe-touchdesigner.

Hi @domisjustanumber,
i found this discussion : google-ai-edge/mediapipe#1895
don't know if it helps.

Tracking eye size / iris ? to get a Z scale ?

I have no clue how to help :)

from mediapipe-touchdesigner.