Question on fine-tuning for face pose evaluation about img2pose HOT 14 CLOSED

The code to fine-tune on 300W-LP is the same as in here, we just didn't release the annotations.
If you want to fine-tune yourself, you will need to create 300W-LP ground-truth data, load the pre-trained WIDER-FACE weights when training, and fine-tune on 300W-LP.

If you prefer, the fine-tuned pre-trained model can be found here.

from img2pose.

Sorry, you missed the link in the first sentence. I cannot see it.

from img2pose.

Edited.

from img2pose.

So would you release the annotations?

As I see, without fine-tuning, the network performance is very poor.

Performance comparison on AFLW2000:

from img2pose.

We have no plans on releasing these annotations at this time.
You can use our instructions to annotate it if you would like.

The performance only appears to be poor because of a problem with Euler angles.
This causes samples that have small qualitative and other format error to be computed as a large error.
You can see an example here.

Apart from evaluating with Euler angles, we recommend using the model without fine-tuning.

from img2pose.

Thank you so much. ❤️

from img2pose.

The performance only appears to be poor because of a problem with Euler angles.

So what if we use 'xyz' and 'zxy' to decode the output at the same time, and choose the best as the final result? (I mean, for each face.)

from img2pose.

We could do that, but to be fair with other models compared to, we only use xyz.

from img2pose.

Alright, thank you.

from img2pose.

Happy new year!

As I have tried. Even though I choose the minimum error between xyz and zxy decoding, the redults are not very well as well.

So, the fine-tuning plays an important role in the pose evalution.

from img2pose.

Thanks, to you too!

Have you also converted the GT to zxy? Just swapping axis won't work, and the errors will be big.
You'll need to do something like:
gt_zxy = Rotation.from_euler("xyz", pose_target[:3], degrees=True).as_euler("zxy", degrees=True)
Also make sure when converting the prediction to Euler angles in zxy format that z and y have a negative sign and x doesn't.

Regardless, the fine-tuning tries to constrain the poses learned to less than 90 degrees, that's why it performs better when tested with Euler angles.

from img2pose.

OK, I retest the model without fine-tuning, the results are as following:

from img2pose.

I still consider this result as a big gap. 🐰

from img2pose.

If what you care most is pose evaluation in Euler angles, then sure, use the fine-tuned model.

from img2pose.