How to improve shape optimization about ffhq-uv HOT 6 CLOSED

From the pre-processing results in the figure below, the face parsing does not exclude the hand area. The fitting process would be affected by the hand texture. More accurate face parsing results might helpful.

from ffhq-uv.

@csbhr , This is a case of occlusion. But I have tested with other clear images too.
Normally, It's coming more round like 3D shapes.
Check this reference input and corresponding output,

from ffhq-uv.

From the 68 landmarks in the image above (gt landmarks in blue and fitted landmarks in red), it can be seen that the landmarks at the bottom of the face are not well fitted. This may be caused by the weight used in landmark loss (see the image below, or the source codes).

On Line79-Line81, we weighted the landmarks of nose and mouth more heavily.
Perhaps, setting the weight of all landmarks to 1 is helpful for fitting the bottom contour of the face.

from ffhq-uv.

@csbhr Tried what you suggested and now the outer shapes are better. But on the cost of a slightly less optimised mouth and nose area. We can't even observe those if we see them separately. We can notice those gaps after overlapping the previous result with the new result.
For now, I tried with assigning all landmarks weights as 1.
I'll try with lesser weight for mouth and nose instead of 20. It may be a better trade-off.
Thanks @csbhr.

from ffhq-uv.

Thank you for sharing your experiments. Hope our codes are useful to you.

from ffhq-uv.

如何用更轻的口鼻

from ffhq-uv.