How to get the result of yaw, pitch, roll in paper? about img2pose HOT 7 CLOSED

Hello,

To get the yaw, pitch, and roll results of our method you can use our notebooks inside evaluation.
We updated the results, so they will differ from the current paper, but we will soon release a new version of it.
On the readme you can find out where to download the BIWI/AFLW2000-3D dataset for evaluation.

Hope this helps.

from img2pose.

I ll try it later, thanks!

from img2pose.

Hi, 3 more questions for your kind help!

1. it seems that pose_para is the original label from BIWI/AFLW2000-3D 's mat, but the code uses pose_target as final label for evaluation by applying get_pose function to calculate the relationship between threed_points (objectPoints in 3D object coordinate) and 2d-target_points (imagePoints in 2D image coordinate ), why not use pose_para as the direct label, instead of the processed label generated by get_pose function?
2. .T[:, :2] seems to generate 2d-target_points from original 3d-pt3d_68, why is this reasonable, why could we treat 2 dimension of 3d-pt3d_68 as 2d-target_points (imagePoints in 3D image coordinate )?
3. How to get the 6 Dof for BIWI/AFLW2000-3D evaluation, if firstly getting 5 landmarks by other face detection method?

(w, h) = ori_img.size
    image_intrinsics = np.array([[w + h, 0, w // 2], [0, w + h, h // 2], [0, 0, 1]])

    mat_contents = sio.loadmat(img_path[:-4] + ".mat")
    target_points = np.asarray(mat_contents['pt3d_68']).T[:, :2]

    _, pose_target = get_pose(threed_points, target_points, image_intrinsics)

    target_bbox = expand_bbox_rectangle(w, h, 1.1, 1.1, target_points, roll=pose_target[2])

    pose_para = np.asarray(mat_contents['Pose_Para'])[0][:3]
    pose_para_degrees = pose_para[:3] * (180 / math.pi)

    if np.any(np.abs(pose_para_degrees) > 99):
        continue        

from img2pose.

Hello,

1. I think you might have an older evaluation notebook, as we updated the AFLW2000-3D evaluation to use the pose_para as ground-truth. Now, we only use get_pose to get the translation vector for comparison. For BIWI, we only have the rotation matrix, so we cannot get a translation vector ground truth for comparison.

2. The 3d-pt3d_68 are landmarks in the image coordinates, with an additional z axis. By just removing the z axis, we get the accurate annotated 2D landmarks. They are not 3D points in an arbitrary 3D coordinates like threed_points are.

3. To get the 6DoF for AFLW2000-3D you just use the pose_target that is output from get_pose, or convert the original pose_para to rotation vector and then combine it with the translation part of the pose_target. For BIWI, as no landmarks are provided, you can't get ground-truth 6DoF.

Hope this helps.

from img2pose.

Hello,

1. I think you might have an older evaluation notebook, as we updated the AFLW2000-3D evaluation to use the pose_para as ground-truth. Now, we only use get_pose to get the translation vector for comparison. For BIWI, we only have the rotation matrix, so we cannot get a translation vector ground truth for comparison.
2. The 3d-pt3d_68 are landmarks in the image coordinates, with an additional z axis. By just removing the z axis, we get the accurate annotated 2D landmarks. They are not 3D points in an arbitrary 3D coordinates like threed_points are.
3. To get the 6DoF for AFLW2000-3D you just use the pose_target that is output from get_pose, or convert the original pose_para to rotation vector and then combine it with the translation part of the pose_target. For BIWI, as no landmarks are provided, you can't get ground-truth 6DoF.

Hope this helps.

For the Q-1, yes，I update to the newest version just now, and found the following changes:

pose_para = np.asarray(mat_contents["Pose_Para"])[0][:3]
pose_target_degrees = pose_para[:3] * (180 / math.pi)

### Before 
pose_target[:3] = Rotation.from_rotvec(pose_target[:3]).as_euler(seq, degrees=True) pose_pred[:3] = Rotation.from_rotvec(pose_pred[:3]).as_euler(seq, degrees=True)

### After
pose_target[:3] = pose_para_degrees  
pose_pred[:3] = convert_to_aflw(pose_pred[:3])

The newest version actually use pose_para as the direct label for the rotation vector, but why should the translation vector still be got by get_pose, instead of using pose_para as that of the rotation vector?

from img2pose.

@vitoralbiero
Hello, It seems that the sequence of yaw, pitch and roll is not right.
In the newest version, rot_mat_2 = np.transpose(rotvec) is used in convert_to_aflw function, so the sequence shoudle be
Yaw: {threed_pose[0]:.3f} Pitch: {threed_pose[1]:.3f} Roll: {threed_pose[2]:.3f}, instead of "yaw: 1, pitch 0, roll 2".
What do you think about that?

### Before
Yaw: {threed_pose[2]:.3f} Pitch: {threed_pose[1]:.3f} Roll: {threed_pose[0]:.3f}
### After
Yaw: {threed_pose[1]:.3f} Pitch: {threed_pose[0]:.3f} Roll: {threed_pose[2]:.3f}

from img2pose.

The newest version actually use pose_para as the direct label for the rotation vector, but why should the translation vector still be got by get_pose, instead of using pose_para as that of the rotation vector?

We cannot use the original translation vector because it was computed using a 3D model that we do not have access to, so the units will be different.

Hello, It seems that the sequence of yaw, pitch and roll is not right.
In the newest version, rot_mat_2 = np.transpose(rotvec) is used in convert_to_aflw function, so the sequence shoudle be
Yaw: {threed_pose[0]:.3f} Pitch: {threed_pose[1]:.3f} Roll: {threed_pose[2]:.3f}, instead of "yaw: 1, pitch 0, roll 2".
What do you think about that?

The AFLW2000-3D format is pitch, yaw, roll, so the function convert_to_aflw returns the Euler angles in this same sequence.

angle = Rotation.from_matrix(rot_mat_2).as_euler('xyz', degrees=True)
return np.array([angle[0], -angle[1], -angle[2]])

So, the evaluation is correct with yaw: 1, pitch: 0, roll: 2.

from img2pose.