4D Humans Poses Conversion Standard about 4d-humans HOT 5 CLOSED

Could you provide more details, or give a short code snippet, about how you do the conversion? Do you get an error when you do the conversion, or does the output looks off when you use it to pose SMPL?
Using a function like this should be enough to do the conversion from rotation matrix to axis angle.

from 4d-humans.

@geopavlakos Thank you so much for your comments! I think I did exactly that and it's still erroring out. The code snippet is blow:

            converted_poses = []
            for pose in smpl_p["body_pose"].astype(float):
                 pose = torch.Tensor(pose)
                 rotVec = rotation_matrix_to_angle_axis(pose)
                 converted_poses.append(rotVec)
            global_orient = torch.tensor(smpl_p["global_orient"])
            # print(global_orient[0])
            converted_poses.append(rotation_matrix_to_angle_axis(global_orient[0]))
            dictionary = {
                "poses": torch.cat(converted_poses).tolist(),
                "betas": smpl_p["betas"].astype(float).tolist(),
                "cam_intrinsics": intrinsics,
                "cam_extrinsics": estimate_extrinsics_cv2(cam_p).tolist()
            }

Another thing that I am unsure that could cause issues may be estimating the extrinsics to the person from 4Dhumans, and here's a code snippet:

def estimate_extrinsics_cv2(translation):
        tra_pred = translation   
        cam_extrinsics = np.eye(4)
        cam_extrinsics[:3, 3] = tra_pred         
        return cam_extrinsics.astype(float)

Thanks and hopefully this could work out : )

from 4d-humans.

I believe that you are appending the global orientation parameters at the end of the pose parameters, whereas they are typically the first three values of the standard 72 SMPL pose parameters, so that could be the problem.
Also, what do you mean with erroring out. Do you get an actual error, or does the mess output look weird?

from 4d-humans.

@geopavlakos Thanks so much for pointing that out! I tried but I think it's still not working. Basically, the error from humannerf is as follows:

Traceback (most recent call last):
  File "train.py", line 45, in <module>
    main()
  File "train.py", line 39, in main
    train_dataloader=train_loader, prog_loader = prog_loader)
  File "core/train/trainers/human_nerf/trainer.py", line 215, in train_batch_wise
    self.train(batch_idx,batch,prog_loader, epoch, train_dataloader)
  File "core/train/trainers/human_nerf/trainer.py", line 233, in train
    div_indices=data['patch_div_indices'],batch = batch)
  File "core/train/trainers/human_nerf/trainer.py", line 148, in get_loss
    rgb = net_output['rgb']
KeyError: 'rgb'

And I think this is because the ray is not able to intersect bbox, basically the orientation/extrinsics are off...

from 4d-humans.

Hmm, I am not sure what is the expected input format for the humannerf code, so not sure if I can help with this error. If you have a working code example that converts the output of a similar network, e.g., SPIN, to the expected format, we could help to do the mapping to 4D Humans.

from 4d-humans.