could the network be trained with only action video but no pose images/video? about deephar HOT 4 CLOSED

Hi @ardeal ,

Thank you for your interest. Here are my comments.

i. For this question, please check our Table 6 in our TPAMI paper. You can train only action, but the scores are lower.
ii. I don't know, it depends on how difficult is your scenario. You'll have to try it.
iii. Similarly to ii, it depends on the videos/images that you have in our scenario.
iv. You can find my e-mail in my personal web page.

Best,
Diogo

from deephar.

Hi Diogo,

Many thanks for your reply!

for my scenario, I have the human action videos, and I could label all videos or split the action to separate frames for training. if I would like to train 2D pose and and 3D pose, should I use MPII 2D pose images and Human3.6 3D pose images? or should I create new 2D and 3D pose images?

for the training, I have no idea about the steps, could you please help to see whether the following understanding is correct or not?

1. input many batches(4 or 8 or 16 images or etc.) of 2D pose images to the network to train.
2. input many batches(4 or 8 or 16 images or etc.) of 3D pose images to the network to train.
3. separate the action videos to many frames(such as 8, or 9 or other number images), and input those frames separated from 1 action to train. and then continue with the next action video.

Thanks,
Ardeal

from deephar.

Hi,

Maybe in your case you'll need to collect or use some other datasets than MPII or H36M, since these datasets have the camera usually in front of the people.
For the training process, you can input as many images in the batch as you can (depending on your GPU memory). For each image that you have, you can supervise the network with the labels that you have, either 2D, 3D poses and action labels (for a sequence of frames). If you don't have a specific label for a given sequence of frames, you propagate zero in the respective loss function.

Best,

from deephar.

Stale issue message

from deephar.