Hello, I wonder to know how many gpus did you use? and the experiment equipment? The article does not mention them, thank you for your reply. And by the way, how much time does one eopch consume?

Hi,
Thanks for sharing this great work.
I retrained the poseur_res50_coco_256x192 model and got AP=73.0
The AP result is lower than the AP=75,4 in Table 2 (a)
How can I get the result in the paper?
Thanks a lot!

[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 104128/104125, 2139.4 task/s, elapsed: 49s, ETA:     0s

Loading and preparing results...
DONE (t=5.50s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *keypoints*
DONE (t=9.69s).
Accumulating evaluation results...
DONE (t=0.32s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] =  0.730
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets= 20 ] =  0.889
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets= 20 ] =  0.798
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] =  0.695
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] =  0.796
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] =  0.784
 Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets= 20 ] =  0.930
 Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets= 20 ] =  0.844
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] =  0.741
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] =  0.847
2022-08-14 04:06:17,317 - mmpose - INFO - The previous best checkpoint /home/dx/projects/Poseur/work_dirs/poseur_res50_coco_256x192/best_AP_epoch_300.pth was removed
2022-08-14 04:06:18,256 - mmpose - INFO - Now best checkpoint is saved as best_AP_epoch_325.pth.
2022-08-14 04:06:18,257 - mmpose - INFO - Best AP is 0.7298 at 325 epoch.
2022-08-14 04:06:18,269 - mmpose - INFO - Epoch(val) [325][407] AP: 0.7298, AP .5: 0.8890, AP .75: 0.7979, AP (M): 0.6954, AP (L): 0.7964, AR: 0.7840, AR .5: 0.9304, AR .75: 0.8438, AR (M): 0.7406, AR (L): 0.8470

Hi, I am trying to reproduce your code. And I found out that "use_heatmap_loss" is set as True in config but the calculation of hm loss is annotated in get_loss function.
It makes me wonder that whether hm loss is used with RLELoss?
This is very important to me, forward to your reply.
Thanks a lot!

using pytorch2onnx.py but failed with :ValueError: expected 5D input (got 4D input) in function pytorch2onnx.Could you give some help?

**### Traceback (most recent call last):
File "tools/train.py", line 201, in
main()
File "tools/train.py", line 175, in main
model = build_posenet(cfg.model)
File "/home/wuxl/project/Poseur/mmpose/mmpose/models/builder.py", line 39, in build_posenet
return POSENETS.build(cfg)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 237, in build
return self.build_func(*args, **kwargs, registry=self)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 61, in build_from_cfg
raise KeyError(
KeyError: 'Poseur is not in the models registry'
Traceback (most recent call last):
File "tools/train.py", line 201, in
Traceback (most recent call last):
File "tools/train.py", line 201, in
main() main()
File "tools/train.py", line 175, in main
model = build_posenet(cfg.model)
File "/home/wuxl/project/Poseur/mmpose/mmpose/models/builder.py", line 39, in build_posenet
return POSENETS.build(cfg)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 237, in build
return self.build_func(*args, **kwargs, registry=self)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 61, in build_from_cfg
raise KeyError(
KeyError: 'Poseur is not in the models registry'
Traceback (most recent call last):

File "tools/train.py", line 175, in main
File "tools/train.py", line 201, in
model = build_posenet(cfg.model)
File "/home/wuxl/project/Poseur/mmpose/mmpose/models/builder.py", line 39, in build_posenet
return POSENETS.build(cfg)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 237, in build
return self.build_func(*args, **kwargs, registry=self)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 61, in build_from_cfg
raise KeyError(
KeyError: 'Poseur is not in the models registry'
main()
File "tools/train.py", line 175, in main
model = build_posenet(cfg.model)
File "/home/wuxl/project/Poseur/mmpose/mmpose/models/builder.py", line 39, in build_posenet
return POSENETS.build(cfg)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 237, in build
return self.build_func(*args, **kwargs, registry=self)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/mmcv/utils/registry.py", line 61, in build_from_cfg
raise KeyError(
KeyError: 'Poseur is not in the models registry'
Killing subprocess 225662
Killing subprocess 225663
Killing subprocess 225664
Killing subprocess 225665
Traceback (most recent call last):
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in
main()
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
sigkill_handler(signal.SIGTERM, None) # not coming back
File "/home/wuxl/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/wuxl/anaconda3/envs/pytorch/bin/python', '-u', 'tools/train.py', '--local_rank=3', 'configs/poseur/coco/poseur_res50_coco_256x192.py', '--launcher', 'pytorch', '--work-dir', 'work_dirs/poseur_res50_coco_256x192']' returned non-zero exit status 1.
(pytorch) wuxl@sitonholy:~/project/Poseur$

Hi! I train the model by bash tools/dist_train.sh configs/poseur/coco/poseur_res50_coco_256x192.py 4 --work-dir work_dirs/poseur_res50_coco_256x192 ,but here get something wrong as the above, can you help me ,thank you!

Thanks for sharing this awesome work, but I encountered with some issues when trying to use this method. May I know the version of mmpose and mmcv you use?

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

Attention: please create a new virtual environment for OpenMMLab 2.0.

I want to retrain on Mpii ,but i can't find implement details ,so how to train on mpii?

Hello, I have trained on my own dataset with 29 keypoints, and there is an error in poseur_head.py : the size of enc_outputs is [32, 17, 2](32 is the batchsize, 17 is the number of keypoints), it doesn't match with my 29 keypoints. Then I find in the class of PoseurTransformer_v3 in transformer.py, num_joints is initialized to 17, so I changed it to 29 and get started training. But the result epoch AP is 0, and the training log's loss and acc are unuausl as blow:

INFO - Epoch [1][50/1446] lr: 9.890e-05, eta: 4 days, 5:02:52, time: 1.198, data_time: 0.115, memory: 23176, enc_rle_loss: 319.5232, dec_rle_loss_0: 1515.8930, dec_rle_loss_1: 1094.4419, dec_rle_loss_2: 917.2394, dec_rle_loss_3: 1053.5126, dec_rle_loss_4: 977.9670, dec_rle_loss_5: 1089.1097, enc_coord_acc: 0.0002, dec_coord_acc: 0.0027, loss: 6967.6868

I don't kown how to train on my own dataset.

It seems the implement of Prediction uncertainty estimation which mentioned in Chapter 3.3 is missing. @YongtaoGe

Hello, I have tried many times, and I really feel that the input shape of the key point detection head is wrong. What is the input exactly? The output of Neck is an array of four tensors, but there is an error in the input to the key point detection head!
"
File "E:\python_console\pose_2\Poseur-main\mmpose\models\heads\poseur_head.py", line 328, in forward
batch_size = mlvl_feats[0].size(0)
AttributeError: 'EasyDict' object has no attribute 'size'
"

hi, it's being 3 month, wehen will the full model release?

Hi, I'm new to this project, thanks for your great work, based on the algorithm, I'm wondering if we can directly train it with multi-person cases, or even multi-class multi-instance cases (ignore the grouping)?

Thank you for making this wonderful repository available to the public.

The URL for poseur_mobilenetv2_coco_256x192 appears to be wrong because downloading a checkpoint for poseur_mobilenetv2_coco_256x192 downloads a checkpoint for poseur_res50_coco_256x192.

• COCO Keypoint Detection Results
  https://github.com/aim-uofa/Poseur#coco-keypoint-detection-results

• poseur_mobilenetv2_coco_256x192 URL
  https://cloudstor.aarnet.edu.au/plus/s/UXr1Dn9w6ja4fM9/download

• poseur_res50_coco_256x192 URL
  https://cloudstor.aarnet.edu.au/plus/s/UXr1Dn9w6ja4fM9/download

Thank you.

Here's a question about RLELoss_poseur that confuses me a lot
In mmpose, The pred_jts are not processed in any way，and passing sigma through the sigmoid function，like this：
pred = output[:, :, :2]
sigma = output[:, :, 2:4].sigmoid()
then，we can calculate the error (or bar_mu in Poseur):
error = (pred - target) / (sigma + 1e-9)
However, I notice that the operation in Poseur .
outputs_coord = tmp.sigmoid()
outputs_sigmas = torch.stack(outputs_sigmas).sigmoid()
Could you tell me why we need to use the extra sigmoid function when getting the coordinates.
Is there something I missed or misunderstood?