How to get mAP and plot curves in python scripts, instead of `eval tools`? about img2pose HOT 4 CLOSED

Maybe something goes wrong for my last test!
I re-ran again and get the right result as bellow,

We can conclude:

1. 1-gpu training script provided by your readme can reproduce the accuracy of paper
2. https://github.com/wondervictor/WiderFace-Evaluation is about 2~4% lower than that of offical eval tools

from img2pose.

Hello,

The eval tools are just Matlab scripts provided with the WIDER FACE dataset.
It is the official scripts to get mAP and plot curves on WIDER FACE.
You can find a short description on how to use it in #8.

Although I haven't tested, you can also try non official python code from any GitHub repo. One example can be found at https://github.com/wondervictor/WiderFace-Evaluation

Hope this helps.

from img2pose.

@vitoralbiero
As the paper reported, the AP@val is

Easy   Val AP: 0.908
Medium Val AP: 0.899
Hard   Val AP: 0.847

However, the result of scripts in https://github.com/wondervictor/WiderFace-Evaluation is much lower than the former one.

Easy   Val AP: 0.8469243842505618
Medium Val AP: 0.827844108399036
Hard   Val AP: 0.7493400384483289

I use the model mentioned in your readme

python3 evaluation/evaluate_wider.py \
--dataset_path datasets/WIDER_Face/WIDER_val/images/ \
--dataset_list datasets/WIDER_Face/wider_face_split/wider_face_val_bbx_gt.txt \
--pretrained_path models/img2pose_v1.pth \
--output_path results/WIDER_FACE/Val/

I also use eval tools provided by WIDER FACE to get the result of your publised model img2pose_v1.pth ,

Easy   Val AP: 0.876
Medium Val AP: 0.855 
Hard   Val AP: 0.79

No matter which test script being used, model img2pose_v1.pth can not achieve the precision of paper, begging for your advices, thanks so much!

from img2pose.

Hi @KindleHe,

I re-ran the evaluate_wider.py script using the img2pose_v1.pth model and got the same results as reported in our paper using eval_tools.
Did you change anything inside evaluation/evaluate_wider.py or are you using it as provided?
When you ran wider_eval.m, does it give you any error?

from img2pose.