jeff-sjtu / crowdpose Goto Github PK

学长您好，请问您方便透露一下你的crowdpose的cvpr初始评审的得分吗？（三个reviewer）。我正在学习你们的论文。

I use both HRNet model trained on COCO and trained on CrowdPose to evaluate on Test-Set. Both models have the problem that their pose estimation results using ground truth bounding box are worse than the results using detected bounding box. Not worse a little, but a lot. Especially for the model trained on COCO, up to 20% difference. Evaluated with ground truth bbox, I have 58.6% mAP, but with detected bbox, I can get 76.6% mAP.

This is my evaluation result :

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.808
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.955
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.870
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.850
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.961
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.899
Average Precision (AP) @[ IoU=0.50:0.95 | area= easy | maxDets= 20 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.790
Average Precision (AP) @[ IoU=0.50:0.95 | area= hard | maxDets= 20 ] = -1.000

Why is there a -1 value? My prediction result format is the same as the prediction json format you provided. Looking forward for your response, thank you！

what is the significance of crowdindex ?

(gluon) root@ops-test:/home/sunchen/pose/cocoapi/PythonAPI/PythonAPI# sh install.sh
running install
/root/miniconda3/envs/sunchen-gluon/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
/root/miniconda3/envs/sunchen-gluon/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running bdist_egg
running egg_info
writing crowdposetools.egg-info/PKG-INFO
writing dependency_links to crowdposetools.egg-info/dependency_links.txt
writing top-level names to crowdposetools.egg-info/top_level.txt
reading manifest file 'crowdposetools.egg-info/SOURCES.txt'
writing manifest file 'crowdposetools.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
building 'crowdposetools._mask' extension
gcc -pthread -B /root/miniconda3/envs/gluon/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/envs/gluon/include -fPIC -O2 -isystem /root/miniconda3/envs/gluon/include -fPIC -I/root/miniconda3/envs/sunchen-gluon/lib/python3.9/site-packages/numpy/core/include -I../common -I/root/miniconda3/envs/gluon/include/python3.9 -c ../common/maskApi.c -o build/temp.linux-x86_64-cpython-39/../common/maskApi.o -Wno-cpp -Wno-unused-function -std=c99
gcc -pthread -B /root/miniconda3/envs/gluon/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/envs/gluon/include -fPIC -O2 -isystem /root/miniconda3/envs/gluon/include -fPIC -I/root/miniconda3/envs/sunchen-gluon/lib/python3.9/site-packages/numpy/core/include -I../common -I/root/miniconda3/envs/gluon/include/python3.9 -c crowdposetools/_mask.c -o build/temp.linux-x86_64-cpython-39/crowdposetools/_mask.o -Wno-cpp -Wno-unused-function -std=c99
crowdposetools/_mask.c: In function ‘__Pyx_modinit_type_init_code’:
crowdposetools/_mask.c:12158:42: error: ‘PyTypeObject {aka struct _typeobject}’ has no member named ‘tp_print’
__pyx_type_14crowdposetools_5_mask_RLEs.tp_print = 0;
^
crowdposetools/_mask.c:12163:43: error: ‘PyTypeObject {aka struct _typeobject}’ has no member named ‘tp_print’
__pyx_type_14crowdposetools_5_mask_Masks.tp_print = 0;
^
crowdposetools/_mask.c: In function ‘__Pyx_ParseOptionalKeywords’:
crowdposetools/_mask.c:12747:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12747:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^
crowdposetools/_mask.c:12747:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12747:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12747:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^
crowdposetools/_mask.c:12747:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12763:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key))
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12763:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key))
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^
crowdposetools/_mask.c:12763:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key))
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12763:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key))
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
crowdposetools/_mask.c:12763:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key))
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^
crowdposetools/_mask.c:12763:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key))
^
In file included from /root/miniconda3/envs/gluon/include/python3.9/unicodeobject.h:1026:0,
from /root/miniconda3/envs/gluon/include/python3.9/Python.h:93,
from crowdposetools/_mask.c:32:
/root/miniconda3/envs/gluon/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^
error: command '/usr/bin/gcc' failed with exit code 1

Hello @Jeff-sjtu , thanks for providing the code and dataset. I am trying to produce numbers with OpenPifPaf on your CrowdPose dataset. When I take a COCO pretrained model, the test AP on CrowdPose is very good and I am trying to understand whether that is realistic. When you assembled your train/test split, did you keep the train/test separation from COCO?

What I am trying to understand is whether there are COCO train images in the CrowdPose val and test split which would distort my numbers.

Thanks for your help.

I am researching pose inference in crowd situations using crowdpose data and performance evaluation tools.
I would like to classify other data by calculating the crowd index like crowdpose. If possible, can you tell me the method or code of how the crowdindex was calculated?

It seems the 'num_keypoints' is always less than the actual marked points. I'm wondering do you have some pre-processing methods that aren't mentioned in the Paper.
BTW, your dataset's quality could use some optimization. Too much 'head' marks are misplaced, and too much bbox and keypoints don't match up. And even have an image (105273.jpg) may come from places like pornhub. I didn't go through all 20 thousand images, so I don't know if it is the only one.

While running python setup.py install on a windows machine the following error occured.

(base) C:\Users\eii\Downloads\CrowdPose-master\crowdpose-api\PythonAPI>python setup.py install
running install
running build
running build_py
running build_ext
building 'crowdposetools._mask' extension
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\eii\miniconda3\lib\site-packages\numpy\core\include
 -I../common -IC:\Users\eii\miniconda3\include -IC:\Users\eii\miniconda3\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\include -IC:\Program Files (x86
)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\
Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program
 Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /Tc../common/maskApi.c /Fobuild\temp.win-amd64-3.9\Release\../common/maskApi.obj -Wno-cpp -Wno-unused-function -std=c99

**cl : Command line error D8021 : invalid numeric argument '/Wno-cpp'
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2**

OS: WIndows 10 Pro 64 Bit
Python: 3.9.2
Conda: 4.10.3

下载经常失败，能给一个百度云的链接么？

MIT? or GPL?
Thank you for clarification.

It seems your dataset doesn't mark the occlusion faithfully. It is apparent that your dataset's format is arranged according to the COCO dataset. According to COCO, every keypoint is marked by three numbers: x-axis, y-axis, and visibility. If the keypoint is visible, the third number is 2, if it is occluded, the number is 1, and 0 for unmarked keypoint.

But in your dataset, the visibility marks seem to be arranged randomly. There are instances with all visible keypoints marked as 1 (occluded). I'm wondering how do you set up your dataset's marking on visibility?

thanks for your great work!

i read the code about sppe GT heatmaps generating, the code above. I don't think that's consistent with the calculation of loss in the paper, like below

The keypoints for head & neck seem to be always annotated as occluded (v=1). Has that some specific reason or does someone have more information about the annotation process?

Thank your for your good job.
We follow your work, but we only find coco weight for 17 joints. could you give our your crowdpose weight based on pytorch? Thank you.

What is the difference between
trainval.json and train.json ?

I found the dic['images'][...]['width'] and dic['images'][...]['height'] in crowdpose_train.json are reversed...
And the same position in crowdpose_val.json is right.
Would this effect the result?

As per the title, I am wondering if the FPS reported in the paper Table 2 includes the person detection. I am wondering this for your model as well as AlphaPose. Thanks.

Hi, thanks for your work!

I have a problem about your evaluation metrics.

Are these AR(easy, medium , hard)(figure1) the AP(easy, medium , hard)(Figure2) shown in your paper?

When I run demo.py, an error appears.

TypeError: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.

In the annotations of the COCO Keypoint dataset the third dimension of the keypoints is a visibility flag v defined as v=0: not labeled (in which case x=y=0), v=1: labeled but not visible, and v=2: labeled and visible.

What is the definition of the third dimension of the keypoints in CrowdPose?