soeaver / rp-r-cnn Goto Github PK

View Code? Open in Web Editor NEW

94.0 2.0 9.0 915 KB

Renovating Parsing R-CNN for Accurate Multiple Human Parsing

License: MIT License

Shell 0.04% Python 80.56% C++ 3.80% Cuda 15.60%

rp-r-cnn's Introduction

RP-R-CNN

Official implementation of Renovating Parsing R-CNN for Accurate Multiple Human Parsing (ECCV2020)

In this repository, we release the RP R-CNN code in Pytorch.

RP R-CNN architecture:

RP R-CNN output:

Citing RP R-CNN

If you use RP R-CNN, please use the following BibTeX entry.

@inproceedings{yang2020eccv,
  title = {Renovating Parsing R-CNN for Accurate Multiple Human Parsing},
  author = {Lu Yang and Qing Song and Zhihui Wang and Mengjie Hu and Chun Liu and Xueshi Xin and Wenhe Jia and Songcen Xu},
  booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
  year = {2020}
}

Installation

8 x TITAN RTX GPU
pytorch1.4
python3.6.8

Install RP R-CNN following INSTALL.md.

Results and Models

On CIHP

Backbone	LR	Det AP	mIoU	Parsing (APp50/APvol/PCP50)	DOWNLOAD
baseline	3x	68.3	56.2	64.6/54.3/60.9
R-50-FPN	3x	67.3	58.2	71.6/58.3/62.2
R-50-FPN	6x	68.2	60.2	74.1/59.5/64.9	GoogleDrive
+tta	6x	73.1	61.8	77.2/61.2/70.5

On MHP-v2

Backbone	LR	Det AP	mIoU	Parsing (APp50/APvol/PCP50)	DOWNLOAD
baseline	3x	68.8	35.6	26.6/40.3/37.9
R-50-FPN	3x	68.1	37.3	40.5/45.2/39.2
R-50-FPN	6x	69.1	38.6	45.3/46.8/43.6	GoogleDrive

'baseline' denotes our implementation Parsing R-CNN.
'+tta' denotes using test-time augmentation, including: soft-nms + bbox voting + h-flipping + multi-scale

ImageNet pretrained weight

Training

To train a model with 8 GPUs run:

python -m torch.distributed.launch --nproc_per_node=8 tools/train_net.py --cfg cfgs/CIHP/e2e_rp_rcnn_R-50-FPN_3x_ms.yaml

Evaluation

multi-gpu evaluation,

python tools/test_net.py --cfg ckpts/CIHP/e2e_rp_rcnn_R-50-FPN_6x_ms/e2e_rp_rcnn_R-50-FPN_6x_ms.yaml --gpu_id 0,1,2,3,4,5,6,7

single-gpu evaluation,

python tools/test_net.py --cfg ckpts/CIHP/e2e_rp_rcnn_R-50-FPN_6x_ms/e2e_rp_rcnn_R-50-FPN_6x_ms.yaml --gpu_id 0

License

RP-R-CNN is released under the MIT license.

rp-r-cnn's People

Contributors

Stargazers

Watchers

Forkers

chaoso berooo cv-ip xiaokai0512 amorostech iwaiter0410 akxakxakx laurabragagnolo chenshiren168

rp-r-cnn's Issues

Error installing RP-R-CNN using google colab

Hello!

I am trying to install RP-R-CNN in google colab, but I am having a problem with the sh make.sh step. When running I am getting the following message:

/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/content/RP-R-CNN/models/ops/csrc -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c /content/RP-R-CNN/models/ops/csrc/cuda/deform_pool_cuda.cu -o build/temp.linux-x86_64-3.6/content/RP-R-CNN/models/ops/csrc/cuda/deform_pool_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -std=c++14
/content/RP-R-CNN/models/ops/csrc/cuda/deform_pool_cuda.cu(42): error: identifier "AT_CHECK" is undefined

/content/RP-R-CNN/models/ops/csrc/cuda/deform_pool_cuda.cu(68): error: identifier "AT_CHECK" is undefined

2 errors detected in the compilation of "/tmp/tmpxft_00000a06_00000000-6_deform_pool_cuda.cpp1.ii".
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1

It appears that there is a problem with deform_pool_cuda.cu.

In another git issue, I found that adding

#ifndef AT_CHECK
#define AT_CHECK TORCH_CHECK 
#endif

at the top of deform_pool_cuda.cu would solve the problem, but it is still happening to me.

Could you check it out? or give me some references or hints, please?

The computer has:

Python 3.6.9

nvcc: NVIDIA (R) Cuda compiler driver
Cuda compilation tools, release 10.1, V10.1.243

Pytorch: 1.7.0+cu101

gcc (Ubuntu 5.5.0-12ubuntu1) 5.5.0 20171010

Has no access to the pretrained model

I attempted to use your pretrained model on CIHP dataset, however, it seems we have no access to this. Please have a check, thanks a lot.

ValueError: num_samples should be a positive integer value, but got num_samples=0

Hello!

I am trying to train RP-R-CNN, but I encounter a problem when runing the training command following the Readme.md. When running

CUDA_VISIBLE_DEVICES=0 python tools/train_net.py --cfg cfgs/CIHP/e2e_rp_rcnn_R-50-FPN_3x_ms.yaml

I am getting the following message:

Traceback (most recent call last):
  File "tools/train_net.py", line 147, in <module>
    main()
  File "tools/train_net.py", line 127, in main
    train_loader = make_train_data_loader(datasets, is_distributed=args.distributed, start_iter=scheduler.iteration)
  File "/data/home/v_julhugoye/human_parsing/RP-R-CNN/rcnn/datasets/dataset.py", line 121, in make_train_data_loader
    sampler = make_data_sampler(datasets, shuffle, is_distributed)
  File "/data/home/v_julhugoye/human_parsing/RP-R-CNN/rcnn/datasets/dataset.py", line 66, in make_data_sampler
    sampler = torch.utils.data.sampler.RandomSampler(dataset)
  File "/data/home/v_julhugoye/.conda/envs/ParsingRCNN/lib/python3.6/site-packages/torch/utils/data/sampler.py", line 94, in __init__
    "value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0
Segmentation fault

So thanks for your help

RuntimeError: Expected object of scalar type Bool but got scalar type Byte for argument #2 'other' in call to _th_and

RP-R-CNN/rcnn/core/test.py

Lines 286 to 287 in dfcc227

 _scores = scores > cfg.FAST_RCNN.SCORE_THRESH 

 inds_all = _scores & fg_labels

`
_scores = scores > cfg.FAST_RCNN.SCORE_THRESH

_scores = _scores.byte()

inds_all = _scores & fg_labels
`
this works

parallel implementation really a bit of redundant

RP-R-CNN/utils/subprocess.py

Line 37 in dfcc227

 cmd = ('python {binary} --range {start} {end} --cfg {cfg_file} --gpu_id {gpu_id}') 

the parallel implementation with subprocess really needs more consideration. a reasonable python interpreter may not be always binding to /usr/bin/python in the code link. it is recommended that impl parallel in
DDP or DP in torch which takes more consideration in detail.

utils/data/datasets/coco.py has_valid_annotation always return false?

RP-R-CNN/utils/data/datasets/coco.py

Lines 21 to 35 in dfcc227

 def has_valid_annotation(anno, ann_types, filter_crowd=True): 

 # if it's empty, there is no annotation 

 if len(anno) == 0: 

 return False 

 if filter_crowd: 

 # if image only has crowd annotation, it should be filtered 

 if 'iscrowd' in anno[0]: 

 anno = [obj for obj in anno if obj["iscrowd"] == 0] 

 if len(anno) == 0: 

 return False 

 # if all boxes have close to zero area, there is no annotation 

 if _has_only_empty_bbox(anno): 

 return False 

 return False

dense pose estimation

thank your for your great work!
I have benefited much from Parsing-rcnn. I have been looking forward to RP-R-CNN for a long time, this paper still give me much surprise. But my question is "why does RP-R-CNN not contain dense pose estimation result like parsing-rcnn".

RuntimeError: Error(s) in loading state_dict for Generalized_RCNN:

Hi,
Thanks for your work,
I've tried to run minimal demo code I wrote but i got this error :

Traceback (most recent call last):
  File "demo.py", line 33, in <module>
    main()
  File "demo.py", line 17, in main
    load_weights(model, w)
  File "D:\Artificial Intelligence\HumanDetection_Tracking\_Human Parsing\RP-R-CNN-master\utils\checkpointer.py", line 29, in load_weights
    model.load_state_dict(model_state_dict)
  File "C:\Users\127051\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 1052, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generalized_RCNN:
        size mismatch for RPN.anchor_generator.cell_anchors.0: copying a param with shape torch.Size([3, 4]) from checkpoint, the shape in current model is torch.Size([15, 4]).
        size mismatch for RPN.head.conv.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([2048, 2048, 3, 3]).
        size mismatch for RPN.head.conv.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([2048]).
        size mismatch for RPN.head.cls_logits.weight: copying a param with shape torch.Size([3, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([15, 2048, 1, 1]).
        size mismatch for RPN.head.cls_logits.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([15]).
        size mismatch for RPN.head.bbox_pred.weight: copying a param with shape torch.Size([12, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([60, 2048, 1, 1]).
        size mismatch for RPN.head.bbox_pred.bias: copying a param with shape torch.Size([12]) from checkpoint, the shape in current model is torch.Size([60]).

I downloaded R-50-FPN on CIHP
My minimal demo code :

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'
import torch
from torchvision import transforms
from rcnn.modeling.model_builder import Generalized_RCNN
from utils.checkpointer import get_weights, load_weights
from rcnn.core.config import cfg
from utils.net import convert_bn2affine_model
import cv2
import glob
import rcnn.core.test as rcnn_test

def main():
	model = Generalized_RCNN(is_train=False)
	w = get_weights('./cfgs/CIHP/e2e_rp_rcnn_R-50-FPN_6x_ms.yaml', './weights/model_latest.pth')
	load_weights(model, w)
	if cfg.MODEL.BATCH_NORM == 'freeze':
		model = convert_bn2affine_model(model)
	model.eval()
	model.to(torch.device(cfg.DEVICE))
	paths = sorted(glob.glob('./Inputs' + '/*'))
	for path in img_path_l:
		image = cv2.imread(path)
		if ~(image is None):
			img = transforms.ToTensor()(image)
			with torch.no_grad():
				result, features = rcnn_test.im_detect_bbox(model, img)
				result = rcnn_test.im_detect_mask(model, result, features)
				print(result)

if __name__ == '__main__':
    main()

from pet.utils.data.structures.parsing import FLIP_MAP

hi~~thanks for your great job!!!
there may no pet module....

RP-R-CNN/utils/data/structures/semantic_segmentation.py

Line 57 in dfcc227