emrahbasaran / spreid Goto Github PK

Code for our CVPR 2018 paper - Human Semantic Parsing for Person Re-identification

License: MIT License

Python 100.00%

spreid's Introduction

Human Semantic Parsing for Person Re-identification

Code for our CVPR 2018 paper - Human Semantic Parsing for Person Re-identification

We have used Chainer framework for the implementation. SPReID^w/fg and SPReID^w/fg-ft results mentioned in Table 5 (with weight sharing setting) in the paper can be reproduced using this code.

Please use the links below to download the semantic parsing model (LIP_iter_30000.chainermodel) and the inceptionv3 weights pre-trained on imagenet (data/dump/):

Directories & Files

/
├── checkpoints/  # checkpoint models are saved into this directory
│
├── data/dump/  # inceptionv3 weights pre-trained on imagenet. download using this [link] (https://www.dropbox.com/sh/x0ey09q1nq7ci39/AACRuJa_f8N0_gIFcEWZUZ7ja?dl=0)
│
├── evaluation_features/ # extracted features are saved into this directory
│
├── evaluation_list/ # there are two image lists to extract features for each evaluation datasets, one for gallery and one for query
│   ├── cuhk03_gallery.txt
│   ├── cuhk03_query.txt
│   ├── duke_gallery.txt
│   ├── duke_query.txt
│   ├── market_gallery.txt
│   └── market_query.txt
│
├── train_list/ # image lists to train the models
│   ├── train_10d.txt # training images collected from 10 datasets
│   ├── train_cuhk03.txt # training images from cuhk03
│   ├── train_duke.txt # training images from duke
│   └── train_market.txt # training images from market
│
├── LIP_iter_30000.chainermodel # download this model using this [link](https://www.dropbox.com/s/nw5h0lw6xrzp5ks/LIP_iter_30000.chainermodel?dl=0)
├── datachef.py
├── main.py
└── modelx.py

Train

cd $SPREID_ROOT
# train SPReID on 10 datasets
python main.py --train_set "train_10d" --label_dim "16803" --scales_reid "512,170" --optimizer "lr:0.01--lr_pretrained:0.01" --dataset_folder "/path/to/the/dataset"
# fine-tune SPReID on evaluation datasets (Market-1501, DukeMTMC-reID, CUHK03) with high-resolution images
python main.py --train_set "train_market" --label_dim_ft "751" --scales_reid "778,255" --optimizer "lr:0.01--lr_pretrained:0.001" --max_iter "50000" --dataset_folder "/path/to/the/dataset" --model_path_for_ft "/path/to/the/model"
python main.py --train_set "train_duke" --label_dim_ft "702" --scales_reid "778,255" --optimizer "lr:0.01--lr_pretrained:0.001" --max_iter "50000" --dataset_folder "/path/to/the/dataset" --model_path_for_ft "/path/to/the/model"
python main.py --train_set "train_cuhk03" --label_dim_ft "1367" --scales_reid "778,255" --optimizer "lr:0.01--lr_pretrained:0.001" --max_iter "50000" --dataset_folder "/path/to/the/dataset" --model_path_for_ft "/path/to/the/model"

Feature Extraction

cd $SPREID_ROOT
# Extract features using the model trained on 10 datasets. You should run this command two times for each dataset using --eval_split "DATASET_gallery" and --eval_split "DATASET_query"
python main.py --extract_features 1 --train_set "train_10d" --eval_split "market_gallery" --scales_reid "512,170" --checkpoint 200000 --dataset_folder "/path/to/the/dataset"
# Extract features using the models trained on evaluation datasets.
python main.py --extract_features 1 --train_set "train_market" --eval_split "market_gallery" --scales_reid "778,255" --checkpoint 50000 --dataset_folder "/path/to/the/dataset"
python main.py --extract_features 1 --train_set "train_duke" --eval_split "duke_gallery" --scales_reid "778,255" --checkpoint 50000 --dataset_folder "/path/to/the/dataset"
python main.py --extract_features 1 --train_set "train_cuhk03" --eval_split "cuhk03_gallery" --scales_reid "778,255" --checkpoint 50000 --dataset_folder "/path/to/the/dataset"

Results

	Market-1501		CUHK03		DukeMTMC-reID
Model	mAP(%)	rank-1	mAP(%)	rank-1	mAP(%)	rank-1
SPReID^w/fg	77.62	90.88	-	87.69	65.66	81.73
SPReID^w/fg-ft	80.54	92.34	-	89.68	69.29	83.80

Citation

@InProceedings{Kalayeh_2018_CVPR,
author = {Kalayeh, Mahdi M. and Basaran, Emrah and Gökmen, Muhittin and Kamasak, Mustafa E. and Shah, Mubarak},
title = {Human Semantic Parsing for Person Re-Identification},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}

spreid's People

Contributors

Stargazers

Watchers

spreid's Issues

environment set

when i run this code,there are some errors.could someone tell me the environment set,such as the version of python,cupy,chainer,cuda,cudnn and so on .thanks

Testing my own images

Hi,
I want to test the pre-trained model available on my own sample images. Can you please explain the method to do this?

Table4

Hi, I'm sorry to bother you again. In your table-4, Inception-V3 is ony for global average polling, SPReID^wo/fg is for global average polling and max(Head,Upper-body,Lower-Body,Shoes),and SPReID^w/fgis for global average polling, max(Head,Upper-body,Lower-Body,Shoes) and Foreground. Is my understand is right? Or which is wrong? How to understand SPReID^combined?Which features it use?
Thank you .

freature campare

I have some questions about the SPReid.
First, I didn't find distance measurement in the article, did you use Euclidean distance to compare the two freature？
Second ,I use the github's code and generate the 6144-dim freature,but the article seem use the 2048-dim freature, I have confused whether I have a mistake understanding and want to confirm about it.
Thank you for your time and help, I will really appreciate it!

'map' object is not subscriptable

/home/zbp/Linux/tjj/SPReID/main.py(145)Train()
144 ipdb.set_trace()
--> 145 tmp1 = batch_tuple_indx[iterk]
146 tmp2 = batch_tuples[dataset][batch_tuple_indx[iterk]]

ipdb> c
Exception in thread prefetch_loop:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/zbp/anaconda3/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/zbp/anaconda3/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/zbp/anaconda3/lib/python3.6/site-packages/chainer/iterators/multiprocess_iterator.py", line 428, in _fetch_run
data = _fetch_dataset[index]
File "/home/zbp/anaconda3/lib/python3.6/site-packages/chainer/dataset/dataset_mixin.py", line 67, in getitem
return self.get_example(index)
File "/home/zbp/Linux/tjj/SPReID/datachef.py", line 32, in get_example
fy = float(self.image_size[0]) / float(input_image.shape[0]) # Height
TypeError: 'map' object is not subscriptable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/zbp/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/zbp/anaconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/zbp/anaconda3/lib/python3.6/site-packages/chainer/iterators/multiprocess_iterator.py", line 336, in _run
alive = self._task()
File "/home/zbp/anaconda3/lib/python3.6/site-packages/chainer/iterators/multiprocess_iterator.py", line 355, in _task
data_all = future.get(_response_time)
File "/home/zbp/anaconda3/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: 'map' object is not subscriptable

Even I use x = self.pool.apply((x,))[0],it doesn't solve the problem.
Thank you.

Ask for trained model

Thanks for release the code and trained parsing model. I want to I want to use your model as a comparison on a new dataset, So, I could appreciate it very much if you could share me the trained reid model on ten dataset and the model finetuned on market.

weight shareing

Hello, could you please share the code of weight sharing，I'm not familiar with chainer

Add License

Would love to see a license added so those of us that want to use this know about usage restrictions.

dataset

Why I have not the same dataset like cuhk01 cuhk02 cuhk03, it is not like what you write in your train.txt, I have cuhk cuhk-03.mat, not like your k03/cam_0/00078_00007.bmp.So could you give a advice where I am wrong?

is there pytorch version,can you share it

The pretrained for human sematic parsing

Thank you for your code!
I have some question about how to pretrain the human semantic parsing model.
I want to know some details how to calculate the loss when pretrain the model by LIP dataset

Weight Sharing Setting when training the model

Dear author,

I would like to ask you about the "weight sharing method' you mentioned in the paper. I couldn't understand what is called weight sharing using the Chainer framework. The whole Inception V3 Network architecture will be changed completely every iteration? Correct me I am wrong.

Thank you in advance.

Trying to test semantic parsing model

Hello,
I tried to test semantic parsing model but got a error as below:

File "/home/kevinh/git/SPReID/modelx.py", line 57, in call
x = self.pool(x)

RuntimeError: Chainer's built-in function class object (AveragePooling2D) which is derived from chainer.FunctionNode has been called as if it were a callable. Use FunctionNode.apply() method instead.
Furthermore, it's not recommended to use built-in function classes directly; use corresponding function aliases (those with snake_case name, such as F.convolution_nd) instead.

I run the code with chainer v4.2, maybe it caused the error?

In addition, whether the performance of human semantic parsing (show in Table 1 of SPReID paper) is measured with 20-class-segmentation settings, or 6-class settings?

Thanks,
Kevin

how to train the model,the details about training process . Reid model and segment model trained sparately?

how do you get the LIP_iter_30000.chainermodel?

do you use the hand-craft dataset in one of the dataset like market1501?

reproduce results for cuhk03

Hi,
I read your human semantic parsing paper for person re-id and trying to reproduce results for cuhk03. For time being I am evaluating results just for re-id without semantic parsing. I have some queries to confirm or ask from you:

have you used cuhk03 detected and labelled combined together? as I saw your txt files and it has images > 13164 or your have used any kind of offline augmentation?
have you evaluated results for 20 splits and then took the average?
for each test split, have you performed single shot single query evaluation taking only one sample in query and gallery?
have you changed anything or added any layer in resnet 50 and resnet 152?
Have you checked for any other distance metric like XQDA? As many of the papers have told that XQDA always gives better than euclidean.

looking forward to your response. thanks in advance,

train: path of model

nohup python main.py --train_set "train_market" --label_dim_ft "751" --scales_reid "778,255" --optimizer "lr:0.01--lr_pretrained:0.001" --max_iter "50000" --dataset_folder "/home/reid/data/market1501/" --model_path_for_ft "/home/reid/SPReID/LIP_iter_30000.chainermodel" >train.out 2>&1 &

I use the code above to run the training, but I get the error below.

File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object 'segmentation' doesn't exist)"

决定不引用这类论文，不是想黑：想不明白，公开得差不多了，为啥训练好的模型和评估函数都不放进去，方便比较？

map

hellow
I have trained market datasets with the market and duke pretrained model. And test with python, but get a bad result mAP 0.16. I think although the pretrained is important, the result is too bad. did some important points need to attention?
By the way , I use the imagename's persion id like '-1_c1s1_001376_05.jpg', using '-1'. and misunderstand that the code's 'market_gallery.txt' , it contain like 'market/bounding_box_test/-1_c1s1_000401_03.jpg 0',but i could not understand the meaning of '0', it seems all the .jpg follows '0'.

I'm new of this chainer,Is problem my vision of python or any other problem ?I doon't know what should i do ?could you help me ?

Load segmentation weights
Push Model to GPU
Main Begins
Exception in thread prefetch_loop:
Traceback (most recent call last):
File "/home/lvv/anaconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/lvv/anaconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/lvv/anaconda2/lib/python2.7/site-packages/chainer/iterators/multiprocess_iterator.py", line 336, in _run
alive = self._task()
File "/home/lvv/anaconda2/lib/python2.7/site-packages/chainer/iterators/multiprocess_iterator.py", line 355, in _task
data_all = future.get(_response_time)
File "/home/lvv/anaconda2/lib/python2.7/multiprocessing/pool.py", line 572, in get
raise self._value
AttributeError: 'NoneType' object has no attribute 'shape'

LIP30000.model

Could I have a look at your training code about LIP_iter_30000?

Missing conv.h5

Hi,
I read through the paper and tried executing the code.
Seems something missing.
it asks for conv.h5, but it's not present.
SPReID/data/dump/conv.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

thanks,
Kind Regards
Arun

I am not getting the dataset of this program, can you please send me the dataset particular to this project

Hi All,

I was trying to run this program in windows system, I dont have the dataset images for this program can you please help me send me the dataset images with label for this program.
So that I can execute this program and learn.
I am new to this python env. as well as new to GIT hub

about the accuracy with pure segmentation branch and no further finetuning

We've trained the model with provided code without segmentation branch and have acquired similar results to your paper. It's mentioned in the paper that using the whole network with segmentation branch and the parameters trained without segmentation branch, an increase of mAP could be seen without further finetuning. However, we didn't observe such phenomenon in our experiments conducted, with models trained on different image sizes. We could not easily find the reason. Is there any key points when doing this? Could you kindly shared something that we may have ignored? Sincerely thank you.

Trying to reproduce human Semantic Parsing in pytorch

Hello, I am a Postgraduate in Beijing University in China. I am trying to reproduce human Semantic parsing in pytorch, I have seen your code in your release code. In my code, I have reproduce the netword in human semantic parsing , which is same as yours human semantic parsing in pytorch. and I use interplot 3030-----> 512512 and cross_entropy to measure loss between results and ground label，when I train in LIP, its results is so bad. I have seen the LIP paper to learn the ASPP. And I have tried to load your released model parameter in LIP_iter_30000.chainermodel to pytorch. But it doesn't work, the results is so bad, I can't see any segmentation in picture. So I think maybe there is something wrong about my code. So if you are free, please tell how to train the semantic parsing.
My email: [email protected] .

in semantic part,how do you decide the segprob?

segprob = F.stack((1.0-y[0],
y[1]+y[2]+y[4]+y[13],
y[5]+y[6]+y[7]+y[11]+y[10]+y[3]+y[14]+y[15],
y[9]+y[16]+y[17]+y[12],
y[18]+y[19]+y[8]), axis=1)
please tell me,why do you know (1.0-y[0]) is oreground,the seconde is head, third is torso-hand, forth is lower-body, last is shoes?
thanks

datasets

this is my directory of datasets :

It's right????

can you share your evaluate code?I got very low result

I used this codehttps://github.com/layumi/Person_reID_baseline_pytorch/blob/master/evaluate_gpu.py
I changed the evaluate part in main.py to

def get_id(img_path,camera_id,labels):
    a, b = img_path.split('\n')
    image_path, label = a.split(' ')
    name = image_path.split('/')[-1]
    cam = name.split('_')[1][1]
    if label[0:2]=='-1':
        labels.append(-1)
    else:
        labels.append(int(label))
    camera_id.append(int(cam))
    return camera_id, labels

def Evaluation():
    # Creat data generator
    batch_tuple = MultiprocessIterator(
        DataChef.ReID10D(args, args.project_folder + '/evaluation_list/' + args.eval_split + '.txt',
                         image_size=args.scales_tr[0]),
        args.minibatch, n_prefetch=2, n_processes=args.nb_processes, shared_mem=20000000, repeat=False, shuffle=False)
    # Keep the log in history
    history = {args.dataset: {'features': []}}
    for dataBatch in batch_tuple:
        dataBatch = list(zip(*dataBatch))
        # Prepare batch data
        IMG = np.array_split(np.array(dataBatch[0]), len(Model), axis=0)
        LBL = np.array_split(np.array(dataBatch[1]), len(Model), axis=0)
        # Forward
        for device_id, img, lbl in zip(range(len(Model)), IMG, LBL):
            Model[device_id](img, lbl, args.dataset,train=False)
        # Aggregate reporters from all GPUs
        reporters = []
        for i in range(len(Model)):
            reporters.append(Model[i].reporter)
            Model[i].reporter = {}  # clear reporter
        # History
        for reporter in reporters:
            for k in reporter[args.dataset].keys():
                history[args.dataset][k].append(reporter[args.dataset][k])
    features = np.concatenate(history[args.dataset]['features'], axis=0)
    F = open(args.project_folder + '/evaluation_list/' + args.eval_split + '.txt').readlines()
    camera_id = []
    labels = []
    for f in F:
        cam, label = get_id(f,camera_id,labels)
    # storing features to an outputfile
    if args.eval_split=='market_gallery':
        result = {'gallery_f': features, 'gallery_label': label,'gallery_cam': cam}
        scipy.io.savemat('pytorch_result_gallery.mat', result)
    else:
        result ={'query_f': features, 'query_label': label,'query_cam': cam}
        scipy.io.savemat('pytorch_result_query.mat', result)

I only have one gpu-gtx1080,and i just can get five of the train-10d dataset .In training part ,I just set the max_iter to 65000 so the loss is 0.03.
I got the result blow
Rank@1:0.490796 Rank@5:0.763955 Rank@10:0.846793 mAP:0.329347
How can i get higher accuracy？Please give me some advices.

cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins

First of all thanks your amazing work!
when I run the code I have met the following issue：
Traceback (most recent call last): File "main.py", line 300, in <module> Train() File "main.py", line 147, in Train Model[device_id](img, lbl, dataset, train=True) File "/home/wangliu/Hr/SPReID/modelx.py", line 387, in __call__ xo = self.segmentation.predictor(x) File "/home/wangliu/Hr/SPReID/modelx.py", line 324, in __call__ x = convolution(x) File "/home/wangliu/Hr/SPReID/modelx.py", line 315, in convolution x = self.convolution[0](x) File "/home/wangliu/Hr/SPReID/modelx.py", line 60, in __call__ x = self.conv(x) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/link.py", line 242, in __call__ out = forward(*args, **kwargs) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/links/connection/convolution_2d.py", line 173, in forward groups=self.groups) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/functions/connection/convolution_2d.py", line 572, in convolution_2d y, = fnode.apply(args) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/function_node.py", line 263, in apply outputs = self.forward(in_data) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/function_node.py", line 369, in forward return self.forward_gpu(inputs) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/functions/connection/convolution_2d.py", line 173, in forward_gpu return self._forward_gpu_core(x, W, b) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/functions/connection/convolution_2d.py", line 180, in _forward_gpu_core cover_all=self.cover_all, dy=self.dy, dx=self.dx) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/chainer/utils/conv.py", line 123, in im2col_gpu h, w, out_h, out_w, kh, kw, sy, sx, ph, pw, dy, dx, col) File "cupy/core/_kernel.pyx", line 558, in cupy.core._kernel.ElementwiseKernel.__call__ File "cupy/core/_kernel.pyx", line 579, in cupy.core._kernel.ElementwiseKernel._get_elementwise_kernel File "cupy/core/_kernel.pyx", line 392, in cupy.core._kernel._get_elementwise_kernel File "cupy/core/_kernel.pyx", line 26, in cupy.core._kernel._get_simple_elementwise_kernel File "cupy/core/_kernel.pyx", line 46, in cupy.core._kernel._get_simple_elementwise_kernel File "cupy/core/carray.pxi", line 148, in cupy.core.core.compile_with_cache File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/cupy/cuda/compiler.py", line 135, in compile_with_cache base = _preprocess('', options, arch) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/cupy/cuda/compiler.py", line 98, in _preprocess result = prog.compile(options) File "/home/wangliu/anaconda3/envs/aligned_reid/lib/python2.7/site-packages/cupy/cuda/compiler.py", line 245, in compile raise CompileException(log, self.src, self.name, options) cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins
Do you know how to solve that？

> > Hi, I use only 1 GPU(GXT 1080Ti 11GB) for training this model. Limited by memory of GPU, I set the batchsize=4 instead of 16 in your code. And I trained the model only on Market-1501 dataset instead of 10 datasets. The result is mAP = 0.385818, r1 precision = 0.627969 much lower than your paper said. So there are two problems.

Hi, I use only 1 GPU(GXT 1080Ti 11GB) for training this model. Limited by memory of GPU, I set the batchsize=4 instead of 16 in your code. And I trained the model only on Market-1501 dataset instead of 10 datasets. The result is mAP = 0.385818, r1 precision = 0.627969 much lower than your paper said. So there are two problems.

How many GPUs you used in this experiment? I wonder whether the batchsize mostly influence the accuracy.

If I can't get the whole 10 datasets and only use 1 dataset for training and testing, how much will the result fall?
Thank you for your wonderful work. Looking forward to your response.

Hi, the setting of mine is the same as yours, but the results are so bad--R1=0.355 ,mAP= 0.2111. I think your result is reasonable and could you tell me what evaluation code you use?

I use the matlab code offered by market-1501 dataset website. And finally the highest result I got was mAP=0.686, r1=85.6. I have no idea about how to get close to the authors' result.

Hello, I use the matlab code offered by market-1501 dataset website too, but it report error: Incorrect use of *, internal matrix dimensions must be consistent , Did you make any changes to the evaluation code? Thanks very much for reply!

Much Lower Result & GPU Problem

Hi, I use only 1 GPU(GXT 1080Ti 11GB) for training this model. Limited by memory of GPU, I set the batchsize=4 instead of 16 in your code. And I trained the model only on Market-1501 dataset instead of 10 datasets. The result is mAP = 0.385818, r1 precision = 0.627969 much lower than your paper said. So there are two problems.

How many GPUs you used in this experiment? I wonder whether the batchsize mostly influence the accuracy.
If I can't get the whole 10 datasets and only use 1 dataset for training and testing, how much will the result fall?
Thank you for your wonderful work. Looking forward to your response.

Dataset path

The parts of human semantic parsing

Hello, please ask about the four parts of human semantic parsing: head, upper body, lower body and shoes.How did I get that?Is it a one-time acquisition based on the foreground?