Git Product home page Git Product logo

disentangled-person-image-generation's Introduction

Disentangled-Person-Image-Generation

Tensorflow implementation of CVPR 2018 paper Disentangled Person Image Generation

Two stages framework

Stage-I: reconstruction

Sampling phase

Dependencies

  • python 2.7
  • tensorflow-gpu (1.4.1)
  • numpy (1.14.0)
  • Pillow (5.0.0)
  • scikit-image (0.13.0)
  • scipy (1.0.1)
  • matplotlib (2.0.0)

Resources

TF-record data preparation steps

You can skip this data preparation procedure if directly using the tf-record data files.

  1. cd datasets
  2. ./run_convert_market.sh to download and convert the original images, poses, attributes, segmentations
  3. ./run_convert_DF.sh to download and convert the original images, poses
  4. [Optional] ./run_convert_RCV.sh to convert the original images and pose coordinates, i.e. (row, column, visibility) (e.g. from OpenPose or MaskRCNN), which can be useful for other datasets. Note: we also provide the convert code for Market-1501 Attribute and Market-1501 Segmentation results from PSPNet. These extra info. are provided for further research. In our experiments, pose mask are ontained from pose key-points (see _getPoseMask function in convert .py files).

Training steps

  1. Download the tf-record training data.
  2. Modify the log_dir and log_dir_pretrain in the run_market_train.sh/run_DF_train.sh scripts.
  3. run run_market_train.sh/run_DF_train.sh

Note: we use a triplet instead of pair real/fake for adversarial training to keep training more stable.

Testing steps

  1. Download the pretrained models and tf-record testing data.
  2. Modify the log_dir and log_dir_pretrain in the run_market_test.sh/run_DF_test.sh scripts.
  3. run run_market_test.sh/run_DF_test.sh

Fg/Bg/Pose sampling on Market-1501

Appearance sampling on DeepFashion dataset

Pose sampling on DeepFashion dataset

Pose interpolation between real images

  • Between same person:
  • Between different persons:

Pose guided person image generation

Citation

@inproceedings{ma2018disentangled,
  title={Disentangled Person Image Generation},
  author={Ma, Liqian and Sun, Qianru and Georgoulis, Stamatios and Van Gool, Luc and Schiele, Bernt and Fritz, Mario},
  booktitle={{IEEE} Conference on Computer Vision and Pattern Recognition},
  year={2018}
}

Related projects

disentangled-person-image-generation's People

Contributors

charliememory avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

disentangled-person-image-generation's Issues

OutOfRangeError (see above for traceback): FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 16, current size 0)

Hi,

I get the following error when attempting to train the Fg Bg reconstruction model, using tensorflow-gpu1.9, python2.7. Thank you for help.

gpu=0
D_arch='DCGAN'
log_dir='./logs'

####################### Stage-I: reconstruction #####################
## Fg Bg reconstruction
model_dir=${log_dir}'/MODEL1_Encoder_GAN_BodyROI7'
python main.py --dataset=Market_train_data \
             --use_gpu=True --img_H=128  --img_W=64 \
             --batch_size=16 --max_step=120000 \
             --d_lr=0.00002  --g_lr=0.00002 \
             --lr_update_step=50000 \
             --model=1 \
             --D_arch=${D_arch} \
             --gpu=${gpu} \
             --z_num=64 \
             --model_dir=${model_dir} \

error message:
/home/ryu/sourcecode/Disentangled-Person-Image-Generation/tflib/plot.py:4: UserWarning:
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called before pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

The backend was originally set to 'TkAgg' by the following code:
File "main.py", line 4, in
from trainer import *
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py", line 20, in
import models
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/models.py", line 6, in
from utils import *
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/utils.py", line 492, in
import matplotlib.pyplot as plt
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/matplotlib/pyplot.py", line 71, in
from matplotlib.backends import pylab_setup
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/matplotlib/backends/init.py", line 16, in
line for line in traceback.format_stack()

matplotlib.use('Agg')
load pn_pairs_num ......
######coord2channel_simple#####
######coord2channel_simple#####
Uppercase local vars:
BATCH_SIZE: 16
DATA_DIR:
DIM: 64
G_OUTPUT_DIM: 24576
IMG_H: 128
IMG_W: 64
ITERS: 200000
LAMBDA: 10
MODE: dcgan
WARNING:tensorflow:From /home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py:199: init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2018-07-13 10:59:03,319:WARNING::From /home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py:199: init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2018-07-13 10:59:06.651736: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.6325
pciBusID: 0000:02:00.0
totalMemory: 10.92GiB freeMemory: 10.76GiB
2018-07-13 10:59:06.651792: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-07-13 10:59:07.038518: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-13 10:59:07.038577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958] 0
2018-07-13 10:59:07.038589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: N
2018-07-13 10:59:07.038950: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10411 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1)
[] MODEL dir: ./logs/MODEL1_Encoder_GAN_BodyROI7
[
] PARAM path: ./logs/MODEL1_Encoder_GAN_BodyROI7/params.json
2018-07-13 10:59:21.620532: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: part_bbox_1. Can't parse serialized Example.
2018-07-13 10:59:21.620573: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: part_bbox_1. Can't parse serialized Example.
2018-07-13 10:59:21.621055: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: part_bbox_1. Can't parse serialized Example.
2018-07-13 10:59:21.621579: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at example_parsing_ops.cc:240 : Invalid argument: Key: part_bbox_1. Can't parse serialized Example.
Traceback (most recent call last):
File "main.py", line 90, in
main(config)
File "main.py", line 82, in main
trainer.train()
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py", line 328, in train
part_bbox_target_fixed, part_vis_fixed, part_vis_target_fixed = self.get_image_from_loader()
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py", line 530, in get_image_from_loader
self.mask_r6, self.mask_r6_target, self.part_bbox, self.part_bbox_target, self.part_vis, self.part_vis_target])
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 16, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]

Caused by op u'batch', defined at:
File "main.py", line 90, in
main(config)
File "main.py", line 23, in main
trainer = DPIG_Encoder_GAN_BodyROI_FgBg(config)
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py", line 42, in init
self.part_bbox, self.part_bbox_target, self.part_vis, self.part_vis_target = self._load_batch_pair_pose(self.dataset_obj)
File "/home/ryu/sourcecode/Disentangled-Person-Image-Generation/trainer.py", line 555, in _load_batch_pair_pose
batch_size=self.batch_size, num_threads=self.num_threads, capacity=self.capacityCoff * self.batch_size)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 988, in batch
name=name)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 762, in _batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 483, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3480, in queue_dequeue_many_v2
component_types=component_types, timeout_ms=timeout_ms, name=name)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3414, in create_op
op_def=op_def)
File "/home/ryu/anaconda3/envs/pose/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1740, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

OutOfRangeError (see above for traceback): FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 16, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_INT64, DT_INT64, DT_INT64, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]

test set

Hi. Impressive work product.
I want to test on my own deep fashion data. But when I run 'run_convert_DF.sh' ,it says I don't have the file named 'all_peaks_dic_DeepFashion.p'.Can you tell me how can I make this file? Thanks!

Which is the pretrain model in train.sh?

Hello, I'm a little bit confused that which the pretained model directory is in the train script. Should I use the pretrained model you provided? Or just use the training model itself?

Limit to GPU memory

Hi,

I run your training code for DeepFashion.
And OOM error happens when i set batch size as 6 for MODEL101_DF_Encoder_GAN_BodyROI7.
Here i only can set batch size as 1, and i will take almost 8.5G memory per GPU.
So how to use 6 batch size for training?

Thanks in advance

How to convert a 352-dim vector to 128*64*(352 channels) by "tile"?

Hi,thanks for your great paper first!

Because the code has not been released yet, I'm confused about this fragment from paper:

the foreground and background features are concatenated and tiled into 128×64×352 appearance feature maps.

Can you tell me how this is achieved? Just copy-paste each value of the 352-dim vector 128*64 times?

error test

hello,
when I run the test.py I have this error : (in tflib/inception_score.py)
ValueError: Shape must be rank 2 but is rank 1 for 'MatMul' (op: 'MatMul') with input shapes: [2048], [2048,1008].
can you help me please?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.