Image Inpainting via Conditional Texture and Structure Dual Generation

Xiefan Guo, Hongyu Yang, Di Huang
In ICCV'2021

Generator. Image inpainting is cast into two subtasks, i.e., structure-constrained texture synthesis (left, blue) and texture-guided structure reconstruction (right, red), and the two parallel-coupled streams borrow encoded deep features from each other. The Bi-GFF module and CFA module are stacked at the end of the generator to further refine the results.

Discriminator. The texture branch estimates the generated texture, while the structure branch guides structure reconstruction.

• Python >= 3.6
• PyTorch >= 1.0
• NVIDIA GPU + CUDA cuDNN

• Clone this repo:

git clone https://github.com/Xiefan-Guo/CTSDG.git
cd CTSDG

• Install PyTorch and dependencies from http://pytorch.org
• Install python requirements:

pip install -r requirements.txt

Image Dataset. We evaluate the proposed method on the CelebA, Paris StreetView, and Places2 datasets, which are widely adopted in the literature.

Mask Dataset. Irregular masks are obtained from Irregular Masks and classified based on their hole sizes relative to the entire image with an increment of 10%.

Analogous to PConv by Liu et.al, initial training followed by finetuning are performed.

python train.py \
  --image_root [path to image directory] \
  --mask_root [path mask directory]

python train.py \
  --image_root [path to image directory] \
  --mask_root [path to mask directory] \
  --pre_trained [path to checkpoints] \
  --finetune True

Distributed training support. You can train model in distributed settings.

python -m torch.distributed.launch --nproc_per_node=N_GPU train.py

To test the model, you run the following code.

python test.py \
  --pre_trained [path to checkpoints] \
  --image_root [path to image directory] \
  --mask_root [path to mask directory] \
  --result_root [path to output directory] \
  --number_eval [number of images to test]

If any part of our paper and repository is helpful to your work, please generously cite with:

@InProceedings{Guo_2021_ICCV,
    author    = {Guo, Xiefan and Yang, Hongyu and Huang, Di},
    title     = {Image Inpainting via Conditional Texture and Structure Dual Generation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {14134-14143}
}