This repository is for Panini-Net introduced in the following paper
Yinhuai Wang, Yujie Hu, Jian Zhang, "Panini-Net: GAN Prior based Degradation-Aware Feature Interpolation for Face Restoration", AAAI 2022, [arxiv]
Colab Demo for Panini-Net Face Restoration & Super Resolution! 
Emerging high-quality face restoration (FR) methods often utilize pre-trained GAN models (i.e., StyleGAN2) as GAN Prior. However, these methods usually struggle to balance realness and fidelity when facing various degradation levels. Besides, there is still a noticeable visual quality gap compared with pre-trained GAN models. In this paper, we propose a novel GAN Prior based degradation-aware feature interpolation network, dubbed Panini-Net, for FR tasks by explicitly learning the abstract representations to distinguish various degradations. Specifically, an unsupervised degradation representation learning (UDRL) strategy is first developed to extract degradation representations (DR) of the input degraded images. Then, a degradation-aware feature interpolation (DAFI) module is proposed to dynamically fuse the two types of informative features (i.e., features from input images and features from GAN Prior) with flexible adaption to various degradations based on DR. Ablation studies reveal the working mechanism of DAFI and its potential for editable FR. Extensive experiments demonstrate that our Panini-Net achieves state-of-the-art performance for multi-degradation face restoration and face super-resolution.
Figure: Overview of Panini-Net
Clone this repository:
git clone https://github.com/jianzhangcs/panini.git
cd paniniThis code is based on MMEditing. Please refer to install.md to install dependencies.
This code uses FFHQ for training and CelebA-HQ for testing.
After downloading and degenerating the datasets, we put the datasets into data/
Download our pre-trained model (moco, panini_mfr, panini_16xsr) and put them into checkpoint/
You can use scripts in file test_script.sh to test.
Note: This work does not focus on real world degradation, but studies how to adapt to multiple degradation, which is currently artificially designed, and the parameters are not fine-tuned specifically for real world degradation, so our generalization performance on real world degradation is poor. If you want to reproduce the results of the paper, you can use the test examples we provide or use our degrade function to generate the test images.
python demo/restoration_single_face_demo.py --config configs/panini_mfr.py --checkpoint checkpoint/panini_mfr_latest.pth --img_path examples/MFR/00001.png --save_path examples/MFR_result/00001.png
python demo/restoration_single_face_demo.py --config configs/panini_sr.py --checkpoint checkpoint/panini_sr_latest.pth --img_path examples/SR/00001.png --save_path examples/SR_result/00001.pngpython demo/restoration_dir_face_demo.py --config configs/panini_mfr.py --checkpoint checkpoint/panini_mfr_latest.pth --img_path examples/MFR --save_path examples/MFR_result
python demo/restoration_dir_face_demo.py --config configs/panini_sr.py --checkpoint my_project/panini_sr/latest.pth --img_path examples/SR --save_path examples/SR_resultYou can use scripts in file 'moco/train.sh' to train.
python main_moco.py --lr 0.03 --batch-size 32 --dist-url 'tcp://localhost:10001' --multiprocessing-distributed --world-size 1 --rank 0 --epochs 200You can use scripts in file train_script.sh to train/test.
python tools/train.py configs/panini_mfr.py --work-dir my_project/panini_mfr
python tools/train.py configs/panini_sr.py --work-dir my_project/panini_srpython tools/train.py configs/panini_mfr.py --work-dir my_project/panini_mfr --resume-from my_project/panini_mfr/latest.pth
python tools/train.py configs/panini_sr.py --work-dir my_project/panini_sr --resume-from my_project/panini_sr/latest.pthpython -m torch.distributed.launch --nproc_per_node=2 tools/train.py configs/panini_mfr.py --work-dir my_project/panini_mfr --launcher pytorch
python -m torch.distributed.launch --nproc_per_node=2 tools/train.py configs/panini_sr.py --work-dir my_project/panini_sr --launcher pytorchIf our work is useful for your research, please consider citing:
@inproceedings{wang2022panini,
title={Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration},
author={Wang, Yinhuai and Hu, Yujie and Zhang, Jian},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
year={2022}
}
This code borrows heavily from MMEditing.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent