Figure: Image editing results using LowRankGAN on StyleGAN2 (first three columns) and BigGAN (last column).

Low-Rank Subspaces in GANs
Jiapeng Zhu, Ruili Feng, Yujun Shen, Deli Zhao, Zhengjun Zha, Jingren Zhou, Qifeng Chen
Conference on Neural Information Processing Systems (NeurIPS)

In the repository, we propose LowRankGAN to locally control the image synthesis from GANs with the novel low-rank subspaces. Concretely, we first relate the image regions with the latent space with the help of Jacobian. We then perform low-rank factorization on the Jacobian to get the principal and null spaces. We finally project the principal space w.r.t. the region of interest onto the null space w.r.t. the rest region. In this way, by altering the latent codes along the directions within the projected space, which we call low-rank subspaces, we manage to precisely control the region of interest yet barely affect the rest region.

[Paper] [Project Page] [Demo]

NEWS: Please also find this repo for the Pytorch code!

Examples of Local Editing
Eyes	Mouth	Nose	Hair

We have already provided some directions under the directory directions/. Users can easily use these directions for image local editing.

MODEL_PATH='stylegan2-ffhq-config-f-1024x1024.pkl'
DIRECTION='directions/ffhq1024/eyes_size.npy'
python manipulate.py $MODEL_PATH $DIRECTION

We also provide the code for users to find customized directions. Please follow the steps below.

Here, we use the FFHQ model officially released in StyleGAN2 as an example. Please download it first.

MODEL_PATH='stylegan2-ffhq-config-f-1024x1024.pkl'
python compute_jacobian.py $MODEL_PATH

JACOBIAN_PATH='outputs/jacobian_seed_4/w_dataset_ffhq.npy'
python compute_directions.py $JACOBIAN_PATH

MODEL_PATH='stylegan2-ffhq-config-f-1024x1024.pkl'
DIRECTION_PATH='outputs/directions/${DIRECTION_NAME}'
python manipulate.py $MODEL_PATH $DIRECTION

@inproceedings{zhu2021lowrankgan,
  title     = {Low-Rank Subspaces in {GAN}s},
  author    = {Zhu, Jiapeng and Feng, Ruili and Shen, Yujun and Zhao, Deli and Zha, Zhengjun and Zhou, Jingren and Chen, Qifeng},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2021}
}