Paper Title: "CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model"

Conference Accepted: ICME 2023 (oral)

In this work, we propose a controllable Chinese landscape painting generation method named CCLAP, which can generate painting with specific content and style based on Latent Diffusion Model. Specifically, it consists of two cascaded modules, i.e., content generator and style aggregator. The content generator module guarantees the content of generated paintings specific to the input text. While the style aggregator module is to generate paintings of a style corresponding to a reference image. Moreover, a new dataset of Chinese landscape paintings named CLAP is collected for comprehensive evaluation. Both the qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance, especially in artfully-composed and artistic conception.

Dataset contains 3560 paintings with corresponding captions.

Send the Email to us if you want to apply for the dataset, showing your school or company and your application purpose. We will give a feedback within 5 days.

Email address: [email protected]

The Content Generator here is a .ckpt format, a Diffuser Models format is available at HERE.

You can play with our demo at huggingface space.

We are very passionate about the issue of creating Chinese landscape paintings and are constantly studying these topics:

• Solve the problem that the model cannot generate human beings well.
• What is the “style” in Chinese landscape painting? The “style” or we call “genre” is a kind of thing that contains the rule or the knowledge about how we color a painting.
• Can we control the composition of the painting?
• A painting contains a spirit.
• How can we generate a painting in a more human way?
• ...

🤝 Feel free to discuss with us privately!

If you find this project useful in your research, please consider cite:

@INPROCEEDINGS{10219843,
  author={Wang, Zhongqi and Zhang, Jie and Ji, Zhilong and Bai, Jinfeng and Shan, Shiguang},
  booktitle={2023 IEEE International Conference on Multimedia and Expo (ICME)}, 
  title={CCLAP: Controllable Chinese Landscape Painting Generation Via Latent Diffusion Model}, 
  year={2023},
  pages={2117-2122},
  doi={10.1109/ICME55011.2023.00362}}