This repository is still in updating ...

This repository contains online resources of the paper "Semantic Compression Embedding for Generative Zero-Shot Learning" accepted to IJCAI-2022. [PDF]

Motivation (Left: existing generative ZSLs, Right: proposed SC-EGG):

The implementation of SC-EGG is mainly based on Python 3.8.8 and PyTorch 1.8.0. We use Weights & Biases (W&B) to keep track and organize the results of experiments. Please follow the online documentation of W&B to quickstart. To install all required dependencies:

$ pip install -r requirements.txt

We trained the model on CUB, SUN and AWA2 following the data split of xlsa17. In order to train the SC-EGG, you should firstly download these datasets as well as the xlsa17. Then decompress and organize them as follows:

.
├── data
│   ├── CUB/CUB_200_2011/...
│   ├── SUN/images/...
│   ├── AWA2/Animals_with_Attributes2/...
│   └── xlsa17/data/...
└── ···

Preprocessing the visual features is also needed:

$ python preprocessing.py --dataset CUB --compression
$ python preprocessing.py --dataset SUN --compression
$ python preprocessing.py --dataset AWA2 --compression

In ./wandb_config, we provide our parameters setting of conventional ZSL (CZSL) and generalized ZSL (GZSL) tasks for CUB, SUN, and AWA2. You can run the following commands to train the SC-EGG from scratch:

$ python train_SUN.py

We present the t-SNE visualizations of real seen/unseen and synthetic unseen visual features in CNN backbone visual space and dense-semantic visual space.

CUB Dataset: SUN Dataset: AWA2 Dataset:

The first column shows the input images, the second column shows the global attention maps of GEN, and the other columns are local attention maps of LEN with top-8 attention scores.

Parts of the codes based on:

• hbdat/cvpr20_DAZLE
• akshitac8/tfvaegan

If this work is helpful for you, please cite our paper.

@inproceedings{ijcai2022p0134,
  title     = {Semantic Compression Embedding for Generative Zero-Shot Learning},
  author    = {Hong, Ziming and Chen, Shiming and Xie, Guo-Sen and Yang, Wenhan and Zhao, Jian and Shao, Yuanjie and Peng, Qinmu and You, Xinge},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {956--963},
  year      = {2022},
  month     = {7},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2022/134},
  url       = {https://doi.org/10.24963/ijcai.2022/134},
}