This is the official repo for AAAI 2022 paper(Oral):

Inharmonious Region Localization by Magnifying Domain Discrepancy (MadisNet)
Jing Liang¹, Li Niu¹, Penghao Wu¹, Fengjun Guo², Teng Long^2
1MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University
²INTSIG

Paper | Bibtex | Models

Our MadisNet has been integrated into our image composition toolbox libcom https://github.com/bcmi/libcom. Welcome to visit and try ＼(^▽^)／

Here are some examples of inharmonious images (top row) and their inharmonious region masks (bottom row). These inharmonious region could be infered by comparing the illuminance or color with surrounding area.

The top figure depicts our proposed framework MadisNet. The bottom figure elaborates our proposed improved HDRNet(iHDRNet).

• Install PyTorch>=1.0 following the official instructions
• git clone https://github.com/bcmi/MadisNet-Inharmonious-Region-Localization.git
• Install dependencies: pip install -r requirements.txt

In this paper, we conduct all of the experiments on the latest released harmonization dataset iHarmoney4.

Following suggestion of DIRL, we tailor the training set to 64255 images and test set to 7237 images respectively, yielding le50_train.txt and le50_test.text files in this project. And you can further divide the training list into training set and validation set, in which we randomly choose 10% items in le50_train.txt as validation set.

If you want to use other datasets, please follow the dataset loader file:dataset/ihd_dataset.py

Please specify the bash file. We provide a training and a test bash examples:scripts/train.sh, scripts/test.sh

One quick training command:

python3  train.py --dataset_root <PATH_TO_DATASET> --checkpoints_dir <PATH_TO_SAVE> --batch_size 8 --gpu_ids 0 --batch_norm --model dirl

• MadisNet-DIRL (Google Drive | One Drive)
• MadisNet-UNet (Google Drive | One Drive)

Here we list the quantitative results with / without our framework based on AP metric. For more details, please refer to our paper.

We also show qualitative comparision with state-of-art methods of other related fields:

To evaluate the effectiveness of inharmonious region localization for multiple inharmonious regions, we also prepare a test set with multiple foregrounds. Based on HCOCO test set, we composite 19,482 synthetic images with multiple foregrounds, in which the number of foregrounds ranges from 2 to 9. The test set can be downloaded from Baidu Cloud.

If you find this work or code is helpful in your research, please cite:

@inproceedings{jing2022inharmonious,
  title={Inharmonious Region Localization by Magnifying Domain Discrepancy},
  author={Jing, Liang and Li, Niu and Penghao, Wu and Fengjun, Guo and Teng, Long},
  booktitle={AAAI},
  year={2022}
}

[1] Inharmonious Region Localization by Magnifying Domain Discrepancy. Jing Liang, Li Niu, Penghao Wu, Fengjun Guo, and Teng Long. Accepted by AAAI 2022. download

[2] Inharmonious Region Localization. Jing Liang, Li Niu, Liqing Zhang. Accepted by ICME 2021. download