GraphIQA: Learning Distortion Graph Representations for Blind Image Quality Assessment

GraphIQA,
Simeng Sun*, Tao Yu*(Equal contribution), Jiahua Xu, Jianxin Lin, Wei Zhou, Zhibo Chen,
arXiv preprint (arXiv:2103.07666)

Learning-based blind image quality assessment (BIQA) methods have recently drawn much attention for their superior performance compared to traditional methods. However, most of them do not effectively leverage the relationship between distortion-related factors, showing limited feature representation capacity. In this paper, we show that human perceptual quality is highly correlated with distortion type and degree, and is biased by image content in most IQA systems. Based on this observation, we propose a Distortion Graph Representation (DGR) learning framework for IQA, called GraphIQA. In GraphIQA, each distortion is represented as a graph i.e. DGR. One can distinguish distortion types by comparing different DGRs, and predict image quality by learning the relationship between different distortion degrees in DGR. Specifically, we develop two sub-networks to learn the DGRs: a) Type Discrimination Network (TDN) that embeds DGR into a compact code for better discriminating distortion types; b) Fuzzy Prediction Network (FPN) that extracts the distributional characteristics of the samples in DGR and predicts fuzzy degrees based on a Gaussian prior. Experiments show that our GraphIQA achieves the state-of-the-art performance on many benchmark datasets of both synthetic and authentic distortion.

This project is run on GPU (NVIDIA 2TX 2080Ti) with CUDA 10.0 and CPU (Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz).

1. [Optional but recommended] Create a env environment using miniconda, run:

   conda create --name GraphIQA python ==3.7.7
   source activate GraphIQA
   

2. Install pytorch 1.5.0:

   conda install pytorch==1.5.1 torchvision==0.6.1 cudatoolkit=10.2 -c pytorch
   

3. Install required packages by running:

   pip install -r requirements.txt
   

4. Download Kadis700k dataset and generate the distorted images according to the official guidance, then restore them in ./data/kadis700k/dist_imgs. Doneload LIVE, LIVEC, KonIQ10k and CSIQ in ./data/databaserelease2, ./data/ChallengeDB, ./data/KonIQ and ./data/csiq respectively.

When pretraining GraphIQA model from scratch on kadid10k dataset, you need to run:

python pretrain.py --dataset kadid-P

When pretraining GraphIQA model from scratch on kadid10k dataset, you need to run:

python pretrain.py --dataset kadis-P

Some available options:

• --dataset: Training and testing dataset, support datasets: kadid-P | kadis-P
• --gpus: The number of required gpus. For pretraining on kadis700k dataset, up to 4 gpus are needed.
• --gpu_ids: Visible devices.
• --bs: Batch size.
• --margin: Margin for triplet loss.

The pretrained model will be uploaded soon.

When finetuning GraphIQA model on target dataset, you need to run:

python finetune.py --dataset DATASET --restore --ckpt PATH/TO/PRETRAINED_MODEL --gpus 1 --gpu_ids 0 --bs 32

Some available options:

• --dataset: Training and testing dataset, support datasets: live | livec | koniq-10k | csiq | kadid10k
• --restore: Enable to load pretrained model.
• --ckpt: Path to pretrained model.
• --gpus: The number of required gpus. For finetuning, 1 gpu is enough.
• --gpu_ids: Visible devices.
• --bs: Batch size.

Please cite us if you find this work helps.

@article{sun2021graphiqa,
  title={GraphIQA: Learning Distortion Graph Representations for Blind Image Quality Assessment},
  author={Sun, Simeng and Yu, Tao and Xu, Jiahua and Lin, Jianxin and Zhou, Wei and Chen, Zhibo},
  journal={arXiv preprint arXiv:2103.07666},
  year={2021}
}