The official code repository for the paper: AugmentIQ: Revolutionizing Image Quality Assessment with Advanced Data Augmentation and Dynamic Data Loading Techniques.

AugmentIQ represents a paradigm shift in the realm of Image Quality Assessment (IQA). This innovative model is a fusion of advanced methodologies from Re-IQA, which offers nuanced image quality measurement techniques, and ImageReward, known for its incisive alignment evaluation between images and textual prompts. Unlike traditional models, AugmentIQ excels in assessing both the aesthetic and technical quality of images and their semantic congruence with given textual descriptors, embodying a dual-capability framework that marks a significant advancement in automated image evaluation.

In essence, AugmentIQ is more than an addition to the compendium of IQA methodologies; it is a groundbreaking approach that aligns with the complexities of modern image generation and processing technologies. Its development signifies a new era in image quality assessment, one that is attuned to both the aesthetic beauty and the semantic relevance of images.

🤗 Please cite AugmentIQ in your publications if it helps with your work. Please star🌟 this repo to help others notice AugmentIQ if you think it is useful. It really means a lot to our open-source research. Thank you! BTW, you may also like ImageReward, ReIQA, the two great open-source repositories upon which we built our architecture.

📣 Attention please: Due to the time limit, the implementation in this repo may not achieve the best result, and also considering we haven't running extensive parameters fintuning process due to time and resource limit,the best results may still be on the way ! 😉

⦿ Contributions:

• Our integrated model, synthesizing the methodsologies of Re-IQA and ImageReward, represents the next step in this evolutionary path. It not only incorporates the technical advancements in assessing image fidelity and aesthetic quality but also introduces a novel dimension of evaluating text-image semantic congruence.

• This integration signifies a broader trend in IQA research, one that acknowledges the multi-dimensional nature of image quality in the age of AI and seeks to develop assessment tools that are as dynamic and multifaceted as the images they evaluate.

⦿ Performance: SAITS outperforms Re-IQA on the [AIGC-3k]

Here we only show the main component of our method: the joint-optimization training approach combining three encoders while frozening their own weights. For the detailed description and explanation, please read our full paper if you are interested.

Fig. 1: Training approach

The implementation of SAITS is in dir IQAx.Please install it via pip install -e . or python setup.py install. Due to the time and resource limit, we haven't performed extensive enough parameter finetuning experiments, if you like this repo, please feel free to fork and PR to help us improve it ! 💚 💛 🤎.

We run on Ubuntu 22.04 LTS with a system configured with a NVIDIA RTX 3090 GPU.

• Use conda to create a env for AugmentIQ and activate it.

conda create -n AugmentIQ python=3.8
conda activate AugmentIQ

• Install the necessary dependencies in the conda env

pip install -r requirements.txt

• Then install AugmentIQ as a package

cd AugmentIQ
pip install -e .

We run on two datasets, more specifically, AGIQA-3k-Database and AIGCIQA2023 Here are some samples taken randomly from the dataset:

Now the directory tree should be the following:

- AIGC-3k
    - image
    - data.csv
- AIGCIQA-2023
    - DATA
    - Image
      - allimg
    - prompts.xlsx

Please refer to the Re-IQA repository to download the content_aware_r50.pth and the quality_aware_r50.pth, and put them under the directory $ROOT/IQAx/IQAx/re-iqa_ckpts/. Also please take a tour to the ImageReward repo and download ImageReward.pt and med_config.json and put them under the $ROOT/IQAx/ImageReward/pretrained_model.

python $ROOT_DIRECTORY/augmentIQ/demo_AIGCIQA.py --aug --n_args=4 --gpu=$gpu

python $ROOT_DIRECTORY/augmentIQ/demo_AIGC3K.py --aug --n_args=4 --gpu=$gpu

❗️Note that paths of datasets and saving dirs may be different on personal computers, please check them in the configuration files.

The training curves and validation curves of our model on AIGCIQA-2023 dataset and are shown below:

The training curves of our model on AIGC-3k dataset are shown below:

The metrics on test dataset is Spearmans Rank Correlation Coefficient(SRCCle), Pearson Correlation Coefficient(PLCC):

I extend my heartfelt gratitude to the esteemed faculty and dedicated teaching assistants of CS3324 for their invaluable guidance and support throughout my journey in image process- ing. Their profound knowledge, coupled with an unwavering commitment to nurturing curiosity and innovation, has been instrumental in my academic and personal growth. I am deeply appreciative of their efforts in creating a stimulating and enriching learning environment, which has significantly contributed to the development of this paper and my under- standing of the field. My sincere thanks to each one of them for inspiring and challenging me to reach new heights in my studies.

If you have any additional questions or have interests in collaboration,please take a look at my GitHub profile and feel free to contact me 😃.