This project explores the early stages of incorporating self-supervised with algorithm unrolling. Code was written as part of a mathematical-engineering master's thesis (60 ECTS) @ Aalborg University, Denmark.

The implementation of ISTA-Net and MAE (with ViT) is based on the paper "ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing" and "Masked Autoencoders Are Scalable Vision Learners" respectively.

Created using Python 3.10.6. See requirements.txt for further details.

The script is designed such that variables are changed directly in the code. The training.py script builds and executes the training loop -- Just provide a list image paths.

The script is designed such that parameters are set in a parameter file, parameters.py or parameters_finetuning.py for pre-training and fine-tuning respectively. The path to the data needs to be specified in the parameter file as well as a model path for fine-tuning. After setting parameters run one of the training files:

• pre-train_istamae.py
• pre-train_mae.py
• train_sr_istamae.py
• train_sr_mae.py

Inspired by the original PyTorch implementation "ISTA-Net-PyTorch" by Jian Zhang and "Masked Autoencoders: A PyTorch Implementation" by Xinlei Chen and Kaiming He.

@mastersthesis{jonhardsson2023,
    author       = {Jónhardsson, Magnus and Jørgensen, Mads and Larsen, Andreas},
    school       = {Aalborg University},
    title        = {Combining Algorithm Unrolling with Self-Supervised Learning for Compressed Sensing Image Super-Resolution},
    year         = {2023}
}