This is official tensorflow implementation of “PIAFusion: A progressive infrared and visible image fusion network based on illumination aware”.

The PyTorch implementation of our project, accomplished by @linklist2, can be fetched from https://github.com/linklist2/PIAFusion_pytorch.

A new benchmark dataset for infrared and visible fusion are released in this paper, which is termed MSRS.

An example of illumination imbalance. From left to right: infrared image, visible image, the fused results of DenseFuse, FusionGAN, and our proposed PIAFusion. The visible image contains abundant information, such as texture details in the daytime (top row). But salient targets and textures are all included in the infrared image at nighttime (bottom row). Existing methods ignore the illumination imbalance issues, causing detail loss and thermal target degradation. Our algorithm can adaptively integrate meaningful information according to illumination conditions.

• tensorflow-gpu 1.14.0
• scipy 1.2.0
• numpy 1.19.2
• opencv 3.4.2

Run: "python main.py --epoch=100 --is_train=True model_type=Illum --DataSet=MSRS" The dataset for training the illumination-aware sub-network can be download from data_illum.h5.

Run: "python main.py --epoch=30 --is_train=True model_type=PIAFusion --DataSet=MSRS" The dataset for training the illumination-aware fusion network can be download from data_MSRS.h5.

Run: "python main.py --is_train=False model_type=PIAFusion --DataSet=MSRS"

Run: "python main.py --is_train=False model_type=PIAFusion --DataSet=RoadScene"

Run: "python main.py --is_train=False model_type=PIAFusion --DataSet=TNO"

Our Multi-Spectral Road Scenarios (MSRS) dataset is constructed on the basis of the MFNet dataset[1].

[1] Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., Harada, T., 2017. Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes, in: Proceedings of the IEEE International Conference on Intelligent Robots and Systems, pp.5108–5115.

@article{Tang2022PIAFusion,
  title={PIAFusion: A progressive infrared and visible image fusion network based on illumination aware},
  author={Tang, Linfeng and Yuan, Jiteng and Zhang, Hao and Jiang, Xingyu and Ma, Jiayi},
  journal={Information Fusion},
  volume = {83-84},
  pages = {79-92},
  year = {2022},
  issn = {1566-2535},
  publisher={Elsevier}
}