Infrared and visible image fusion with ResNet and zero-phase component analysis

Hui Li, Xiao jun Wu, Tariq S. Durrani

Research output: Contribution to journalArticlepeer-review

42 Citations (Scopus)
18 Downloads (Pure)


In image fusion approaches, feature extraction and processing are key tasks, and the fusion performance is directly affected by the different features and processing methods undertaken. However, most of deep learning-based methods use deep features directly without them. This leads to the fusion performance degradation in some cases. To solve these drawbacks, in our paper, a deep features and zero-phase component analysis (ZCA) based novel fusion framework is proposed. Firstly, the residual network (ResNet) is used to extract deep features from source images. Then ZCA and l1-norm are utilized to normalize the deep features and obtain initial weight maps. The final weight maps are obtained by employing a soft-max operation in association with the initial weight maps. Finally, the fused image is reconstructed using a weighted-averaging strategy. Compared with the existing fusion methods, experimental results demonstrate that the proposed framework achieves better performance in both objective assessment and visual quality. The code of our fusion algorithm is available at

Original languageEnglish
Article number103039
Number of pages10
JournalInfrared Physics and Technology
Early online date12 Sep 2019
Publication statusPublished - 30 Nov 2019


  • deep learning
  • image fusion
  • infrared image
  • residual network
  • visible image
  • zero-phase component analysis

Cite this