Kavli Affiliate: Jiansheng Chen
| First 5 Authors: Jiawei Li, Jiansheng Chen, Jinyuan Liu, Huimin Ma,
| Summary:
Infrared and visible image fusion has gradually proved to be a vital fork in
the field of multi-modality imaging technologies. In recent developments,
researchers not only focus on the quality of fused images but also evaluate
their performance in downstream tasks. Nevertheless, the majority of methods
seldom put their eyes on the mutual learning from different modalities,
resulting in fused images lacking significant details and textures. To overcome
this issue, we propose an interactive graph neural network (GNN)-based
architecture between cross modality for fusion, called IGNet. Specifically, we
first apply a multi-scale extractor to achieve shallow features, which are
employed as the necessary input to build graph structures. Then, the graph
interaction module can construct the extracted intermediate features of the
infrared/visible branch into graph structures. Meanwhile, the graph structures
of two branches interact for cross-modality and semantic learning, so that
fused images can maintain the important feature expressions and enhance the
performance of downstream tasks. Besides, the proposed leader nodes can improve
information propagation in the same modality. Finally, we merge all graph
features to get the fusion result. Extensive experiments on different datasets
(TNO, MFNet and M3FD) demonstrate that our IGNet can generate visually
appealing fused images while scoring averagely 2.59% mAP@.5 and 7.77% mIoU
higher in detection and segmentation than the compared state-of-the-art
methods. The source code of the proposed IGNet can be available at
https://github.com/lok-18/IGNet.
| Search Query: ArXiv Query: search_query=au:”Jiansheng Chen”&id_list=&start=0&max_results=3