Abstract
Given the potential diminishment of semantic information caused by previous non-region-specific maximum or weighted intensity losses under different lighting conditions, we propose a novel high-level semantic perception fusion framework, termed Semantic Region Adaptive Fusion (SRAFusion). We establish a prioritized preservation scheme for high-level semantic information (HLSI), gradient information, and intensity information, ranked in descending order of priority. Based on this prioritization scheme and prior knowledge of semantic distribution in the source images, we construct a ground-truth for the fusion task. Specifically, we capture the HLSI distribution of the source images by utilizing two independent semantic segmentation networks. Subsequently, we introduce Semantic Region Decision Blocks (SRDB) to partition the original scene into region with bimodal HLSI, region with unimodal HLSI, and region lacking HLSI. We then design specific loss functions to constrain the aforementioned regions, facilitating the integration of complete semantic information. Furthermore, taking into account the susceptibility of the visible segmentation network to lighting conditions, we employ a two-stage training strategy involving coarse-tuning and fine-tuning. This method aims to optimize one-stage training strategy and achieve a more accurate region delineation. Finally, qualitative and quantitative experiments conducted on publicly available datasets such as MFNet , RoadScene , and TNO demonstrate the superiority of our SRAFusion over state-of-the-art methods. Our code will be available: https://github.com/WenziCao/SRAFusion.

This publication has 63 references indexed in Scilit: