SSTM: Semantic Segmentation-based Template Matching method for heterogenous images

Global navigation satellite systems (GNSS) have disadvantages such as signal interference, loss, and spoofing. This has led to the development of new methods to complement or replace satellite navigation. Absolute visual localization is one of the main methods of vision-based localization. The goal is to localize the current view of the UAV in a reference satellite map or in a georeferenced image of a previous flight. During nighttime, due to poor lighting conditions, infrared images are used as a substitute for visible light images. However, the significant differences between infrared and visible light images pose challenges in matching them. Therefore, this paper proposes a Semantic Segmentation-based Template Matching (SSTM) method for heterogeneous images. The method proposed in this paper breaks the general idea of using a single feature extraction algorithm for heterogeneous image matching. In this paper, the matching of UAV infrared images with visible light satellite remote sensing images is realized by unifying two different image feature extraction modules of the feature description language and the matching scheme of this feature description language. The advantages of this method are demonstrated through comparisons with other advanced techniques. With the development of semantic segmentation techniques, it is believed that this modular “translation” method can achieve matching between arbitrary sensors.