Intelligent recognition of multi-scale marine targets remains pivotal in studying marine resources and transportation. Multi-scale marine target recognition faces challenges such as blurred image, noise interference, varied target sizes, and random target positions. However, these hardly affect the judgment of human brain which could adeptly capture multi-scale targets and disregard noise interference. Therefore, this study proposes an innovative approach to recognize multi-scale marine targets through taking full advantages of the texture, color and structural information provided by remote sensing images and the quick classification ability of human brains, called Selective Kernel & Multi-dimensional Multimodal Data Fusion Module Network (SK-MMFMNet), which fuses remote sensing images and electroencephalography (EEG) signals to improve the accuracy of classifying multi-scale marine targets. In this study, we construct a multi-scale marine target dataset, which includes both remote sensing images of islands, wind turbines, and ships and their corresponding EEG signals from subjects while viewing remote sensing images. Then, the proposed approach extends the Multimodal Transfer Module (MMTM) based on attention mechanism to a dual fusion module across channel and spatial dimensions to fusing MobileNetV3 and EEGNet. Also, we embed the Selective Kernel Module into MobileNetV3 for addressing multi-scale features. The average experimental results across the three multi-scale marine target sub-dataset show that SK-MMFMNet exhibited accuracy improvements of 2.88 %, 21.60 %, and 1.08 %, moreover, F1-Score increments of 24.60 %, 162.22 %, and 14.32 % compared to MobileNetV3, EEGNet, and MMTMNet (MMTM-based fusion network). Visual analysis via Grad-CAM demonstrates that benefiting from EEG signals and Selective Kernel Module, our proposed SK-MMFMNet adjusts the network attention to exactly focus on the multi-scale target area, and thus achieves the best performance. Meanwhile, T-SNE visualization also proves the effectiveness of the three fusion modules and EEG signals for feature extraction. This study offers a valuable and promising insight for intelligent recognition of multi-scale marine targets.

中文翻译：

---

SK-MMFMNet：用于多尺度海洋目标识别的遥感图像和脑电信号多维融合网络

多尺度海洋目标的智能识别仍然是研究海洋资源和运输的关键。多尺度海洋目标识别面临图像模糊、噪声干扰、目标尺寸变化、目标位置随机等挑战。然而，这些几乎不影响人脑的判断，人脑能够熟练地捕捉多尺度目标并忽略噪声干扰。因此，本研究提出了一种充分利用遥感图像提供的纹理、颜色和结构信息以及人脑的快速分类能力来识别多尺度海洋目标的创新方法，称为选择性核和多维多模态数据融合模块网络（SK-MMFMNet），融合遥感图像和脑电图（EEG）信号，以提高多尺度海洋目标分类的准确性。在本研究中，我们构建了一个多尺度的海洋目标数据集，其中包括岛屿、风力涡轮机和船舶的遥感图像以及受试者在观看遥感图像时相应的脑电信号。然后，所提出的方法将基于注意力机制的多模态传输模块（MMTM）扩展到跨通道和空间维度的双融合模块，以融合MobileNetV3和EEGNet。此外，我们将选择性内核模块嵌入到 MobileNetV3 中以解决多尺度特征。三个多尺度海洋目标子数据集的平均实验结果表明，SK-MMFMNet 的准确率提高了 2.88%、21.60% 和 1.08%，此外，F1-Score 增量分别为 24.60%、162.22% 和 14.32%与 MobileNetV3、EEGNet 和 MMTMNet（基于 MMTM 的融合网络）相比。通过 Grad-CAM 的可视化分析表明，受益于 EEG 信号和选择性内核模块，我们提出的 SK-MMFMNet 调整网络注意力以准确聚焦于多尺度目标区域，从而实现最佳性能。同时，T-SNE可视化也证明了三个融合模块和EEG信号进行特征提取的有效性。这项研究为多尺度海洋目标的智能识别提供了有价值且有前景的见解。