Multimodality is a universal characteristic of multi-source monitoring data for rotating machinery. The correlation fusion of multimodal information is a general law to strengthen the cognition of fault features, and an effective way to improve the reliability and robustness of fault diagnosis methods. However, the physical connotation gaps between multimodal information hinder the construction of correlations, preventing mainstream machine learning (ML) based intelligent diagnosis methods from reliably and effectively taking advantage of multimodal fusion. To address these issues, a physics-inspired multimodal fusion convolutional neural network (PMFN) is proposed in this paper. It is the first attempt to integrate physical knowledge into ML models to bridge physical connotation gaps between multimodal fault information. Specifically, the characterization patterns of rotating machinery fault in multimodal information are embedded in the attention mechanism to focus on representative fault features with physical properties, and generate the universal representation of multimodal information. Furthermore, the cross-modal correlation fusion module is introduced to adaptively construct the correlations of multimodal information, thereby highlighting the unique feature of unimodal information and the shared representation of multimodal information. Finally, the superiority of the proposed fusion method is verified by two cases of industrial gearbox and bearing-rotor system.

中文翻译：

---

基于物理启发的多模态机器学习，用于基于自适应相关融合的旋转机械故障诊断

多模态是旋转机械多源监测数据的普遍特征。多模态信息的关联融合是加强故障特征认知的一般规律，是提高故障诊断方法可靠性和鲁棒性的有效途径。然而，多模态信息之间的物理内涵差距阻碍了相关性的构建，阻碍了主流的基于机器学习（ML）的智能诊断方法可靠有效地利用多模态融合。为了解决这些问题，本文提出了一种受物理启发的多模态融合卷积神经网络（PMFN）。这是首次尝试将物理知识集成到机器学习模型中，以弥合多模态故障信息之间的物理内涵差距。具体来说，将多模态信息中旋转机械故障的表征模式嵌入到注意力机制中，以关注具有物理属性的代表性故障特征，并生成多模态信息的通用表示。此外，引入跨模态相关性融合模块来自适应地构建多模态信息的相关性，从而突出单模态信息的独特性和多模态信息的共享表示。最后，通过工业齿轮箱和轴承-转子系统的两个案例验证了所提出融合方法的优越性。