Developing fast surrogates for Partial Differential Equations (PDEs) will accelerate design and optimization in almost all scientific and engineering applications. Neural networks have been receiving ever-increasing attention and demonstrated remarkable success in computational modeling of PDEs, however; their prediction accuracy is not at the level of full deployment. In this work, we utilize the transformer architecture, the backbone of numerous state-of-the-art AI models, to learn the dynamics of physical systems as the mixing of spatial patterns learned by a convolutional autoencoder. Moreover, we incorporate the idea of multi-scale hierarchical time-stepping to increase the prediction speed and decrease accumulated error over time. Our model achieves similar or better results in predicting the time-evolution of Navier–Stokes equations compared to the powerful Fourier Neural Operator (FNO) and two transformer-based neural operators OFormer and Galerkin Transformer. The code and data are available on .

中文翻译：

---

使用变压器的偏微分方程的多尺度时间步进

开发偏微分方程 (PDE) 的快速代理将加速几乎所有科学和工程应用的设计和优化。然而，神经网络已受到越来越多的关注，并在偏微分方程的计算建模方面取得了显着的成功。他们的预测精度还没有达到全面部署的水平。在这项工作中，我们利用变压器架构（众多最先进的人工智能模型的支柱）来学习物理系统的动态，作为卷积自动编码器学习的空间模式的混合。此外，我们结合了多尺度分层时间步长的思想来提高预测速度并减少随着时间的推移累积的误差。与强大的傅里叶神经算子（FNO）和两个基于变压器的神经算子 OFormer 和 Galerkin Transformer 相比，我们的模型在预测纳维-斯托克斯方程的时间演化方面取得了相似或更好的结果。代码和数据可在 上获取。