SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024.
The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $\mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $\epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.

中文翻译：

---

西格斯特

SIAM Review，第 66 卷，第 2 期，第 317-317 页，2024 年 5 月。
本期的 SIGEST 文章是“Stiefel 流形及其他的非平滑优化：近端梯度方法和最新变体”，作者：Shiyang Chen、Shiqian Ma、Anthony苏曼初，张桐。这项工作考虑了 Stiefel 流形上的非光滑优化，Stiefel 流形是 $\mathbb{R}^n$ 中正交 $k$ 框架的流形。作者提出了一种新颖的近端梯度算法，创造了 ManPG，用于最小化平滑的、潜在的非凸函数和凸的、潜在的非平滑函数（其参数存在于 Stiefel 流形上）的总和。与现有方法相比，现有方法要么计算成本高（由于昂贵的子问题或收敛速度慢），要么缺乏严格的收敛保证，ManPG 进行了彻底的分析，并具有可以有效计算的子问题。 Stiefel 流形上的非光滑优化问题出现在许多应用中。在统计稀疏主成分分析 (PCA) 中，即寻找具有很少非零条目的主成分的 PCA，就是一个典型的例子。无监督特征选择（机器学习）和对解卷积信号进行稀疏约束的盲解卷积（逆问题）是这种通用目标结构的重要实例。这项工作的核心是针对一类重要问题的理论上有充分依据且高效的新颖优化方法与证明这种新方法有效性的一组计算实验之间的完美相互作用。为了使近端梯度适用于 Stiefel 流形，他们在迭代中添加了一个回缩步骤，以保持迭代可行。作者证明了 ManPG 全局收敛到一个驻点，并分析了其计算复杂性，以将后者近似到 $\epsilon$ 精度。数值讨论的重点是稀疏 PCA 的结果以及计算压缩模式的问题，即独立粒子薛定谔方程的空间局部解。 2020 年的原创文章发表在《SIAM Journal on Optimization》上，引起了相当大的关注。在准备这个 SIGEST 版本时，作者添加了对一些后续工作的讨论，这些工作涉及求解非光滑目标黎曼优化的算法。这些工作主要是由 ManPG 算法推动的，包括流形近端点算法、流形近端线性算法、随机 ManPG、零阶 ManPG、黎曼近端梯度法和黎曼近端牛顿法。