当前位置: X-MOL 学术Appl. Comput. Harmon. Anal. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Stable parameterization of continuous and piecewise-linear functions
Applied and Computational Harmonic Analysis ( IF 2.5 ) Pub Date : 2023-08-09 , DOI: 10.1016/j.acha.2023.101581
Alexis Goujon , Joaquim Campos , Michael Unser

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function spaces lacks stability. In this paper, we investigate an alternative representation of CPWL functions that relies on local hat basis functions and that is applicable to low-dimensional regression problems. It is predicated on the fact that any CPWL function can be specified by a triangulation and its values at the grid points. We give the necessary and sufficient condition on the triangulation (in any number of dimensions and with any number of vertices) for the hat functions to form a Riesz basis, which ensures that the link between the parameters and the corresponding CPWL function is stable and unique. In addition, we provide an estimate of the 2L2 condition number of this local representation. As a special case of our framework, we focus on a systematic parameterization of Rd with control points placed on a uniform grid. In particular, we choose hat basis functions that are shifted replicas of a single linear box spline. In this setting, we prove that our general estimate of the condition number is exact. We also relate the local representation to a nonlocal one based on shifts of a causal ReLU-like function. Finally, we indicate how to efficiently estimate the Lipschitz constant of the CPWL mapping.



中文翻译:

连续和分段线性函数的稳定参数化

整流线性单元 (ReLU) 神经网络在深度学习中发挥着重要作用,可生成连续和分段线性 (CPWL) 函数。虽然它们提供了强大的参数表示,但参数和函数空间之间的映射缺乏稳定性。在本文中,我们研究了 CPWL 函数的替代表示,该表示依赖于局部帽子基函数并且适用于低维回归问题。其前提是任何 CPWL 函数都可以通过三角测量及其在网格点处的值来指定。我们给出了帽子函数形成 Riesz 基的三角剖分(任意维度和任意数量的顶点)的充分必要条件,这确保了参数和相应的CPWL函数之间的链接是稳定且唯一的。此外,我们还提供了对2L2该本地表示的条件号。作为我们框架的一个特例,我们专注于系统参数化d控制点放置在统一的网格上。特别是,我们选择帽子基函数,它们是单个线性箱样条的移位副本。在这种情况下,我们证明我们对条件数的一般估计是准确的。我们还基于因果 ReLU 函数的移位将局部表示与非局部表示联系起来。最后,我们指出如何有效地估计 CPWL 映射的 Lipschitz 常数。

更新日期:2023-08-09
down
wechat
bug