This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL .

中文翻译：

---

学习从 RGB 图像中恢复光谱反射率

本文解决了 RGB 图像的光谱反射率恢复 (SRR) 问题。由于捕获地面实况光谱反射率和相机光谱灵敏度具有挑战性且成本高昂，因此大多数现有方法都是在合成图像上进行训练，并对所有未见过的测试图像使用相同的参数，这是次优的，尤其是在真实图像上测试训练模型时，因为它们切勿利用测试图像的内部信息。为了解决这个问题，我们采用自监督元辅助学习（MAXL）策略，利用每个测试图像微调训练有素的网络参数，以将外部信息与内部信息结合起来。据我们所知，这是第一个成功地将 MAXL 策略应用于该问题的工作。我们不依赖于简单的端到端训练，而是提出了一种新颖的架构，根据我们的数学分析将光谱反射率和相应的 RGB 图像之间的物理关系集成到网络中。此外，由于场景的光谱反射率与其照明无关，而相应的 RGB 图像则不然，因此我们从多个照明下捕获的 RGB 图像中恢复场景的光谱反射率，以进一步减少未知数。定性和定量评估证明了我们提出的网络和 MAXL 的有效性。我们的代码和数据可在https://github.com/Dong-Huo/SRR-MAXL 。