3D-aware portrait lighting control is an emerging and promising domain, thanks to the recent advance of generative adversarial networks and neural radiance fields. Existing solutions typically try to decouple the lighting from the geometry and appearance for disentangled control with an explicit lighting representation (e.g., Lambertian or Phong). However, they either are limited to a constrained lighting condition (e.g., directional light) or demand a tricky-to-fetch dataset as supervision for the intrinsic compositions (e.g., the albedo). We propose NeRFFaceLighting to explore an implicit representation for portrait lighting based on the pretrained tri-plane representation to address the above limitations. We approach this disentangled lighting-control problem by distilling the shading from the original fused representation of both appearance and lighting (i.e., one tri-plane) to their disentangled representations (i.e., two tri-planes) with the conditional discriminator to supervise the lighting effects. We further carefully design the regularization to reduce the ambiguity of such decomposition and enhance the ability of generalization to unseen lighting conditions. Moreover, our method can be extended to enable 3D-aware real portrait relighting. Through extensive quantitative and qualitative evaluations, we demonstrate the superior 3D-aware lighting control ability of our model compared to alternative and existing solutions.

由于生成对抗网络和神经辐射领域的最新进展，3D 感知肖像照明控制是一个新兴且有前途的领域。现有的解决方案通常会尝试将照明与几何和外观分离，以便使用明确的照明表示（例如 Lambertian 或 Phong）来分离控制。然而，它们要么限于受限的光照条件（例如，定向光），要么需要一个难以获取的数据集来监督内在成分（例如，反照率）。我们提出NeRFFaceLighting基于预训练的三平面表示探索肖像照明的隐式表示，以解决上述限制。我们通过使用条件鉴别器从外观和照明的原始融合表示（即一个三平面）中提取阴影到它们的分离表示（即两个三平面）来处理这个解缠结的照明控制问题，以监督照明效果。我们进一步精心设计了正则化，以减少这种分解的歧义，并增强对看不见的光照条件的泛化能力。此外，我们的方法可以扩展以启用 3D 感知的真实肖像重新照明。通过广泛的定量和定性评估，