GAMES Webinar 2021 – 178期(绘制专题) | 王至博 (Tsinghua University)，Duan Gao (Tsinghua University)
【GAMES Webinar 2020-178期】(绘制专题)
报告嘉宾1：王至博 (BNRist and school of software, Tsinghua University)
报告题目：Single Image Portrait Relighting via Explicit Multiple Reflectance Channel Modeling
Portrait relighting aims to render a face image under different lighting conditions. Existing methods do not explicitly consider some challenging lighting effects such as specular and shadow, and thus may fail in handling extreme lighting conditions. In this paper, we propose a novel framework that explicitly models multiple reflectance channels for single image portrait relighting, including the facial albedo, geometry as well as two lighting effects, i.e., specular and shadow. These channels are finally composed to generate the relit results via deep neural networks. Current datasets do not support learning such multiple reflectance channel modeling. Therefore, we present a large-scale dataset with the ground-truths of the channels, enabling us to train the deep neural networks in a supervised manner. Furthermore, we develop a novel module named Lighting guided Feature Modulation (LFM). In contrast to existing methods which simply incorporate the given lighting in the bottleneck of a network, LFM fuses the lighting by layer-wise feature modulation to deliver more convincing results. Extensive experiments demonstrate that our proposed method achieves better results and is able to generate challenging lighting effects.
Zhibo Wang（王至博） is currently working toward his Ph.D. degree in BNRist and School of Software, Tsinghua University. He received his B.S degree in Microelectronics Science and Technology from Nanjing University, Nanjing, China, in 2017. His research interests include facial animation, 3D face reconstruction and facial image editing.
报告嘉宾2：Duan Gao (Tsinghua University)
报告题目：Deferred Neural Lighting: Free-viewpoint Relighting from Unstructured Photographs
We present deferred neural lighting, a novel method for free-viewpoint re- lighting from unstructured photographs of a scene captured with handheld devices. Our method leverages a scene-dependent neural rendering network for relighting a rough geometric proxy with learnable neural textures. Key to making the rendering network lighting aware are radiance cues: global il- lumination renderings of a rough proxy geometry of the scene for a small set of basis materials and lit by the target lighting. As such, the light transport through the scene is never explicitely modeled, but resolved at rendering time by a neural rendering network. We demonstrate that the neural textures and neural renderer can be trained end-to-end from unstructured photographs captured with a double hand-held camera setup that concurrently captures the scene while being lit by only one of the cameras’ flash lights. In addi- tion, we propose a novel augmentation refinement strategy that exploits the linearity of light transport to extend the relighting capabilities of the neural rendering network to support other lighting types (e.g., environment lighting) beyond the lighting used during acquisition (i.e., flash lighting). We demonstrate our deferred neural lighting solution on a variety of real-world and synthetic scenes exhibiting a wide range of material properties, light transport effects, and geometrical complexity.
I am a fourth-year Ph.D. student at Tsinghua University, advised by Prof. Kun Xu. I received my bachelor degree from Department of Computer Science and Technology, Nanjing University in 2017. My research interests include: appearance modeling, physically-based rendering and real-time rendering.
过洁，南京大学计算机科学与技术系副研究员。2013年毕业于南京大学，获得博士学位。目前主要研究方向为真实感绘制、实时绘制、复杂材质分析建模以及虚拟现实技术等。发表学术论文40余篇，包括计算机图形学领域顶级会议/期刊ACM SIGGRAPH/SIGGRAPH ASIA、IEEE TVCG，视觉领域顶级会议CVPR、ECCV 等。主持人主页：。
GAMES主页的“使用教程”中有 “如何观看GAMES Webinar直播？”及“如何加入GAMES微信群？”的信息；