GAMES Webinar 2021 – 206期(绘制专题) | Mengtian Li (Kuaishou Technology)，Shuichang Lai（Nanjing University）
【GAMES Webinar 2021-206期】(绘制专题)
报告嘉宾1：Mengtian Li (Kuaishou Technology)
报告题目：Volumetric Appearance Stylization with Stylizing Kernel Prediction Network
This work aims to efficiently construct the volume of heterogeneous single scattering albedo for a given medium that would lead to desired color appearance. We achieve this goal by formulating it as a volumetric style transfer problem in which an input 3D density volume is stylized using color features extracted from a reference 2D image. Unlike existing algorithms that require cumbersome iterative optimizations, our method leverages a feed-forward deep neural network with multiple well-designed modules. At the core of our network is a stylizing kernel predictor (SKP) that extracts multi-scale feature maps from a 2D style image and predicts a handful of stylizing kernels as a highly non-linear combination of the feature maps. Each group of stylizing kernels represents a specific style. A volume autoencoder (VolAE) is designed and jointly learned with the SKP to transform a density volume to an albedo volume based on these stylizing kernels. Since the autoencoder does not encode any style information, it can generate different albedo volumes with a wide range of appearance once training is completed. Additionally, a hybrid multi-scale loss function is used to learn plausible color features and guarantee temporal coherence for time-evolving volumes. Through comprehensive experiments, we validate the effectiveness of our method and show its superiority by comparing against state-of-the-arts. We show that with our method a novice user can easily create a diverse set of realistic translucent effects for 3D models (either static or dynamic), neglecting any cumbersome process of parameter tuning.
Mengtian Li is currently an algorithm engineer at Kuaishou Technology. He received his M.Sc. degree from Department of Computer Science and Technology, Nanjing University in 2020, supervised by Jie Guo. His research interests include: Image Translation and deep learning for rendering.
报告嘉宾2：Shuichang Lai（Nanjing University）
报告题目：Highlight-Aware Two-Stream Network for Single-Image SVBRDF Acquisition
Our work is about the task of estimating spatially-varying reflectance (i.e., SVBRDF) from a single, casually captured image. Central to our method is a highlight-aware (HA) convolution operation and a two-stream neural network equipped with proper training losses. HA convolution helps to reduce the impact of strong specular highlights on diffuse components and at the same time, hallucinates plausible contents in saturated regions. Our method is effective to recover clear SVBRDFs from a single casually captured image. Since we impose very few constraints on the capture process, even a non-expert user can create high-quality SVBRDFs that cater to many graphical applications. In this talk, we will introduce the structure of our neural network and demonstrate through quantitative analysis and qualitative visualization that the proposed method is effective to recover clear SVBRDFs from a single casually captured image, and performs favorably against state-of-the-arts.
Shuichang Lai is a postgraduate student at Nanjing University. He received his bachelor degree from School of Computer Science and Engineering, Nanjing University of Science and Technology in 2020. His research interests include: physically-based rendering and material estimation.
GAMES主页的“使用教程”中有 “如何观看GAMES Webinar直播？”及“如何加入GAMES微信群？”的信息；