GAMES Webinar 2024 – 330期(Physics-based Inverse Graphics) | 李臻(如视科技),邓画予(上海交通大学)

【GAMES Webinar 2024-330期】(VR&AR专题-Physics-based Inverse Graphics)





We present a multi-view inverse rendering method for large-scale real-world indoor scenes that reconstructs global illumination and physically-reasonable SVBRDFs. Unlike previous representations, where the global illumination of large scenes is simplified as multiple environment maps, we propose a compact representation called Texture-based Lighting (TBL). It consists of 3D meshs and HDR textures, and efficiently models direct and infinite-bounce indirect lighting of the entire large scene. Based on TBL, we further propose a hybrid lighting representation with precomputed irradiance, which significantly improves the efficiency and alleviate the rendering noise in the material optimization. To physically disentangle the ambiguity between materials, we propose a three-stage material optimization strategy based on the priors of semantic segmentation and room segmentation. Extensive experiments show that the proposed method outperforms the state-of-the-arts quantitatively and qualitatively, and enables physically-reasonable mixed-reality applications such as material editing, editable novel view synthesis and relighting.








Deep learning has great potential for modeling the dynamics of complex particle systems like fluids. Existing methods need supervision with consecutive particle properties, such as positions and velocities. We introduce NeuroFluid, a differentiable two-stage network that infers state transitions and interactions in fluid particle systems from sequential visual observations. Additionally, we present latent intuitive physics, a transfer learning framework that infers hidden properties of fluids from a single 3D video and simulates the observed fluid in new scenes. Our approach uses latent features from a learnable prior distribution based on underlying particle states to capture invisible and complex physical properties. We train a parametrized prior learner on visual observations to approximate the visual posterior of inverse graphics, using both particle states and the visual posterior from a neural renderer. This converged prior learner is embedded in our probabilistic physics engine, enabling simulations on unseen geometries, boundaries, and dynamics without knowing the true physical parameters.


邓画予,现为上海交通大学二年级博士生,导师为杨小康老师和王韫博老师,本科毕业于上海交通大学IEEE试点班(计算机方向)。主要研究方向为三维场景理解,直觉物理,AI for science科学计算等。部分成果发表于ICML/ICLR/CVPR。


张青,中山大学计算机学院副教授。主要从事计算摄像和计算机图形学等方面的研究,特别是2D/3D内容生成与编辑、逆渲染、图像增强等。在ACM TOG、TPAMI、IJCV、TVCG等期刊以及SIGGRAPH (Asia)、CVPR等会议发表论文40余篇,谷歌学术引用3300余次。主持或参与多项国家级科研项目及课题。获2019年湖北省自然科学二等奖, 2022年世界人工智能大会青年优秀论文奖, CCF CAD&CG 2021最佳论文奖。


GAMES主页的“使用教程”中有 “如何观看GAMES Webinar直播?”及“如何加入GAMES微信群?”的信息;

You may also like...