GAMES Webinar 2019 – 97期(SIGGRAPH 2019 报告) | 张浩(清华大学),董思言(山东大学)

【GAMES Webinar 2019-97期】


报告时间:2019年6月6日 晚8:00-8:45(北京时间)
报告题目:InteractionFusion: Real-time Reconstruction of Hand Poses and Deformable Objects in Hand-object Interactions


Hand-object interaction is challenge to be reconstructed but important for many applications like HCI, robotics and so on. Previous works focus on either the hand or the object while we jointly track the hand poses, fuse the 3D object model and reconstruct its rigid and nonrigid motions, and perform all these tasks in real time. To achieve this, we first use a DNN to segment the hand and object in the two input depth streams and predict the current hand pose based on the previous poses by a pre-trained LSTM network. With this information, a unified optimization framework is proposed to jointly track the hand poses and object motions. The optimization integrates the segmented depth maps, the predicted motion, a spatial-temporal varying rigidity regularizer and a real-time contact constraint. A nonrigid fusion technique is further involved to reconstruct the object model. Experiments demonstrate that our method can solve the ambiguity caused by heavy occlusions between hand and object, and generate accurate results for various objects and interacting motions.


Hao Zhang received the BS and ME degrees in optical engineering from Beihang University,Beijing, China, in 2013 and 2016, respectively. He is currently working toward the PhD degree in the School of Software of Tsinghua University. His research interests include dynamic reconstruction and motion analysis.





报告时间:2019年6月6日 晚8:45-9:30(北京时间)
报告题目:Multi-Robot Collaborative Dense Scene Reconstruction


We present an autonomous scanning approach which allows multiple robots to perform collaborative scanning for dense 3D reconstruction of unknown indoor scenes. Our method plans scanning paths for several robots, allowing them to efficiently coordinate with each other such that the collective scanning coverage and reconstruction quality is maximized while the overall scanning effort is minimized. To this end, we define the problem as a dynamic task assignment and introduce a novel formulation based on Optimal Mass Transport (OMT). Given the currently scanned scene, a set of task views are extracted to cover scene regions which are either unknown or uncertain. These task views are assigned to the robots based on the OMT optimization. We then compute for each robot a smooth path over its assigned tasks by solving an approximate traveling salesman problem. In order to showcase our algorithm, we implement a multi-robot auto-scanning system. Since our method is computationally efficient, we can easily run it in real time on commodity hardware, and combine it with online RGB-D reconstruction approaches. In our results, we show several real-world examples of large indoor environments; in addition, we build a benchmark with a series of carefully designed metrics for quantitatively evaluating multi-robot autoscanning. Overall, we are able to demonstrate high-quality scanning results with respect to reconstruction quality and scanning efficiency, which significantly outperforms existing multi-robot exploration systems.


I’m a PhD student of Interdisciplinary Research Center, Shandong University, supervised by Prof. Baoquan Chen. My research interests are Computer Graphics, Computer Vision and Robotics, with particular interests in 3D Reconstruction and SLAM. My PhD study started since September 2017. Currently, I’m in Visual Computing and Learning Lab, Center on Frontiers of Computing Studies, Peking University. I’m also an intern in AICFVE, Beijing Film Academy.



GAMES主页的“使用教程”中有 “如何观看GAMES Webinar直播?”及“如何加入GAMES微信群?”的信息;





Liu, Ligang

刘利刚,中国科学技术大学教授、博导。分别于1996年及2001年于浙江大学获得应用数学学士及博士学位。从事计算机图形学研究,已在该领域顶级期刊ACM Transactions on Graphics上发表论文三十余篇。曾于微软亚洲研究院、浙江大学、哈佛大学工作或访问。曾获微软青年教授奖、陆增镛CAD&CG高科技奖一等奖、国家自然科学奖二等奖等奖项。任多个国内外会议的大会共同主席或论文共同主席及多个国际学术期刊编委。负责创建了中科大《计算机图形学前沿》暑期课程及CCF CAD&CG专委图形学在线交流平台GAMES。

You may also like...