GAMES Webinar 2019 – 97期(SIGGRAPH 2019 报告) | 张浩(清华大学),董思言(山东大学)

【GAMES Webinar 2019-97期】


报告时间:2019年6月6日 晚8:00-8:45(北京时间)
报告题目:InteractionFusion: Real-time Reconstruction of Hand Poses and Deformable Objects in Hand-object Interactions


Hand-object interaction is challenge to be reconstructed but important for many applications like HCI, robotics and so on. Previous works focus on either the hand or the object while we jointly track the hand poses, fuse the 3D object model and reconstruct its rigid and nonrigid motions, and perform all these tasks in real time. To achieve this, we first use a DNN to segment the hand and object in the two input depth streams and predict the current hand pose based on the previous poses by a pre-trained LSTM network. With this information, a unified optimization framework is proposed to jointly track the hand poses and object motions. The optimization integrates the segmented depth maps, the predicted motion, a spatial-temporal varying rigidity regularizer and a real-time contact constraint. A nonrigid fusion technique is further involved to reconstruct the object model. Experiments demonstrate that our method can solve the ambiguity caused by heavy occlusions between hand and object, and generate accurate results for various objects and interacting motions.


Hao Zhang received the BS and ME degrees in optical engineering from Beihang University,Beijing, China, in 2013 and 2016, respectively. He is currently working toward the PhD degree in the School of Software of Tsinghua University. His research interests include dynamic reconstruction and motion analysis.





报告时间:2019年6月6日 晚8:45-9:30(北京时间)
报告题目:Multi-Robot Collaborative Dense Scene Reconstruction


We present an autonomous scanning approach which allows multiple robots to perform collaborative scanning for dense 3D reconstruction of unknown indoor scenes. Our method plans scanning paths for several robots, allowing them to efficiently coordinate with each other such that the collective scanning coverage and reconstruction quality is maximized while the overall scanning effort is minimized. To this end, we define the problem as a dynamic task assignment and introduce a novel formulation based on Optimal Mass Transport (OMT). Given the currently scanned scene, a set of task views are extracted to cover scene regions which are either unknown or uncertain. These task views are assigned to the robots based on the OMT optimization. We then compute for each robot a smooth path over its assigned tasks by solving an approximate traveling salesman problem. In order to showcase our algorithm, we implement a multi-robot auto-scanning system. Since our method is computationally efficient, we can easily run it in real time on commodity hardware, and combine it with online RGB-D reconstruction approaches. In our results, we show several real-world examples of large indoor environments; in addition, we build a benchmark with a series of carefully designed metrics for quantitatively evaluating multi-robot autoscanning. Overall, we are able to demonstrate high-quality scanning results with respect to reconstruction quality and scanning efficiency, which significantly outperforms existing multi-robot exploration systems.


I’m a PhD student of Interdisciplinary Research Center, Shandong University, supervised by Prof. Baoquan Chen. My research interests are Computer Graphics, Computer Vision and Robotics, with particular interests in 3D Reconstruction and SLAM. My PhD study started since September 2017. Currently, I’m in Visual Computing and Learning Lab, Center on Frontiers of Computing Studies, Peking University. I’m also an intern in AICFVE, Beijing Film Academy.



GAMES主页的“使用教程”中有 “如何观看GAMES Webinar直播?”及“如何加入GAMES微信群?”的信息;





You may also like...