Announcing the Final Examination of Ce Zheng for the degree of Doctor of Philosophy
It is crucial to understand humans in visual content for various computer vision applications. Human pose estimation (HPE) has been extensively studied to accurately locate joints and construct body representations from images and videos, facilitating advancements in human-computer interaction, action recognition, and motion analysis. Building upon HPE, human mesh recovery (HMR) takes on the more complex task of estimating the 3D pose and shape of the entire human body. HMR has emerged as a critical area of research with applications in digital human avatar modeling, AI coaching, virtual reality, and more. However, HPE and HMR present significant challenges including intricate body articulation, occlusion, depth ambiguity, and the limited availability of annotated 3D data. Despite notable progress, the research community continues to tackle these obstacles, striving to achieve robust, accurate, and efficient solutions in HPE and HMR, moving closer to the ultimate objectives in the field. This dissertation addresses several challenges in the field of HPE and HMR. We begin by focusing on video-based HPE, leveraging the transformer architecture to capture both spatial relationships between body joints and temporal correlations across frames. This approach allows us to effectively utilize the comprehensive connectivity and expressive power of transformers, enhancing the accuracy of pose estimation in video sequences. Building upon this, we then tackle the issue of heavy computational and memory burden in image-based HMR. Our proposed methods demonstrate superior performance while significantly reducing computational and memory requirements compared to existing state-of-the-art techniques. Furthermore, we extend our work to video-based HMR, a diffusion-based framework is proposed for reconstructing high-quality human mesh outputs given input video sequences. These advancements provide practical and efficient solutions that align with the demands of real-world applications in HPE and HMR.
Committee in Charge:
Chen Chen, Chair, Computer Science
Mubarak Shah, Computer Science
Qian Lou, Computer Science
Hao Zheng, Electrical and Computer Engineering