In the last decades, we have witnessed advances in both hardware and associated algorithms resulting in unprecedented access to volumes of 2D and, more recently, 3D data capturing human movement. We are no longer satisfied with recovering human pose as an image-space 2D skeleton, but seek to obtain a full 3D human body representation. The main challenges in acquiring 3D human shape from such raw measurements are identifying which parts of the data relate to body measurements and recovering from partial observations, often arising out of severe occlusion. For example, a person occluded by a piece of furniture, or being self-occluded in a profile view. In this paper, we propose POP, a novel and efficient paradigm for estimation and completion of human shape to produce a full parametric 3D model directly from single RGBD images, even under severe occlusion. At the heart of our method is a novel human body pose retrieval formulation that explicitly models and handles occlusion. The retrieved result is then refined by a robust optimization to yield a full representation of the human shape. We demonstrate our method on a range of challenging real world scenarios and produce high-quality results not possible by competing alternatives. The method opens up exciting AR/VR application possibilities by working on 'in-the-wild' measurements of human motion.

Marin, R., Melzi, S., Mitra, N., Castellani, U. (2019). POP: Full parametric model estimation for occluded people. In Proceedings on 3DOR 2019 (pp.1-8). The Eurographics Association [10.2312/3dor.20191055].

POP: Full parametric model estimation for occluded people

Simone Melzi;
2019

Abstract

In the last decades, we have witnessed advances in both hardware and associated algorithms resulting in unprecedented access to volumes of 2D and, more recently, 3D data capturing human movement. We are no longer satisfied with recovering human pose as an image-space 2D skeleton, but seek to obtain a full 3D human body representation. The main challenges in acquiring 3D human shape from such raw measurements are identifying which parts of the data relate to body measurements and recovering from partial observations, often arising out of severe occlusion. For example, a person occluded by a piece of furniture, or being self-occluded in a profile view. In this paper, we propose POP, a novel and efficient paradigm for estimation and completion of human shape to produce a full parametric 3D model directly from single RGBD images, even under severe occlusion. At the heart of our method is a novel human body pose retrieval formulation that explicitly models and handles occlusion. The retrieved result is then refined by a robust optimization to yield a full representation of the human shape. We demonstrate our method on a range of challenging real world scenarios and produce high-quality results not possible by competing alternatives. The method opens up exciting AR/VR application possibilities by working on 'in-the-wild' measurements of human motion.
Si
paper
Model fitting; RGBD; Human modelling
English
12th Eurographics Workshop on 3D Object Retrieval, 3DOR 2019, in conjunction with the 40th Annual Conference of the European Association for Computer Graphics, EG 2019
978-3-03868-077-2
Marin, R., Melzi, S., Mitra, N., Castellani, U. (2019). POP: Full parametric model estimation for occluded people. In Proceedings on 3DOR 2019 (pp.1-8). The Eurographics Association [10.2312/3dor.20191055].
Marin, R; Melzi, S; Mitra, N; Castellani, U
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/350566
Citazioni
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
Social impact