Pose Estimation and Segmentation of People in 3D Movies

People

Abstract

We seek to obtain a pixel-wise segmentation and pose estimation of multiple people in a stereoscopic video. This involves challenges such as dealing with unconstrained stereoscopic video, non-stationary cameras, and complex indoor and outdoor dynamic scenes. The contributions of our work are two-fold: First, we develop a segmentation model incorporating person detection, pose estimation, as well as colour, motion, and disparity cues. Our new model explicitly represents depth ordering and occlusion. Second, we introduce a stereoscopic dataset with frames extracted from feature-length movies ``StreetDance 3D" and ``Pina". The dataset contains 2727 realistic stereo pairs and includes annotation of human poses, person bounding boxes, and pixel-wise segmentations for hundreds of people. The dataset is composed of indoor and outdoor scenes depicting multiple people with frequent occlusions. We demonstrate results on our new challenging dataset, as well as on the H2view dataset from (Sheasby et al. ACCV 2012).

Paper

ICCV 2013 Paper / PAMI 2015 Paper / Poster

BibTeX

@InProceedings{Alahari13,
    author = "Alahari, K. and Seguin, G. and Sivic, J. and Laptev, I.",
    title = "Pose Estimation and Segmentation of People in 3D Movies",
    booktitle= "Proc. IEEE International Conference on Computer Vision",
    year = "2013"
}
@InProceedings{Seguin15,
    author = "Seguin, G. and Alahari, K. and Sivic, J. and Laptev, I.",
    title = "Pose Estimation and Segmentation of People in 3D Movies",
    journal={Pattern Analysis and Machine Intelligence, IEEE Transactions on},
    volume={37},
    number={8},
    pages={1643--1655},
    year={2015}
}

Dataset

The Inria 3DMovie Dataset contains all the stereo pairs and their annotations used in our ICCV 2013 paper. All the archives unpack themselves into a single inria_stereo_dataset folder.

Code

Extended results

Acknowledgements

This work is partly supported by the Quaero Programme, funded by OSEO, the MSR-INRIA laboratory, ERC grants Activia and LEAP, Google and the EIT ICT Labs.

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright.