Josef Sivic

Senior researcher (Directeur de Recherche)

INRIA - Willow Project

Département d'Informatique de l'Ecole Normale Supérieure

Centre de Recherche INRIA de Paris

Office: Building C, 4th floor, office 416 (Directions)

E-mail: Josef dot Sivic at ens dot fr , Tel: +33.(0)1 8049 4422

Senior researcher and principal investigator

Intelligent Machine Perception project

Czech Institute of Informatics, Robotics, and Cybernetics
Czech Technical University in Prague

Office: Building B, 6th floor, office B-639 ( Directions )


I am currently on leave (“détachement”) to the Czech Technical University in Prague but keep close ties to the Willow team where I remain a close external collaborator.  For latest updates please visit my webpage in Prague.

I am senior researcher (roughly equivalent to associate professor with tenure) at INRIA working in the Willow project at the Department of Computer Science at Ecole Normale Supérieure . I also lead the Intelligent Machine Perception project at the Institute of Informatics, Robotics and Cybernetics at the Czech Technical University in Prague. I completed my PhD at the University of Oxford working with Professor Andrew Zisserman and received my habilitation degree at Ecole Normale Supérieure in Paris. Before coming to France I spent six months as a post-doc in the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT working with Prof. William Freeman. My research is in computer vision - a branch of computer science and engineering that aims to extract information from images. I am particularly interested in developing learnable image representations for automatic visual recognition and scene understanding. I lead the ERC project LEAP .


News

Hiring Postdocs/Phds in Deep Learning, Vision and Robotics at the Intelligent Machine Perception group in Prague. Email me for more details.

Antoine Miech (co-advised with I. Laptev) has won the Google Fellowship . An interview with Antoine is here .

Two papers on visual localization have been accepted for spotlight presentation at CVPR 2018. Pre-prints are available online here and here .

Antoine Miech (co-advised with I. Laptev) has won the Google Cloud & YouTube-8M Video Understanding Challenge . The workshop paper describing the winning entry is now online . Bravo Antoine!

Our work on weakly-supervised learning of visual relations has been accepted for oral presentation at ICCV 2017. The pre-print is online .

Our work on learning geometric matching has been accepted for spotlight presentation at CVPR 2017. The pre-print is online .

Our work on weakly supervised place recognition has been accepted for oral presentation at CVPR 2016. The pre-print is online .

Our work on learning from instruction videos has been accepted for oral presentation at CVPR 2016. The pre-print is online .

chair_small.jpeg

Our work on weakly supervised learning with convolutional neural networks has been accepted to CVPR 2015. The pre-print is online .

Our work on place recognition across large changes in illumination has been accepted to CVPR 2015. The pre-print is online .

Our work on aligning paintings to 3D models has been published in ACM Transactions on Graphics. The pre-print is available online .

Our work on transferring mid-level image representations using convolutional neural networks has been accepted for oral presentation at CVPR 2014. The paper is available online .

chairs4.jpg

Our work on 2D-3D alignment for 3D object category recognition has been accepted for oral presentation at CVPR 2014 (The paper is available online ) .

I have been awarded an ERC Starting Grant 2013 .

IJCV special issues on Visual Media Geo-Localization and Video Recognition that I have co-edited have been published.


Publications



Teaching

ENS/INRIA Visual Recognition and Machine Learning Summer School, Paris, 22-26 July 2013
INRIA Visual Rec ogni tion and Machine Learning Summer School, Grenoble, 9-13 July 2012
ENS/INRIA Visual Recognition and Machine Learning Summer School, Paris, 25-29 July 2011
Computer Vision and Machine Learning Winter School, ENS Lyon, 24-28 January 2011
INRIA Visual Recognition and Machine Learning Summer School, Grenoble, 26-30 July 2010

Ecole Normale Supérieure de Paris / Ecole Normale Supérieure de Cachan

  • Object Recognition and Computer Vision, Fall 2016
  • Object Recognition and Computer Vision, Fall 2015
  • Object Recognition and Computer Vision, Fall 2014
  • Object Recognition and Computer Vision, Fall 2013
  • Object Recognition and Computer Vision, Fall 201 2
  • Object Recognition and Computer Vision, Fall 2011
  • Object Recognition and Computer Vision, Fall 2010
  • Object Recognition and Computer Vision, Fall 2009
  • Object Recognition and Computer Vision, Fall 2008

(co-lecturing with J. Ponce, C. Schmid and I. Laptev)



Students and Collaborators

Post-docs:

Phd students:

  • Zongmian Li (co-advised with I. Laptev and N. Mansard)
  • Dmitry Zhukov (co-advised with I. Laptev)
  • Ignacio Rocco (co-advised with R. Arandjelovic)
  • Antoine Miech (co-advised with I. Laptev)
  • Julia Peyre (co-advised with I. Laptev and C. Schmid)
  • Theophile Dalens (co-advised with M. Aubry)
  • Jean-Baptiste Alayrac (co-advised with S. Lacoste-Julien and I. Laptev)

Graduated:

M.Sc. students:

  • Zongmian Li, 2017 (co-advised with I. Laptev and N. Mansard)
  • Dmitry Zhukov, 2017 (co-advised with I. Laptev and J.-P. Alayrac)
  • Ignacio Rocco, 2016 (co-advised with R. Arandjelovic)
  • Antoine Miech, 2016 (co-advised with I. Laptev and P. Bojanowski)
  • Julia Peyre, 2015 (co-advised with I. Laptev and C. Schmid)
  • Theophile Dalens, 2014 (co-advised with M. Aubry)
  • Maxime Oquab, 2013 (co-advised with L. Bottou and I. Laptev)
  • Tuang-Hung Vu, 2013 (co-advised with I. Laptev and A. Oliva)
  • Piotr Bojanowski, 2012 (co-advised with I. Laptev and C. Schmid)
  • Michael Mathieu, 2011 (co-advised with I. Laptev)
  • Guillaume Seguin, 2011 (co-advised with I. Laptev)
  • Ugo Jardonnet, 2010 (co-advised with T. Cour and I. Laptev)
  • Jose Lezama, 2010 (co-advised with I. Laptev)
  • Vincent Delaitre, 2010 (co-advised with I. Laptev)
  • Warith Harchaoui, 2009 (co-advised with I. Laptev)
  • Mathieu Prevot, 2008 (co-advised with B. Russell)
  • Oliver Whyte, 2008 (co-advised with A. Zisserman)
  • Jan Knopp, visiting student, 2008 (co-advised with T. Pajdla)

Former collaborators:


Professional activities


Demos

Object Retrieval in Videos

Person Retrieval in Videos

Web-scale particular object and building search

Efficient Deblurring for Shaken and Partially Saturated Images

Software


Selected Projects

Weakly-supervised learning of visual relations

J. Peyre., I. Laptev, C. Schmid, and J. Sivic

In ICCV 2017 | PDF | Project page

Weakly-supervised learning of visual relations

J. Peyre., I. Laptev, C. Schmid, and J. Sivic

In ICCV 2017 | PDF | Project page

Convolutional neural network architecture for geometric matching

I. Rocco, R. Arandjelovic and  J. Sivic

In CVPR 2017 | PDF | Project page

Unsupervised learning from narrated instruction videos

J.-B. Alayrac, P. Bojanowski, N. Agrawal, I. Laptev, J. Sivic and S. Lacoste-Julien

In CVPR 2016 | PDF | Project page

NetVLAD: CNN architecture for weakly supervised place recognition

R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla and J. Sivic

In CVPR 2016 | PDF | Project page

Is object localization for free? – Weakly-supervised learning with convolutional neural networks

M. Oquab, L. Bottou, I. Laptev, J. Sivic

In CVPR 2015 | PDF | Project page

Linking Past to Present: Discovering Style in Two Centuries of Architecture

Lee, S., Maisonneuve, N., Crandall, D., Efros, A. and Sivic, J.

In ICCP 2015 | PDF | Project page

24/7 place recognition by view synthesis

Torii, A., Arandjelovic, R., Sivic, J., Pajdla, T. and Okutomi, M.

In CVPR 2015 | PDF | Project page

Seeing 3D chairs: exemplar part-based 2D-3D alignment using a large dataset of CAD models

Aubry, M., Maturana, D., Efros, A., Russell, B. and Sivic, J.

In CVPR 2014 | PDF | Project page

Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks

M. Oquab, L. Bottou, I. Laptev, J. Sivic

In CVPR 2014 | PDF | Project page

Painting-to-3D Model Alignment Via Discriminative Visual Elements

M. Aubry, B. Russell and J. Sivic

In ACM Transactions on Graphics (TOG), 2013

PDF | Project page

Pose estimation and segmentation of people in 3D movies

Alahari, K., Seguin, G., Sivic, J. and Laptev, I.

In ICCV 2013 || PDF | Project page

Finding actors and actions in movies

Bojanowski, P., Laptev, I., Ponce, J., Schmid, C. and Sivic, J.

In ICCV 2013 | PDF | Project page

Visual place recognition with repetitive structures

Torii, A., Sivic, J., Pajdla, T. and Okutomi, M.

In CVPR 2013 | PDF | Project page

Learning per-location classifiers for visual place recognition

Gronat, P., Obozinski, G., Sivic, J. and Pajdla, T.

In CVPR 2013 | PDF | Project page

Scene semantics from long-term observation of people

V. Delaitre, D. Fouhey, I. Laptev, J. Sivic, A. Gupta and A. Efros

In ECCV 2012 | PDF | Project page

People watching: human actions as a cue for single-view geometry

D. Fouhey, V. Delaitre, A. Gupta, A. Efros, I. Laptev and J. Sivic

In ECCV 2012 | PDF | Project page

What makes Paris look like Paris?

Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic and Alexei Efros

In SIGGRAPH 2012 | PDF | Project page

Data-driven crowd analysis in videos

Mikel Rodriguez, Josef Sivic, Ivan Laptev and Jean-Yves Audibert

In ICCV 2011 | PDF | Project page

Density-aware person detection and tracking in crowds .

Mikel Rodriguez, Ivan Laptev, Josef Sivic and Jean-Yves Audibert

In ICCV 2011 | PDF | Project page

Learning person-object interactions for action recognition in still images

Vincent Delaitre, Josef Sivic and Ivan Laptev

In NIPS 2011 | PDF | Project page

Automatic alignment of paintings and photographs depicting a 3D scene

Bryan Russell, Josef Sivic, Jean Ponce and Helene Dessales

In 3rd International IEEE Workshop on 3D Representation for Recognition (2011) | PDF | Project page

Visual localization by linear combination of image descriptors

Akihiko Torii, Josef Sivic and Tomas Pajdla

In 2nd IEEE Workshop on Mobile Vision (2011) | PDF

Deblurring shaken and partially saturated images

Oliver Whyte, Josef Sivic and Andrew Zisserman

IEEE Workshop on Color and Photometry in Computer Vision (2011) PDF | Project page

Track to the future: Spatio-temporal video segmentation with long-range motion cues

Jose Lezama, Karteek Alahari, Josef Sivic and Ivan Laptev

In CVPR 2011| PDF | Project page

Avoiding confusing features in place recognition

Jan Knopp, Josef Sivic, and Tomas Pajdla

In ECCV 2010 | PDF | Project page

Descriptor learning for efficient retrieval

James Philbin, Michael Isard, Josef Sivic, and Andrew Zisserman

In ECCV 2010 | PDF

Non-uniform deblurring for shaken images

Oliver Whyte, Josef Sivic, Andrew Zisserman and Jean Ponce

In CVPR 2010 | PDF | Project page

Recognizing human actions in still images: a study of bag-of-features and part-based representations

Vincent Delaitre, Ivan Laptev and Josef Sivic

In BMVC 2010 | PDF

Get out of my picture! Internet-based inpainting

Oliver Whyte, Josef Sivic and Andrew Zisserman

In BMVC 2009 | PDF | Project page

Segmenting Scenes by Matching Image Composites

Bryan Russell, Alexei A. Efros, Josef Sivic, Bill Freeman and Andrew Zisserman

In NIPS 2009 | PDF | Bibtex | Project page

Automatic annotation of human actions in video

Olivier Duchenne, Ivan Laptev, Josef Sivic, Francis Bach and Jean Ponce

In ICCV 2009 | PDF | Bibtex | AVI

"Who are you?'': Learning person specific classifiers from video

Josef Sivic, Mark Everingham and Andrew Zisserman

In CVPR 2009 | PDF | Bibtex | AVI

SIFT Flow: Dense Correspondence across Different Scenes

Ce Liu, Jenny Yuen, Antonio Torralba, Josef Sivic and William T. Freeman

In ECCV 2008 | PDF | Bibtex | Project page

Creating and Exploring a Large Photorealistic Virtual Space

Josef Sivic, Biliana Kaneva, Antonio Torralba, Shai Avidan and William T. Freeman

In  IEEE Workshop on Internet Vision, at CVPR’08 | PDF | Bibtex | Supplementary Video

Unsupervised Discovery of Visual Object Class Hierarchies

Josef Sivic, Bryan Russell, Andrew Zisserman, William T. Freeman and Alexei A. Efros

In CVPR 2008 | PDF | Bibtex

Geometric LDA: A Generative Model for Particular Object Discovery

James Philbin, Josef Sivic, Andrew Zisserman

In BMVC 2008 | PDF | Bibtex | Extended journal version

Lost in Quantization: Improving Particular Object Retrieval in Large Scale Image Databases

James Philbin, Ondrej Chum, Michael Isard, Josef  Sivic, Andrew Zisserman

In CVPR 2008 | PDF | Bibtex

Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval

Ondrej Chum, James Philbin, Josef Sivic, Michael Isard, Andrew Zisserman

In ICCV 2007 | PDF | Bibtex | Project page and Demo

Object retrieval with large vocabularies and fast spatial matching

James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, Andrew Zisserman

In CVPR 2007 | PDF | Bibtex | Project page and Demo

Using Multiple Segmentations to Discover Objects and their Extent in Image Collections

Bryan C. Russell, Alexei A. Efros, Josef Sivic, William T. Freeman, Andrew Zisserman

In CVPR 2006 | PDF | Bibtex | Project page and source code

Hello! My name is... Buffy -- Automatic Naming of Characters in TV Video

Mark Everingham, Josef  Sivic, Andrew Zisserman

In BMVC 2006 | PDF | Bibtex | Project page | Extended Journal Version

Winner of best industrial paper prize

Finding people in repeated shots of the same scene

Josef Sivic, C. Lawrence Zitnick, Richard Szeliski

In BMVC 2006  | PDF | Bibtex | Extended Technical Report

Person spotting: video shot retrieval for face sets

Josef Sivic, Mark Everingham, Andrew Zisserman

In CIVR 2005 | PDF | Bibtex | Project page and Demo

Discovering objects and their location in images

Josef Sivic, Bryan C. Russell, Alexei A. Efros, Andrew Zisserman, William T. Freeman

In ICCV 2005 | PDF | Bibtex | Source Code (pLSA)

Object Level Grouping for Video Shots

Josef Sivic, Frederick Schaffalitzky, Andrew Zisserman

In ECCV 2004 | PDF | Bibtex | Extended Journal Version

Video Data Mining Using Configurations of Viewpoint Invariant Regions

Josef Sivic, Andrew Zisserman

In CVPR 2004 | PDF | Bibtex

Video Google: A Text Retrieval Approach to Object Matching in Videos

Josef Sivic, Andrew  Zisserman

In ICCV 2003 | PDF | Bibtex | Project page and Demo