Reconnaissance d’objets et vision artificielle 2022/2023
Object recognition and computer vision 2022/2023
Ivan Laptev, Jean Ponce, Cordelia Schmid and Josef Sivic
Lecture time: Tuesday 16:15-19:15
Lecture room: Salle Dussane, ENS Ulm, 45 rue d’Ulm, Paris (4/10, 25/10, 8/11, 15/11, 22/11, 29/11, 6/12,13/12)
Salle Jaurès, ENS Ulm, 29 rue d’Ulm, Paris (11/10, 18/10)
Teaching assistants: Ricardo Garcia, Guillaume Le Moing and Antoine Yang
Automated object recognition -- and more generally scene analysis -- from photographs and videos is the grand challenge of computer vision. This course presents the image, object, and scene models, as well as the methods and algorithms, used today to address this challenge.
There will be three programming assignments representing 50% (10% + 20% + 20%) of the grade. The supporting materials for the programming assignments and final projects will be in Python and make use of Jupyter notebooks. For additional technical instructions on the assignments please follow this link.
The final project will represent 50% of the grade.
You can discuss the assignments and final projects with other students in the class. Discussions are encouraged and are an essential component of the academic environment. However, each student has to work out their assignment alone (including any coding, experiments or derivations) and submit their own report. For the final project, you may work alone or in a group of maximum of 3 people. If working in a group, we expect a more substantial project, and an equal contribution from each student in the group. The final project report needs to explicitly specify the contribution of each student. Both students are expected to present the project at the oral presentation and contribute equally to writing the report. The assignments and final projects will be checked to contain original material. Any uncredited reuse of material (text, code, results) will be considered as plagiarism and will result in zero points for the assignment / final project. If a plagiarism is detected, the student will be reported to MVA.
Computer vision and machine learning talks
You are welcome to attend seminars in the Willow group. Please see the current seminar schedule. Typically, these are one hour research talks given by visiting speakers. The talks are at 2 Rue Simone IFF. When you enter the building, tell the receptionist you are going for a seminar.
Topic and reading materials.
Class logistics: assignments, final projects, grading (I. Laptev)
Mikolajczyk & Schmid, Scale and affine invariant interest point detectors, IJCV 2004; D. Lowe, Distinctive image features from scale-invariant keypoints, IJCV 2004;
Camera geometry (J. Ponce); Instance-level recognition II. - Correspondence, image matching (I. Laptev);
History: J. Mundy - Object recognition in the geometric era: A retrospective.; Camera geometry: Forsyth&Ponce Ch.1-2. Hartley&Zisserman - Ch.6
Instance-level recognition III. - Efficient visual search (J. Sivic)
Muja & Lowe, Fast approx. nearest neighbors with automatic algorithm configuration, VISAPP'09; Sivic & Zisserman, Video Google: Efficient visual search of videos (chapter from this book), Philbin et al., Object retrieval with large vocabularies and fast spatial matching, CVPR'07; Jegou et al., Improving bag-of-features for large scale image search, IJCV 2010; Jegou et al., Aggregating local image descriptors into compact codes, PAMI 2011; Iscen et al., Efficient Diffusion on Region Manifolds, CVPR 2017; Arandjelovic et al., NetVLAD: CNN architecture for weakly-supervised place recognition, PAMI 2018.
Sparse coding and dictionary learning for image analysis (3hrs, J. Ponce)
Bach, Mairal, Ponce, Sapiro, Tutorial on sparse coding and dictionary learning for image analysis, at CVPR'10; Eboli, Sun and Ponce, End-to-end interpretable learning of non-blind image deblurring, ECCV 2022; Lecouat, Ponce and Mairal, Lucas-kanade reloaded: End-to-end super-resolution from raw image bursts. ICCV 2021.
No lecture (Toussaint)
Supervised learning and deep learning; Optimization and regularization for neural networks; Sequence models and transformers (A. Joulin)
Neural networks for visual recognition I. (G. Varol)
Basics of CNNs by A. Vedaldi:
Neural networks for visual recognition II. (I. Laptev)
CVPR’08; Pascal VOC Challenge; Girshick et al., Rich feature hierarchies for accurate object detection and semantic segmentation, CVPR 2014; Girshick, Fast R-CNN, CVPR 2015; Ren et al., Faster R-CNN: Towards real-time object detection with region proposal networks, NIPS 2015. Redmon et al., You only look once: Unified, real-time object detection, CVPR 2016; Zhou et al., Objects as points, 2019; Long et al., Fully convolutional networks for semantic segmentation, CVPR 2015; Chen et al., DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs, PAMI 2017; He et al., Mask R-CNN, ICCV 2017.
Human action recognition (C. Schmid)
Materials: Brox and Malik, Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation, PAMI 2011; Weinzaepfel et al. Deepflow: Large displacement optical flow with deep matching, CVPR 2013; Laptev et al., Learning realistic human actions from movies, CVPR 2008; Wang et al., Dense trajectories and motion boundary descriptors for action recognition, CVPR 2011; Simonyan and Zisserman, Two-stream convolutional networks for action recognition in videos, NIPS 2014; Tran et al. Learning spatiotemporal features with 3D convolutional networks, ICCV 2015.
Weakly-supervised learning; Self-supervised learning; Vision for robotics (I. Laptev)
Materials: ECCV 2016; Oquab et al., Is object localization for free? - Weakly-supervised learning with convolutional neural networks, CVPR 2015;. Alayrac et al., Unsupervised learning from narrated instruction videos, CVPR 2016; Varol et al., Learning from Synthetic Humans, CVPR 2017; Hasson et al., Learning joint reconstruction of hands and manipulated objects, CVPR 2019; Miech et al., End-to-End Learning of Visual Representations from Uncurated Instructional Videos, CVPR 2020.
Generative models (G. Varol); Deep Learning and 3D data (M. Aubry)
- GANs: I. Goodfellow et al., Generative adversarial nets, NIPS 2014; T. Karras et al., A Style-Based Generator Architecture for Generative Adversarial Networks. CVPR 2019.
- Diffusion: J. Ho et al. Denoising diffusion probabilistic models, NeurIPS 2020; R. Rombach et al. High-Resolution Image Synthesis with Latent Diffusion Models, CVPR 2022.
DL and 3D:
1. 3D analysis: Qi et al., Volumetric and multi-view cnns for object classification on 3d data, CVPR 2016; Qi et al., PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, CVPR 2017; Groueix et al. 3D-CODED: 3D correspondences by deep deformation, ECCV 2018.
2. 3D generation: Fan et al. A point set generation network for 3d object reconstruction from a single image, CVPR 2017, Groueix et al. AtlasNet: A papier-mâché approach to learning 3d surface generation, CVPR. 2018; Park et al. Deepsdf: Learning continuous signed distance functions for shape representation, CVPR 2019, Midenhall et al. , Nerf: Representing scenes as neural radiance fields for view synthesis, ECCV 2020.
3. Training with synthetic data: Tobin et al. Domain randomization for transferring deep neural networks from simulation to the real world IROS 2017; Torralba and Efros Unbiased look at dataset bias CVPR 2011; Ganin et al. Domain-adversarial training of neural networks JMLR 2016.
Final project presentations and evaluation
Jan 9: 10:30-12:00; 13:00-16:00
Jan 10: 10:30-12:00; 13:00-16:00
The presentations will be virtual. Links will be provided.
Final project reports due on 16/01
D.A. Forsyth and J. Ponce, "Computer Vision: A Modern Approach", Prentice-Hall, 2nd edition, 2011
J. Ponce, M. Hebert, C. Schmid and A. Zisserman "Toward Category-Level Object Recognition", Lecture Notes in Computer Science 4170, Springer-Verlag, 2007
O. Faugeras, Q.T. Luong, and T. Papadopoulo, "Geometry of Multiple Images", MIT Press, 2001.
R. Hartley and A. Zisserman, "Multiple View Geometry in Computer Vision", Cambridge University Press, 2004.
J. Koenderink, "Solid Shape", MIT Press, 1990
R. Szeliski, "Computer Vision: Algorithms and Applications, 2nd ed.", 2022. Online book.