Introduction to Machine Learning (2021 - 2022)


This is the old website. The website for the current year is here.


Statistical machine learning is a growing discipline at the intersection of computer science and applied mathematics (probability / statistics, optimization, etc.) and which increasingly plays an important role in technological innovation.

Unlike a course on traditional statistics, statistical machine learning is particularly focused on the analysis of data in high dimension, as well as the efficiency of algorithms to process the large amount of data encountered in multiple application areas such as image or sound analysis, natural language processing, bioinformatics or finance.

The objective of this class is to present the main theories and algorithms in statistical machine learning. The methods covered will rely amongst others on convex analysis arguments. The practical sessions (more than half of which will be realized with computers) will lead to simple implementations of the algorithms seen in class and with applications to various domains such as computer vision or natural language processing.

Prerequisite: probability theory (notion of random variables, convergence of random variables, conditional expectation), coding skills in python.

General information

This class is part of the Computer science courses taught at ENS in L3 in Spring 2021-2022.

Teachers: Alessandro Rudi and Umut Simsekli.
Practical sessions: Ulysse Marteau-Ferey.

The class will last 52 hours (30 hours of class + 22 hours of practical sessions) and can be validated for 12 ECTS.
Final grade: approximately 50% final exam, 50% homework.

Previous years: Spring 2021, Spring 2020, Spring 2019, Fall 2018, 2017, 2016, 2015, 2014, 2013, 2012

Schedule and lecture notes

Thursday mornings from 8h30 to 12h15, room H. Cartan. Typical session will be a lecture from 8h30 to 10h20, followed by a 20min break and the practical work (PW) from 10h40 to 12h15. Lecture notes and solutions to practical work and exercises will be updated here on the fly.
Home assignment 1: (Download here). It is due by May 1st, 2022. It is to be returned by email to ulysse[dot]marteau-ferey[at]inria[dot]fr.
Home assignment 2: (Download here). It is due by May 29th, 2022. It is to be returned by email to ulysse[dot]marteau-ferey[at]inria[dot]fr.

2nd of June 2022, 8h30 to 12h30, the room H. Cartan. You can bring your notes.

# Date Teacher Title
1 10/02/2022 U. Simsekli Introduction
2 17/02/2022 Alessandro Rudi
Ulysse Marteau-Ferey
Supervised learning and linear regression
TD1 (Data: classificationA_train, classificationA_test, classificationB_train, classificationB_test, classificationC_train, classificationC_test, mnist_digits.mat, solution, NEW: all in one zip: ALL)
3 24/02/2022 Alessandro Rudi
Ulysse Marteau-Ferey
Logistic regression and convex analysis

No Class
4 10/03/2022 Alessandro Rudi
Ulysse Marteau-Ferey
Convex optimization
TD4, TD4-english-version, solution to theoretical questions, solution to practical questions
5 17/03/2022 Alessandro Rudi
Ulysse Marteau-Ferey
Exercise sheet, solution
6 24/03/2022 Alessandro Rudi
Ulysse Marteau-Ferey
Learning with Kernels
Numerical tour of Ridge and Lasso by Gabriel Peyre
7 31/03/2022 Alessandro Rudi
Ulysse Marteau-Ferey
Elements of Statistical Machine Learning
Numerical tour of logistic classification by Gabriel Peyre , solution and slight nodification , data for first part
8 07/04/2022 Umut Simsekli
Ulysse Marteau-Ferey
Model Based ML - Maximum Likelihood
TD on sgd , data , solution to the TD on SGD ,
9 14/04/2022 Umut Simsekli
Ulysse Marteau-Ferey
Unsupervised Learning
TD on KNN , Small recap on KNN , data , TD on PCA , solution to the TD on KNN
10 21/04/2022 Umut Simsekli
Ulysse Marteau-Ferey
MCMC Sampling, lecture notes
TD on MCMC , Solution to the TD on MCMC

No Class

No Class
11 12/05/2022 Umut Simsekli
Ulysse Marteau-Ferey
Neural Networks
TD on Neural Networks
12 19/05/2022 Umut Simsekli
Ulysse Marteau-Ferey

No Class
13 02/06/2022 Alessandro Rudi
Umut Simsekli
8h30 to 12h30, the room H. Cartan. You can bring your notes.