Summary
Statistical machine learning is a growing discipline at the intersection
of computer science and applied mathematics (probability / statistics,
optimization, etc.) and which increasingly plays an important role in
technological innovation.
Unlike a course on traditional statistics, statistical machine learning is
particularly focused on the analysis of data in high dimension, as well as
the efficiency of algorithms to process the large amount of data
encountered in multiple application areas such as image or sound analysis,
natural language processing, bioinformatics or finance.
The objective of this class is to present the main theories and algorithms
in statistical machine learning. The methods covered will rely amongst
others on convex analysis arguments. The practical sessions (more than
half of which will be realized with computers) will lead to simple
implementations of the algorithms seen in class and with applications to
various domains such as computer vision or natural language processing.
Prerequisite: probability theory (notion of random variables,
convergence of random variables, conditional expectation), coding skills
in python.
General information
This class is part of the Computer science courses taught at ENS in L3 in
Spring 2024.
Teachers:
Alessandro Rudi
and
Umut Simsekli.
Practical sessions:
Benjamin Dupuis.
The class will last 52 hours (30 hours of class + 22 hours of practical
sessions) and
can
be validated for 9 ECTS.
Final grade: approximately 50% final exam, 50% homework.
Previous years:
Spring
2023,
Spring
2022,
Spring
2021,
Spring
2020,
Spring
2019,
Fall 2018,
2017,
2016,
2015,
2014,
2013,
2012
Schedule and lecture notes
Thursday mornings from 8h30 to 12h15, room Emmy Noether. Typical
session will be a lecture from 8h30 to 10h20, followed by a 20min break
and the practical work (PW) from 10h40 to 12h15.
Lecture notes and solutions to practical work and exercises will be
updated here on the fly.
# 
Date 
Teacher 
Title 
1 
08/02/2024 
U. Simsekli 
Introduction

2 
15/02/2024 
Alessandro Rudi
Benjamin Dupuis 
Supervised
learning and linear regression
TD1
(Data: classificationA_train,
classificationA_test,
classificationB_train,
classificationB_test,
classificationC_train,
classificationC_test,
mnist_digits.mat,
solution,
NEW: all in one zip: ALL)

3 
29/02/2024 
Alessandro Rudi
Benjamin Dupuis 
Logistic
regression and convex analysis Convex
optimization
TD4,
TD4englishversion,
solution
to theoretical questions, solution
to practical questions,

4 
07/03/2024 
Alessandro Rudi
Benjamin Dupuis 
Kernels
Exercise
sheet, solution 
5 
14/03/2024 
Alessandro Rudi
Benjamin Dupuis 
Learning with Kernels
Numerical
tour of Ridge and Lasso by Gabriel Peyre 
6 
21/03/2024 
Alessandro Rudi
Benjamin Dupuis

Elements of Statistical Machine Learning
Numerical
tour of logistic classification by Gabriel Peyre ,
solution and slight nodification ,
data
for first part

7 
28/03/2024 
Umut Simsekli
Benjamin Dupuis 
Model Based ML  Maximum Likelihood
TD
on sgd ,
data
,
solution to the TD on SGD ,

8 
11/04/2024 
Umut Simsekli
Benjamin Dupuis 
Unsupervised Learning
TD
on KNN
,
Small
recap on KNN
,
data
,
TD
on PCA ,
solution to the TD on KNN

9 
02/05/2024 
Umut Simsekli
Benjamin Dupuis 
MCMC Sampling, lecture notes
TD
on MCMC
,
Solution
to the TD
on MCMC

10 
23/05/2024 
Umut Simsekli
Benjamin Dupuis 
Neural Networks
TD
on Neural Networks

11 
30/05/2024 
Benjamin Dupuis

Exam
8h30 to 12h30, the same room. No notes will be allowed. 