Summary
Statistical machine learning is a growing discipline at the intersection
of computer science and applied mathematics (probability / statistics,
optimization, etc.) and which increasingly plays an important role in
technological innovation.
Unlike a course on traditional statistics, statistical machine learning is
particularly focused on the analysis of data in high dimension, as well as
the efficiency of algorithms to process the large amount of data
encountered in multiple application areas such as image or sound analysis,
natural language processing, bioinformatics or finance.
The objective of this class is to present the main theories and algorithms
in statistical machine learning. The methods covered will rely amongst
others on convex analysis arguments. The practical sessions (more than
half of which will be realized with computers) will lead to simple
implementations of the algorithms seen in class and with applications to
various domains such as computer vision or natural language processing.
Prerequisite: probability theory (notion of random variables,
convergence of random variables, conditional expectation), coding skills
in python.
General information
This class is part of the Computer science courses taught at ENS in L3 in
Spring 2024.
Teachers:
Alessandro Rudi
and
Umut Simsekli.
Practical sessions:
Benjamin Dupuis.
The class will last 52 hours (30 hours of class + 22 hours of practical
sessions) and
can
be validated for 9 ECTS.
Final grade: approximately 50% final exam, 50% homework.
Previous years:
Spring
2023,
Spring
2022,
Spring
2021,
Spring
2020,
Spring
2019,
Fall 2018,
2017,
2016,
2015,
2014,
2013,
2012
Schedule and lecture notes
Thursday mornings from 8h30 to 12h15, room Emmy Noether. Typical
session will be a lecture from 8h30 to 10h20, followed by a 20min break
and the practical work (PW) from 10h40 to 12h15.
Lecture notes and solutions to practical work and exercises will be
updated here on the fly.
| # |
Date |
Teacher |
Title |
| 1 |
08/02/2024 |
U. Simsekli |
Introduction
|
| 2 |
15/02/2024 |
Alessandro Rudi
Benjamin Dupuis |
Supervised
learning and linear regression
TD1
(Data: classificationA_train,
classificationA_test,
classificationB_train,
classificationB_test,
classificationC_train,
classificationC_test,
mnist_digits.mat,
solution,
NEW: all in one zip: ALL)
|
| 3 |
29/02/2024 |
Alessandro Rudi
Benjamin Dupuis |
Logistic
regression and convex analysis Convex
optimization
TD4,
TD4-english-version,
solution
to theoretical questions, solution
to practical questions,
|
| 4 |
07/03/2024 |
Alessandro Rudi
Benjamin Dupuis |
Kernels
Exercise
sheet, solution |
| 5 |
14/03/2024 |
Alessandro Rudi
Benjamin Dupuis |
Learning with Kernels
Numerical
tour of Ridge and Lasso by Gabriel Peyre |
| 6 |
21/03/2024 |
Alessandro Rudi
Benjamin Dupuis
|
Elements of Statistical Machine Learning
Numerical
tour of logistic classification by Gabriel Peyre ,
solution and slight nodification ,
data
for first part
|
| 7 |
28/03/2024 |
Umut Simsekli
Benjamin Dupuis |
Model Based ML - Maximum Likelihood
TD
on sgd ,
data
,
solution to the TD on SGD ,
|
| 8 |
11/04/2024 |
Umut Simsekli
Benjamin Dupuis |
Unsupervised Learning
TD
on KNN
,
Small
recap on KNN
,
data
,
TD
on PCA ,
solution to the TD on KNN
|
| 9 |
02/05/2024 |
Umut Simsekli
Benjamin Dupuis |
MCMC Sampling, lecture notes
TD
on MCMC
,
Solution
to the TD
on MCMC
|
| 10 |
23/05/2024 |
Umut Simsekli
Benjamin Dupuis |
Neural Networks
TD
on Neural Networks
|
| 11 |
30/05/2024 |
Benjamin Dupuis
|
Exam
8h30 to 12h30, the same room. You can bring your notes. |