Introduction to graphical models

Francis Bach - Guillaume Obozinski
INRIA/ENS - Ecole des Ponts et Chaussees

Master recherche specialite "Mathematiques Appliquees",
Parcours M2 Mathematiques, Vision et Apprentissage (ENS Cachan), 1er semestre, 2013/2014


Internship proposals
Large-scale convex optimization for structured prediction, INRIA-ENS


This year, the class will be taught in English. All notes and assignment will be given in English. In class or for all assignments, students may use either French or English. 

Classes will take place on Wednesdays from 9am to 12pm at ENS Cachan, in Amphi Tocqueville, except for two classes (one on Friday 11/10 (1.30pm-4.30pm) in Amphi Tocqueville and one on Wednesday 16/10 in Amphi Marie-Curie).


Dates of classes

Note that some class notes (in French) are available for most of the classes from earlier years.
Date Lecturer Topics Corresponding chapters in class notes
 Scribe notes
October, 9
Tocqueville
 Guillaume Obozinski Introduction
Maximum likelihood
Models with one node		 5
Slides
 Huu Dien Khue Le, Robin Benesse
lecture1.pdf
lecture1.zip
October, 11
Friday
Tocqueville
1.30pm - 4.30pm		 Guillaume Obozinski Linear regression
Logistic regression
Generative classification		 6, 7
 Aymeric Reshef, Claire Vernade
lecture2.pdf
lecture2.zip
October, 16
Curie		 Francis Bach K-means
EM
Gaussian mixtures
Graph theory		 10, 11
 Marie d’Autume, Jean-Baptiste Alayrac
lecture3.pdf
lecture3.zip
October, 23
Tocqueville		 Francis Bach Directed graphical models
undirected graphical models		 2
 Vincent Bodin, Thomas Moreau
lecture4.pdf
lecture4.zip
October, 30
Tocqueville		 Guillaume Obozinski Exponential families
Information theory
Mixture of experts		 8, 10, 19
 Thomas Belhalfaoui, Lénaïc Chizat
lecture5.pdf
lecture5.zip
November, 6
Tocqueville		 Guillaume Obozinski Gaussian variables
Factorial analysis		 13, 14
November, 13
Tocqueville		 Guillaume Obozinski Sum-produc algorithm
HMM		 4, 12
November, 20
Tocqueville		 Francis Bach Inexact inference 21
November, 27
Tocqueville		 Francis Bach Bayesian methods




Homeworks

For October 23rd, 2013. [pdf]. Data: [classificationA.train] [classificationA.test] [classificationB.train] [classificationB.test] [classificationC.train] [classificationC.test] [Solution] [Code]

 For November 6th 2013: [pdf], Data: [EMGaussian.data] [EMGaussian.test] [Solution] [Code]

For December 18th 2013: [pdf]. This homework has to be done alone.

Projects

Project proposals

The final project allows a further understanding of certain aspects of the course. The following schedule has to be respected.
Mid-november Choose a project (one or two students per projects)
Before 29/11 Send an email to instructors detailing the choice of project.
Before 04/12 Send a draft (1 page) + first results.
Before 18/12 Submit your exam (secretariat or email).
18/12 Poster session in Batiment Cournot (C102-103) - 9am to 12pm
Before 10/01 Submit your project report (~6 pages)

Description

This course provides a unifying introduction to probabilistic modelling through the framework of graphical models, together with their associated  learning and inference algorithms.

References - Class notes

The course will be based on the book in preparation of Michael Jordan (UC Berkeley) and on research articles.  The course notes ("polycopie") may be obtained from the Mastere's adminstrative assistant.