Midnovember  Choose a
project (one or two students
per projects) 
Before 29/11  Send an email to instructors detailing the choice of project. 
Before 04/12  Send a draft (1 page) + first results. 
Before 18/12  Submit
your exam (secretariat or
email). 
18/12  Poster session in Batiment
Cournot (C102103)  9am to 12pm 
Before 10/01  Submit your project report (~6 pages) 
Probabilistics PCA  Interpretation of PCA as a graphical model
close to factorial analysis. A situation where EM has no local
minima. Tipping, M. E., Bishop, C. M. 1999. Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B 61(3):611622. [pdf] 
Learning graph structure  multinomial
models 
For complete discrete data, learning of
parameters and directed acyclic graph. D. Heckerman, D. Geiger, D. Chickering. Learning Bayesian networks: The Combination of Knowledge and Statistical Data. Machine Learning, 20:197243, 1995. 
Learning graph structure  Gaussian models 
For complete Gaussian data, learning of
parameters and directed acyclic graph. D. Geiger, D. Heckerman. Learning Gaussian networks. Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pp. 235243. 
Variational methods for inference  Class of method for approximate inference. An introduction to variational methods for graphical models. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. In M. I. Jordan (Ed.), Learning in Graphical Models, Cambridge: MIT Press, 1999 Its application to Bayesian inference. Beal, M.J. and Ghahramani, Z. Variational Bayesian Learning of Directed Graphical Models with Hidden Variables To appear in Bayesian Analysis 1(4), 2006. 
Simulation methods for inference (particle
filtering) 
A simulation for dynamic graphical models Chapter of "polycopie" S. Arulampalam, S. Maskell, N. J. Gordon, and T. Clapp, A Tutorial on Particle Filters for Online Nonlinear/NonGaussian Bayesian Tracking, IEEE Transactions of Signal Processing, Vol. 50(2), pages 174188, February 2002. Doucet A., Godsill S.J. and Andrieu C., "On sequential Monte Carlo sampling methods for Bayesian filtering," Statist. Comput., 10, 197208, 2000 
SemiMarkovian models  A class of models allowing to model the time
spent in any given state for a Markov Chain and an HMM. Note from Kevin Murphy [pdf] 
Learning parameters in an undirected graphical model (Markov random fields)  Chapitet 9 of "polycopie" and articles. 
Dynamic graphical models  Chapter of "polycopie". Specific topics to be defined. 
General applications of the sumproduct algorithms (e.g., to the FFT)  The
generalized
distributive law, S. M. Aji, R. J. Mceliece Information Theory, IEEE Transactions on, Vol. 46, No. 2. (2000), pp. 325343. 
Independent Component Analysis  A. Hyvärinen, E. Oja (2000): Independent
Component Analysis: Algorithms and Application, Neural
Networks, 13(45):411430, 2000. Course of Hervé LeBorgne: http://www.eeng.dcu.ie/~hlborgne/pub/th_chap3.pdf

Canonical Correlation Analysis 
CCA is analogous to PCA for the joint analysis of two random
vectors X and Y.

Clustering through a mixture of PCA 
M. E Tipping et C. M Bishop, “Mixtures of probabilistic principal component analyzers,” Neural computation 11, no. 2 (1999): 443–482. 
Stochastic relational models 

Conditional Random Fields 
Charles Sutton, Andrew McCallum An Introduction to Conditional Random Fields for Relational Learning . In Lise Getoor and Ben Taskar, editors, Introduction to Statistical Relational Learning. MIT Press. 2007. 
Dirichlet Process 

Factorial HMM 
Z. Ghahramani et M. I Jordan, “Factorial
hidden
Markov models,” Machine
learning 29, no. 2 (1997): 245–273. 
Generalized PCA 
M. Collins, S. Dasgupta, et R. E
Schapire, “A
generalization of principal component analysis to the exponential
family,” Advances in neural
information processing systems 1 (2002): 617–624. 
Structure learning by L1 regularization 
J. Friedman, T. Hastie, et R.
Tibshirani, “Sparse
inverse
covariance estimation with the graphical lasso,” Biostatistics
9, no. 3 (2008): 432. 
Mixture of logconcave densities 
Interesting nonparametric
models for unimodal distributions. distributions,” Computational Statistics & Data Analysis 51, no. 12 (2007): 6242–6251. 
Constraint satisfaction and Sudoku 
T. K Moon et J. H Gunther, “Multiple constraint satisfaction by belief propagation: An example using sudoku,” dans Adaptive and Learning Systems, 2006 IEEE Mountain Workshop on, 2006, 122–126. 
Bioinformatics  Chapter
of polycopie (ask login/pwd by email) Phylogenetic HMM: A. Siepel et D. Haussler, “Phylogenetic hidden Markov models,” Statistical methods in molecular evolution (2005), 3, 325–351. 
Vision/Speech  Image segmentation by EM. Blobworld: Image Segmentation Using ExpectationMaximization and Its Application to Image Querying; Chad Carson, Serge Belongie, Hayit Greenspan and Jitendra Malik. IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(8), 10261038, August 2002. Articles from Kevin Murphy: "Using the Forest to See the Trees:A Graphical Model Relating Features, Objects and Scenes" Kevin Murphy, Antonio Torralba, William Freeman. NIPS'03 (Neural Info. Processing Systems) Dynamic Bayesian Networks for AudioVisual Speech Recognition A. Nefian, L. Liang, X. Pi, X. Liu and K. Murphy. EURASIP, Journal of Applied Signal Processing, 11:115, 2002 
Robotics  Automatic construction of maps) Simultaneous Localization and Mapping with Sparse Extended Information Filters Thrun et al. The International Journal of Robotics Research.2004; (see also chapter of "polycopie" on Kalman filtering) 
Text  Naive Bayes: A. McCallum and K. Nigam. A comparison of event models for Naive Bayes text classification. In AAAI98 Workshop on Learning for Text Categorization, 1998. Latent Dirichlet allocation. D. Blei, A. Ng, and M. Jordan. Journal of Machine Learning Research, 3:9931022, January 2003. [ .ps.gz  .pdf  code ] 
Text  Natural language processing  S. Vogel, H. Ney, and C. Tillmann.
HMMbased word alignment in statistical translation. In Proceedings
of the 16th conference on Computational linguistics, pp.
836841, Morristown, NJ, USA, 1996. Association for Computational
Linguistics. Non contextual probabilistic grammars: Notes de cours de CMU, 1999 
N most probable configurations  Implementation of an algorithm (HMM or more
complex graphs), from the following articles: Dennis Nilsson, Jacob Goldberger. An Efficient Algorithm for Sequentially finding the NBest List , IJCAI, 1999 Chen Yanover, Yair Weiss, Finding the M Most Probable Configurations Using Loopy Belief Propagation, NIPS 2003. 
Computation of treewidth  Comparing the classical heuristics and finer
methods: Mark Hopkins and Adnan Darwiche A Practical Relaxation of ConstantFactor Treewidth Approximation Algorithms Proceedings of the First European Workshop on Probabilistic Graphical Models 2002 Also some exact methods Stefan Arnborg, Derek G. Corneil, Andrzej Proskurowski, Complexity of finding embeddings in a ktree, SIAM Journal on Algebraic and Discrete Methods (1997) 