Simon Lacoste-Julien Assistant Professor Department of Computer Science and Operations Research (DIRO) and Montreal Institute for Learning Algorithms (MILA) Université de Montréal 2920 chemin de la Tour (map) Montréal (QC) Canada Pavillon André-Aisenstadt, 3rd floor office: 3339

Update August 2016: I just moved to the Université de Montréal.

(Until August 2016) I was a researcher at INRIA in the SIERRA project team which is part of the Computer Science Department of École Normale Supérieure in Paris.

I did my PhD in Computer Science at the University of California, Berkeley under the supervision of Michael I. Jordan, and (basically) a B.Sc. Triple Honours in Mathematics, Physics and Computer Science at McGill University. I then worked with Zoubin Ghahramani as a postdoc in the Machine Learning Group of the University of Cambridge. In September 2011, I got a Research in Paris fellowship to work with Francis Bach in the SIERRA project team, and then I joined as a researcher in September 2013.

Research Interests

• Structured prediction
• Latent variable models
• Convex optimization
• Approximate inference
• Combining generative and discriminative methods / frequentist and Bayesian statistics -- see the NIPS 2009 workshop on The Generative and Discriminative Learning Interface that I have co-organized, and my AISTATS 2011 paper on the loss-calibrated Bayesian below.
• Applications: NLP, information retrieval, computer vision, computational biology

Students and Postdocs

• Jean-Baptiste Alayrac, co-advised with Josef Sivic and Ivan Laptev
• Aristide Baratin
• Gauthier Gidel
• Gabriel Huang
• Rémi Leblond
• Rémi Le Priol

Alumni

• Anton Osokin, Postdoctoral Researcher, Willow team, INRIA Paris
• Anastasia Podosinnikova, Postdoctoral Researcher, MIT

Teaching

• Winter 2017: IFT 6085 Advanced Structured Prediction - Grad class at Université de Montréal
• Fall 2016: IFT 6269 Probabilistic Graphical Models - Grad class at Université de Montréal
• 2015-2016: Machine learning programming project class - Master M2 MASH (Université Dauphine)
• Fall 2015: An introduction to graphical models - Master M2 "Mathematiques, Vision, Apprentissage" - Ecole Normale Superieure de Cachan
• Fall 2015: Statistical machine learning - Master M1 - Ecole Normale Superieure (Paris)
• 2014-2015: Machine learning programming project class - Master M2 MASH (Université Dauphine)
• Fall 2014: Statistical machine learning - Master M1 - Ecole Normale Superieure (Paris)

Papers

[TR] SEARNN: Training RNNs with Global-Local Losses, R. Leblond*, J.-B. Alayrac*, A. Osokin and S. Lacoste-Julien, arXiv:1706.04499 [cs.LG], June 2017. *Both authors contributed equally

[new!] A Closer Look at Memorization in Deep Networks, D. Arpit*, S. Jastrzebski*, N. Ballas*, D. Krueger*, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio and S. Lacoste-Julien, International Conference on Machine Learning (ICML 2017), Sydney, Australia, August 2017. *Equal contribution.

[new!] Frank-Wolfe Algorithms for Saddle Point Problems, G. Gidel, T. Jebara and S. Lacoste-Julien, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), Fort Lauderdale, Florida, USA, April 2017. [project website]

[new!] ASAGA: Asynchronous Parallel SAGA, R. Leblond, F. Pedregosa and S. Lacoste-Julien, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), Fort Lauderdale, Florida, USA, April 2017. [project website]

[TR] On Structured Prediction Theory with Calibrated Convex Surrogate Losses, A. Osokin, F. Bach and S. Lacoste-Julien, arXiv:1703.02403 [cs.LG], March 2017.

[TR] Joint Discovery of Object States and Manipulating Actions, J.-B. Alayrac, J. Sivic, I. Laptev and S. Lacoste-Julien, arXiv:1702.02738 [cs.CV], Feb 2017.

[TR] Convergence Rate of Frank-Wolfe for Non-Convex Objectives, S. Lacoste-Julien, arXiv:1607.00345 [math.OC], June 2016.

[new!] PAC-Bayesian Theory Meets Bayesian Inference, P. Germain, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS16), Barcelona, Spain, December 2016.

[new!] Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs, A. Osokin*, J.-B. Alayrac*, I. Lukasewitz, P. Dokania and S. Lacoste-Julien, International Conference on Machine Learning (ICML 2016), New York City, USA, June 2016. *Both authors contributed equally. [project website]

[new!] Beyond CCA: Moment Matching for Multi-View Models, A. Podosinnikova, F. Bach and S. Lacoste-Julien, International Conference on Machine Learning (ICML 2016), New York City, USA, June 2016. [code]

[new!] Unsupervised Learning from Narrated Instruction Videos, J.-B. Alayrac, P. Bojanowski, N. Agrawal, I. Laptev, J. Sivic and S. Lacoste-Julien, Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vega, USA, June 2016. [project website]

On the Global Linear Convergence of Frank-Wolfe Optimization Variants, S. Lacoste-Julien and M. Jaggi, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015. [code]

Barrier Frank-Wolfe for Marginal Inference, R. Krishnan, S. Lacoste-Julien and D. Sontag, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015. [code]

Variance Reduced Stochastic Gradient Descent with Neighbors, T. Hofmann, A. Lucchi, S. Lacoste-Julien, and Brian McWilliams, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015.

Rethinking LDA: Moment Matching for Discrete ICA, A. Podosinnikova, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015. [code] [project's web page]

On Pairwise Costs for Network Flow Multi-Object Tracking, V. Chari, S. Lacoste-Julien, I. Laptev and J. Sivic, Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, USA, June 2015. [project website]

Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering, S. Lacoste-Julien, F. Lindsten and F. Bach. International Conference on Artificial Intelligence and Statistics (AISTATS 2015), San Diego, California, USA, May 2015. MCMCSki IV poster prize honourable mention (2014).

SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives, A. Defazio, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS14), Montreal, Canada, December 2014.

An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Algorithms, S. Lacoste-Julien and M. Jaggi, appeared at the NIPS 2013 Workshop on Greedy Algorithms, Frank-Wolfe and Friends, arXiv:1312.7864 [math.OC], December 2013.

SiGMa: Simple Greedy Matching for Aligning Large Knowledge Bases, S. Lacoste-Julien, K. Palla, A. Davies, G. Kasneci, T. Graepel and Z. Ghahramani, 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2013), Chicago, USA, August 2013.
Previous longer preprint: arXiv:1207.4525v1 [cs.AI], July 2012.

Block-Coordinate Frank-Wolfe Optimization for Structural SVMs, S. Lacoste-Julien*, M. Jaggi*, M. Schmidt and P. Pletscher, International Conference on Machine Learning (ICML 2013), Atlanta, USA, June 2013. *Both authors contributed equally. [code (Matlab / Octave)]

A Simpler Approach to Obtaining an O(1/t) Convergence Rate for the Projected Stochastic Subgradient Method, S. Lacoste-Julien, M. Schmidt and F. Bach, arXiv:1212.2002v2 [cs.LG], December 2012.

On the Equivalence between Herding and Conditional Gradient Algorithms, F. Bach, S. Lacoste-Julien and G. Obozinski, International Conference on Machine Learning (ICML 2012), Edinburgh, UK, June 2012.

Approximate Gaussian Integration using Expectation Propagation, J.P. Cunningham, P. Hennig and S. Lacoste-Julien, arXiv:11111.6832v1 [stat.ML], November 2011.

A Kernel Approach to Tractable Bayesian Nonparametrics., F. Huszár and S. Lacoste-Julien, arXiv:1103.1761v3, [stat.ML], March 2011.

Approximate Inference for the Loss-Calibrated Bayesian, S. Lacoste-Julien, F. Huszár, and Z. Ghahramani, International Conference on Artificial Intelligence and Statistics (AISTATS11), Florida, April 2011.

Discriminative Machine Learning with Structure, S. Lacoste-Julien, PhD Thesis, University of California, Berkeley, 2009.

DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification. S. Lacoste-Julien, F. Sha, and M. Jordan, Neural Information Processing Systems Conference (NIPS08), Vancouver, British Columbia, December 2008.

Word Alignment via Quadratic Assignment. S. Lacoste-Julien, B. Taskar, D. Klein, and M. Jordan, Human Language Technology conference - North American chapter of the Association for Computational Linguistics (HLT-NAACL06), New York, June 2006.

Structured Prediction, Dual Extragradient and Bregman Projections. B. Taskar, S. Lacoste-Julien, and M. Jordan, Journal of Machine Learning Research (JMLR), Special Topic on Machine Learning and Large Scale Optimization, 7, 1627-1653, 2006.

Structured Prediction via the Extragradient Method. B. Taskar, S. Lacoste-Julien, and M. Jordan, Neural Information Processing Systems Conference (NIPS05), Vancouver, British Columbia, December 2005. [Longer version]

A Discriminative Matching Approach to Word Alignment. B. Taskar, S. Lacoste-Julien, and D. Klein, Empirical Methods in Natural Language Processing (EMNLP05), Vancouver, British Columbia, October 2005.

Meta-Modelling Hybrid Formalisms. S. Lacoste-Julien, H. Vangheluwe, J. de Lara and P. Mosterman, IEEE International Symposium on Computer Aided Control System Design, special section on multi-paradigm modelling. Taiwan, September 2004.

A UC Berkeley class project which has been cited a few times as a tutorial: An introduction to Max-Margin Markov Networks. S. Lacoste-Julien, 2003.