I am a Professor in Statistics and Learning at École Polytechnique, Palaiseau, in the applied mathematics department.

My main research interests are statistics, optimization, stochastic approximation, Federated Learning, high-dimensional learning, non-parametric statistics, scalable kernel methods.

Before that, I was postdoctoral Researcher at EPFL (Ecole Polytechnique Fédérale de Lausanne), in the MLO team, directed by Martin Jaggi. Before that, I was Ph.D. student in the Sierra Team, which is part of the DI/ENS (Computer Science Department of École Normale Supérieure). I was supervised by Francis Bach. I graduated from Ecole Normale Supérieure de Paris (Ulm) in 2014 and got a Master's Degree in Mathematics, Probability and Statistics (at Université Paris-Sud, Orsay).

From March to August 2016, I was a visiting scholar researcher at University of California Berkeley, under the supervision of Martin Wainwright.

News!

05/23: Two papers have just been accepted to ICML 2023! Congratulations to Margaux Zaffran and Alexis Ayme for their second PhD papers!

04/23: Welcome to Renaud Gaucher (PhD candidate), Rémi Leluc (Postdoc), Damien Ferbach (Intern), and Mahmoud Hegazy (Intern), that are joining my group for the next months (or years)!

04/23: I successfully defended my Accreditation to supervise research. Many thanks to the reviewers and the jury.

4/23 A couple of new preprints are available: a survey on stochastic approximation methods, beyond the gradient case, and a paper on a constructive approach to build counter-examples in first-order optimization. Links below!

01/23: Lecture in Orsay : lecture page.

12/22: We are looking for a PhD student with Hadrien Hendrikx. More details!

12/22: I am looking for a postdoc on Federated Learning! If you are innterested, send me an email for more details.

05/22: Two papers have just been accepted to ICML 2022! Congratulations to Margaux Zaffran and Alexis Ayme for their first PhD papers!

March 2022 - LPSM. Federated Learning and optimization: from a gentle introduction to recent results Slides.

01/22: Three papers have been accepted to AISTATS 2022! Special congratulations to Maxence for his first paper.

01/22: Happy new year! We have released the first version of PEPit: a python package on computer assisted proofs. This can be incredibly useful if you are interested in worst case guarantees on first order methods. The code is available on Github and you can have a look at

Congratulations to Baptiste Goujaud and Celine Mourcer for their work on the package. This is also a joint work with François Glineur, Julien Hendrickx, Adrien Taylor.

11/21 - Two new preprints are availbale, respectively on Compression on Gaussian random codebooks with applications to FL, and on Utility-Privacy tradeoffs in Heterogeneous FL frameworks. See below for links and details!

Two papers have been accepted at Neurips 2021! Federated Expectation Maximization with heterogeneity mitigation and variance reduction, and Preserved central model for faster bidirectional compression in distributed settings, see below for links and details!

29/09 - FLOW - Federated Learning One World Seminar. Presentation on Preserved central model for faster bidirectional compression in distributed settings. Slides.

On September 16, 2021, join us at the Federated Learning Workshop, a full-day hybrid event that takes place both online and in Paris. A great panel of speakers from academia and industry will forecast the most promising directions for future research on federated learning and the development of new benchmarks and application challenges.

07/2021: Lab on Optimization, CEMRACS
Notebook on optimization methods - COLAB ;
Notebook on optimization methods;
Bits of code for the correction;
Notebook on optimization methods - CORRECTION.

07/2021: Tutorial on Stochastic Optimization, Hi! PARIS Summer School 2021 on AI & Data for Science, Business and Society. Slides (without annotations); Slides (with annotations);

06/2021: A couple of new papers are available on arxiv, especially on ``super acceleration'' (faster than momentum acceleration under assumptions on the Hessian matrix) and Quantized Langevin Dynamics.

04/2021: Maxence Noble is starting his research internship! Maxence will be working on utility and privacy tradeoffs in Federated Learning, and is co-supervised by Aurélien Bellet.

04/2021: Alexis Ayme is starting his research internship and then PhD thesis! Alexis will be working on learning missing data, and is co-supervised by Claire Boyer and Erwan Scornet. His web page.

12/2020: Margaux Zaffran is starting her PhD! Margaux will be working on Electricity Price Prediction, with EDF. She is co-supervised with Julie Josse (Inria), Yannig Goude and Olivier Féron.

10/2020: Baptiste Goujaud is starting his PhD! Baptiste will be working on First Order Optimization. He is supervised by myself and Éric Moulines. Baptiste has already worked on optimization during his time at MILA, especially the tuning and convergence of first order optimization algorithms.

October 2020: Our team is still looking for a Postdocs and Research engineers, with very competitive conditions!! If you have a PhD in statistics, optimization and Machine learning and are interested to join a great team in Paris, send me an email.


09/2020: Our paper ``Debiasing Stochastic Gradient Descent to handle missing values'' has been accepted at NeurIPS2020. This is a joint work with Aude Sportisse, Claire Boyer and Julie Josse. see Arxiv version

06/2020: Our paper ``On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent'' was accepted at ICML 2020. This is a joint work with Scott Pesme and Nicolas Flammarion at EPFL. see Arxiv version

04/2020: The applied mathematics department at École Polytechnique has open positions for Tenure Track professors, one on Statistics and the other one on Statistics and Energy. These positions offer competitive conditions.

10/03/2020: Optimization for Machine Learning workshop. in Luminy. On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent, Slides here!

26/01/2020: The third edition of the "Advances in Machine Learning, theory meets practice", at Applied Machine Learning Days (AMLD) in Lausanne, that we were co-organizing with Sebastian Stich was a nice occasion to bring theoreticians and practitioners together: thanks to the speakers for their great talks!
See the workshop page for slides and details.

01/2020: Our paper on using optimal transport for NLP has been accepted to AISTATS2020!

12/2019: Our paper on unsupervised time series representation has successfully passed the Neurips reproducibility challenge! About 70 papers published at NeurIPS 2019 were picked by independent researchers, to reproduce the results, assess if the description of the framework was complete, and provide feedback. You can find the discussion on our paper here and the full 12 pages report on our work here. F. Liljefors, M. M. Sorkhei and S. Broomé were able to reproduce and reimplement our methods from the description given in the paper, and to obtain the same results as in our original paper! We would like to thank them for their work on our paper!

12/2019: I will be presenting two papers at NeurIPS 2019 in Vancouver. The first one one distributed optimization, more specifically Local SGD, with K. K. Patel, and the second one on a new methods to generate representations of time series in an unsupervised fashion, with J.Y. Franceschi M. Jaggi. Links to the papers below!

12/2019: Constantin Philippenko is starting his PhD! Constantin will be working on Federated Learning, especially on problems arising from privacy concerns. He will be supervised by myself and Éric Moulines, and will also be working with Richard Vidal and Leatitia Kameni from the research team at Accenture. Welcome to my first PhD student! :)

10/2019: I will be giving a lecture on Large Scale Learning at the Autumn School in Machine Learning, Tbilisi, Geogia. You can find the slides here.

Publications and Preprints

Counter-examples in first-order optimization: a constructive approach.
with B. Goujaud, A. Taylor.
Preprint, 2023
Conformal Prediction with Missing Values.
with M Zaffran, J Josse, Y Romano
Preprint, 2023, Accepted to ICML23
Naive imputation implicitly regularizes high-dimensional linear models
with A. Ayme, C. Boyer, E. Scornet
Preprint, 2023, Accepted to ICML23
Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning
with G Fort, E Moulines, HT Wai
Preprint, 2022, Under minor revision at IEEE TSP
Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes .
with B. Goujaud, A. Taylor.
Preprint, 2022
Adaptive Conformal Predictions for Time Series
with M. Zaffran, O. Féron, Y. Goude, J. Josse
Preprint, 2022, Accepted to ICML22
Minimax rate of consistency for linear models with missing values
with A. Ayme, C. Boyer, E. Scornet
Preprint, 2022, Accepted to ICML22
PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python.
with B. Goujaud, C. Moucer, F. Glineur, J. Hendrickx, A. Taylor.
Preprint, 2022
Differentially Private Federated Learning on Heterogeneous Data.
with Maxence Noble, Aurélien Bellet
Accepted in AISTATS, 2022. Preprint, 2021
QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning
with Maxime Vono, Vincent Plassier, Alain Durmus and Eric Moulines
Accepted in AISTATS, 2022. Arxiv Preprint, 2021
Super-Acceleration with Cyclical Step-sizes
with Baptiste Goujaud, Damien Scieur, Adrien Taylor, Fabian Pedregosa
Accepted in AISTATS, 2022. Arxiv Preprint, 2021
Dostovoq: Doubly Stochastic Voronoi Vector Quantization SGD for Federated Learning.
with Louis Leconte, Edouard Oyallon, Eric Moulines, Gilles PAGES
Preprint, 2021
Federated Expectation Maximization with heterogeneity mitigation and variance reduction
with Gersende Fort, Eric Moulines, Geneviève Robin
Accepted at NeurIPS 2021, Arxiv Preprint, 2021
Preserved central model for faster bidirectional compression in distributed settings
with Constantin Philippenko
Accepted at NeurIPS 2021, Arxiv Preprint, 2021
Artemis: tight convergence guarantees for bidirectional compression in Federated Learning
with Constantin Philippenko
Arxiv Preprint, 2020
On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
with Scott Pesme, Nicolas Flammarion
ICML 2020
Debiasing Stochastic Gradient Descent to handle missing values
with Aude Sportisse, Claire Boyer and Julie Josse
Neurips, 2020
Unsupervised Scalable Representation Learning for Multivariate Time Series
with Jean-Yves Franceschi and Martin Jaggi,
Neurips 2019, 1901.10738.
Communication trade-offs for synchronized distributed SGD with large step size
with Kumar Kshitij Patel,
Neurips 2019 arXiv:1904.11325 .
Context Mover's Distance & Barycenters: Optimal transport of contexts for building representations
with Sidak Pal Singh, Andreas Hug, Martin Jaggi,
ICLR 2019, workshop, accepted at AISTATS 2020
Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains
Accepted in the Annals of Statistics, 2019 arXiv:1707.06386 [math.ST].
Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression
Journal of Machine Learning Research (JMLR), arXiv:1602.05419 [math.ST].
Non-parametric Stochastic Approximation with Large Step sizes
with Francis Bach.
Published in the Annals of Statistics.

Workshops

December 2021, Differential Privacy for Heterogeneous Federated Learning, with Maxence Noble and Aurelien Bellet PPML 2021 . [paper]
16 September 2021, Organizer (with Owkin, Accenture, SFDS) of the Federated Learning Workshop, Sorbonnne Paris Université.
January 2020, Organizer of the "Theory Meets Practice" Workshop AMLD 2020, Lausanne. [ Event page]
January 2019, Organizer of the "Advances in Machine Learning, Theory Meets Practice" Workshop AMLD 2019, Lausanne. [ Event page]
December 2016, Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression, with Nicolas Flammarion, Nips, OPT16. [abstract]
December 2015, Adaptivity of stochastic gradient descent, Nips Workshop. [abstract][slides]

Teaching

2021-2022 : Collaborative and reliable learning , 3A (~first year of Masters) MAP 578, Polytechnique, with El Mahdi El Mhamdi.
2019-2022 : Statistiques , 2A (~third year bachelor) MAP 433, Polytechnique, with E. Moulines, G. Fort, M. Lerasle, S. Gadat.
2019-2022 : Generallization Properties of learning algorithms , M2 Data Science.
2019-2022 : Optimization and Deep learning , M2 Data Science for Business, Polytechnique, with E. Scornet.
2019-2021 : Statistics , M2 Data Science for Business, Polytechnique.
2018-2019 : Probabilities and Statistics , 1A (~third year bachelor) MAP 361, Polytechnique.
2016-2017 : Teaching assistant, Statistics , Master 1 (31NU02MS), University Paris Diderot.
2016-2017 : Teaching assistant, Fundamental Statistics , Master 1 (ULMT42), University Paris Diderot.
2015-2016 : Teaching assistant, Calculus , (MM1), University Paris Diderot.
2014-2015 : Teaching assistant, Linear Algebra , (MM1), University Paris Diderot.
2010-2014 : Oral interrogations in ``Classes préparatoires'' (PC, MP*).

Reviews and Commintees

Reviewer for JMLR, AOS, COLT, IEEE, ACM, ICML, annales de l'IHP, Constructive Approximation, ALT, AISTATS.

I was a member of the Jury for Belhal Karimi's PhD defense, on September, the 19th, 2019.

I am a referee for Luigi Carratino's PhD dissertation, that was defended in early spring 2020.

I am part of the scientific committee for the seminar le Palaisien.

Some Talks

March 2022 - LPSM. Federated Learning and optimization: from a gentle introduction to recent results Slides.

September 2021 - FLOW - Federated Learning One World Seminar. Presentation on Preserved central model for faster bidirectional compression in distributed settings. Slides.

March 2021, Bi-directional compression for Federated Learning: Artemis & MCM Séminaire, Télécom Paris. Slides here

February 2021, Debiasing Averaged Stochastic Gradient Descent to handle missing values, Séminaire de Statistiques Parisien. Slides here

March 2020, On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent, Optimization for Machine Learning workshop. Slides here!

March 2020, On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent, Parisian Seminar of Optimization.

January 2020, On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent, Inria Paris

October 2019, Large Scale Learning and Optimization, Tbilisi, Georgia. 6 hours lecture at ASML, slides here!

April 2019, Journées Calcul et Apprentissage, Lyon

December 2018, Communication trade-offs for synchronized distributed SGD with large step size, CMStatistics 2018, Pisa, Italy [slides]

January 2018, Tutoriel d’Optimization, Journées “YSP” organisées par la SFDS, Institut Henri Poincaré [slides]

Decembrer 2017, Stochastic algorithms in Machine learning, Tutoriel at “journée algorithmes stochastiques", Paris Dauphine. [slides]

November 2017, Stochastic approximation and Markov chains, Invited talk, Télécom Paristech, Paris [slides]

February 2017, Scalable methods for Statistics, a short presentation, Cambridge, UK . [slides]

March 2016, Non parametric stochastic approxiation, UC Berkeley.

October 2015, Tradeoffs of learning in Hilbert spaces, ENSAI Rennes. [slides]

June 2015, Non parametric Stochastic Approximation, Machine Learning Summer School, Tubingen.

Thesis defense!

I defended my Accreditation to Supervise resaech on Tuesday, March 28, at 10 am/

You can download the final version of the manuscript

You can also have a look at the slides.

Thesis defense!

I defended my thesis on Thursday, September 28, at 2.30 pm, at INRIA.

You can download the final version of the manuscript (or here if you want to print it).

You can also have a look at the slides.