Mark Schmidt's Home Page

Mark Schmidt
    INRIA - SIERRA team
    Laboratoire d'Informatique de l'École Normale Supérieure
    23, Avenue d'Italie
    CS 81321
    75214 Paris Cedex 13

2011-2013: Postdoc with CNRS/ENS/INRIA with Francis Bach as part of the SIERRA project.
2010: Postdoc at University of British Columbia with Michael Friedlander in the Laboratory for Scientific Computing.
2005-2010: Ph.D. student at University of British Columbia with Kevin Murphy in the Laboratory for Computational Intelligence.
2006: Internship at Siemens Medical Solutions with Rómer Rosales and Glenn Fung at the Computer-Aided Diagnosis and Therapy Group.
2003-2005: M.Sc. student at University of Alberta with Russ Greiner as part of the Brain Tumor Analysis Project.

Research

List of Publications - with links to code, presentations/posters, and appendices.

Selected recent papers:

Selected applications:

Code

Matlab Software - full list of software packages.

code.zip - package containing most recent version of various codes.

Some Highlights:

batching (optimizer that combines an L-BFGS line-search method with a growing batch-size strategy)
alphaBeta (approximate pairwise energy minimization with attractive energies)
thesis (quasi-Newton methods for non-differentiable optimization and and structure learning in graphical models)
UGM (exact and approximate decoding, inference, sampling, and parameter estimation in discrete undirected graphical models)
minConf (optimization of differentiable functions with simple constraints, examples 1, examples 2)
crfChain (chain-structured conditional random fields with categorical features)
L1General (minimizing differentiable functions with L1-regularization, examples).
blogreg (MCMC simulation of binary probit/logistic regression posterior distributions)
lasso (a variety of methods to solve L1-regularized least squares)
minFunc (unconstrained optimization of differentiable functions, examples)

Presentations

Selected recent talks and posters:

SFU 2012 slides (Opening up the black box)
UBC 2012 slides (Linearly-Convergent Stochastic-Gradient Methods)
NIPS 2011 slides (Convergence Rates of Inexact Proximal-Gradient Methods)
Xerox 2011 slides (Hybrid Deterministic-Stochastic Methods)
INRA 2011 slides (Structure Learning in Undirected Graphical Models)
OPT 2010 slides (Limited-Memory Quasi-Newton and Hessian-Free Newton Methods)
AISTATS 2010 slides (Convex Structure Learning in Log-Linear Models)
IBM 2009 slides (Optimizing Costly Functions with Simple Constraints)
UAI 2009 poster (Group Sparse Priors for Covariance Estimation)
CVPR 2009 poster (Increased Discrimination in Level Set Methods)

Miscellaneous

Notes, addenda, etc.

Fast Stochastic Gradient - Note showing that constant-step SGD converges fast under a strong assumption.
Big-N Problems - An overview of methods for solving problems with a large number of training examples.
Batching and Shrinking with the Hinge Loss - Smooth-SVM solver with a growing-sample strategy and hinge-loss bounds.
MLSS 2011 practicals - Supporting material from MLSS 2011 practical sessions on convex optimization.
Convergence Rates - Notes from a presentation on non-asymptotic rates of deterministic and stochastic optimization methods.
Singular Value Thresholding - Notes from a presentation on nuclear norm regularization and proximal-gradient algorithms.
Gradient Descent - Scan of the 1847 paper where Cauchy proposes gradient descent (in French).
Generalized Interventional Potentials - Notes on variants of interventional potentials, and extensions to higher-order interactions.
Polar Cones - Notes on polar cones and optimality conditions for non-smooth constrained convex optimization.
Structural SVMs - An introduction to structural support vector machines, and an overview of methods for solving them.