ERC SIERRA

Sparse Structured Methods for Machine Learning





SIERRA is a research project funded by the European Research Council (ERC) and coordinated by Francis Bach. It is located within the joint INRIA/CNRS/Ecole Normale Superieure computer science laboratory in downtown Paris. The goals of the project are to explore sparse structured methods for machine learning, with applications in computer vision and audio processing.



Related courses / tutorials :
 


July 2012: Computer Vision and Machine Learning Summer School, Grenoble - Kernel methods and sparse methods for computer vision
July 2011: Computer Vision and Machine Learning Summer School, Paris - Kernel methods and sparse methods for computer vision
September 2010: ECML/PKDD Tutorial on Sparse methods for machine learning (Theory and algorithms)
July 2010: Computer Vision and Machine Learning Summer School, Grenoble - Kernel methods and sparse methods for computer vision
July 2010: Signal processing summer school, Peyresq - Sparse method for machine learning
June 2010: CVPR Tutorial on Sparse Coding and Dictionary Learning for Image Analysis - slides of ML part
December 2009: NIPS Tutorial on Sparse methods for machine learning (Theory and algorithms)
September 2009: ICCV Tutorial on Sparse Coding and Dictionary Learning for Image Analysis
September 2008Machine Learning Summer School - Ile de Re - Learning with sparsity inducing norms (slides)
October 2008: ECCV Tutorial on Supervised Learning: Introduction - Part I (Theory) - Part II (Algorithms)

January 2008: Workshop RASMA, Franceville, Gabon - Introduction to kernel methods (slides in French)




Related publications:


2012

G. Obozinski and F. Bach. Convex Relaxation for Combinatorial Penalties. Technical report, HAL 00694765, 2012. [pdf]

N. Le Roux, M. Schmidt, F. Bach. A Stochastic Gradient Method with an Exponential Convergence Rate for Strongly-Convex Optimization with Finite Training Sets. Technical report, HAL 00674995, 2012. [pdf]


R. Jenatton, A. Gramfort, V. Michel, G. Obozinski, E. Eger, F. Bach, B. Thirion. Multi-scale Mining of fMRI data with Hierarchical Structured Sparsity. SIAM Journal on Imaging Sciences, 2012, 5(3):835-856, 2012. [pdf]

F. Bach, S. Lacoste-Julien, G. Obozinski. On the Equivalence between Herding and Conditional Gradient Algorithms. Proceedings of the International Conference on Machine Learning (ICML), 2012. [pdf]

A. Joulin, F. Bach, J. Ponce. Multi-Class Cosegmentation. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2012. [pdf]

F. Bach, R. Jenatton, J. Mairal, G. Obozinski. Structured sparsity through convex optimization. Technical report, HAL 00621245-v2, to appear in Statistical Science, 2012. [pdf]

F. Bach, R. Jenatton, J. Mairal, G. Obozinski. Optimization with sparsity-inducing penalties. Foundations and Trends in Machine Learning, 4(1):1-106, 2012. [pdf]

J. Mairal, F. Bach, J. Ponce. Task-Driven Dictionary Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4):791-804, 2012. [pdf]


2011

F. Bach. Learning with Submodular Functions: A Convex Optimization Perspective. Technical Report HAL 00645271, 2011. Submitted to Foundations and Trends in Machine Learning. [pdf]


C. Archambeau, F. Bach. Multiple Gaussian process models. Technical Report Arxiv 110.5238, 2011. [pdf]

F. Bach, E. Moulines. Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning. Advances in Neural Information Processing Systems (NIPS), 2011. [pdf] [long-version-pdf-HAL]

M. Schmidt, N. Le Roux, F. Bach. Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization. Advances in Neural Information Processing Systems (NIPS), 2011. [pdf] [long-version-pdf-HAL]

E. Grave, G. Obozinski, F. Bach. Trace Lasso: a trace norm regularization for correlated designs. Advances in Neural Information Processing Systems (NIPS), 2011. [pdf] [long-version-pdf-HAL]


F. Bach. Shaping Level Sets with Submodular Functions. Advances in Neural Information Processing Systems (NIPS), 2011. [pdf] [long-version-pdf-HAL]

B. Mishra, G. Meyer, F. Bach, R. Sepulchre. Low-rank optimization with trace norm penalty. Technical report, Arxiv 1112.2318, 2011. [pdf]


R. Jenatton, J.-Y. Audibert and F. Bach. Structured Variable Selection with Sparsity-inducing Norms. Journal of Machine Learning Research, 12, 2777-2824, 2011. [pdf] [code]


N. Le Roux, F. Bach. Local component analysis. Technical report, HAL 00617965, 2011. [pdf]

J. Mairal, R. Jenatton, G. Obozinski, F. Bach. Convex and Network Flow Optimization for Structured Sparsity. Journal of Machine Learning Research, 12, 2681-2720. [pdf]

R. Jenatton, J. Mairal, G. Obozinski, F. Bach. Proximal Methods for Hierarchical Sparse CodingJournal of Machine Learning Research, 12, 2297-2334, 2011. [pdf]


F. Couzinie-Devy, J. Mairal, F. Bach and J. Ponce. Dictionary Learning for Deblurring and Digital Zoom. Technical report, HAL : inria- 00627402, 2011. [pdf]

Y-L. Boureau, N. Le Roux, F. Bach, J. Ponce, and Y. LeCun. Ask the locals: multi-way local pooling for image recognition. Proceedings of the International Conference on Computer Vision (ICCV), 2011. [pdf]

M. Solnon, S. Arlot, F. Bach. Multi-task Regression using Minimal Penalties. Technical report, HAL 00610534, 2011. [pdf]

A. Lefèvre, F. Bach, C. Févotte. Online algorithms for Nonnegative Matrix Factorization with the Itakura-Saito divergence. Technical report, HAL 00602050, 2011. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2011. [pdf]

T. Hocking, A. Joulin, F. Bach and J.-P. Vert. Clusterpath: an Algorithm for Clustering using Convex Fusion PenaltiesProceedings of the International Conference on Machine Learning (ICML), 2011. [pdf]

L. Benoît, J. Mairal, F. Bach, J. Ponce, Sparse Image Representation with Epitomes. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2011. [pdf]

A. Lefèvre, F. Bach, C. Févotte, Itakura-Saito nonnegative matrix factorization with group sparsityProceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2011. [pdf]

F. Bach, R. Jenatton, J. Mairal and G. Obozinski. Convex optimization with sparsity-inducing norms. In S. Sra, S. Nowozin, S. J. Wright., editors, Optimization for Machine Learning, MIT Press, 2011. [pdf]



2010

F. Bach. Convex Analysis and Optimization with Submodular Functions: a Tutorial. Technical report, HAL 00527714, 2010. [pdf]

F. Bach. Structured Sparsity-Inducing Norms through Submodular FunctionsAdvances in Neural Information Processing Systems (NIPS), 2010. [pdf] [long version, arxiv] [slides]

J. Mairal, R. Jenatton, G. Obozinski, F. Bach. Network Flow Algorithms for Structured SparsityAdvances in Neural Information Processing Systems (NIPS), 2010. [pdf]

F. Bach, S. D. Ahipasaoglu, A. d'Aspremont. Convex Relaxations for Subset Selection. Technical report Arxiv 1006-3601. [pdf]

M. Hoffman, D. Blei, F. Bach. Online Learning for Latent Dirichlet AllocationAdvances in Neural Information Processing Systems (NIPS), 2010. [pdf]

F. Bach, S. D. Ahipasaoglu, A. d'Aspremont. Convex Relaxations for Subset Selection. Technical report, ArXiv 1006.3601, 2010. [pdf]

R. Jenatton, J. Mairal, G. Obozinski, F. Bach. Proximal Methods for Sparse Hierarchical Dictionary LearningProceedings of the International Conference on Machine Learning (ICML), 2010. [pdf] [slides]

M. Journée, F. Bach, P.-A. Absil and R. Sepulchre. Low-Rank Optimization on the Cone of Positive Semidefinite Matrices. SIAM Journal on Optimization, 20(5):2327–2351, 2010. [pdf] [code]

Y-L. Boureau, F. Bach, Y. LeCun, J. Ponce. Learning Mid-Level Features For RecognitionProceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2010. [pdf]

R. Jenatton, G. Obozinski, F. Bach. Structured Sparse Principal Component Analysis. Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), 2010. [pdf] [code]

F. Bach. Self-Concordant Analysis for Logistic Regression. Electronic Journal of Statistics, 4, 384-414, 2010. [pdf]

J. Mairal, F. Bach, J. Ponce, G. Sapiro. Online Learning for Matrix Factorization and Sparse Coding. Journal of Machine Learning Research, 11, 10-60, 2010. [pdf] [code]




Related software:


Sparse modeling software - SPAM (C)

Computing regularization paths for multiple kernel learning - version 1.0 (matlab)

Predictive low-rank decomposition for kernel methods - version 1.0 (matlab/C)

Support Kernel Machine - Multiple kernel learning (matlab)

SimpleMKL - version 1.0 (matlab)

Grouplasso - version 1.0 (matlab)

Hierarchical kernel learning - version 3.0 (matlab)