Polyak Steps for Adaptive Fast Gradient Methods.
TITLE: Polyak Steps for Adaptive Fast Gradient Methods.
AUTHORS: M. Barré, A. d'Aspremont.
ABSTRACT: Accelerated algorithms for minimizing smooth strongly convex functions
usually require knowledge of the strong convexity parameter mu. In the case
of an unknown mu, current adaptive techniques are based on restart schemes.
When the optimal value f^* is known, these strategies recover the accelerated
linear convergence bound without additional grid search. In this paper we
propose a new approach that has the same bound without any restart, using an
online estimation of strong convexity parameter. We show the robustness of the
Fast Gradient Method when using a sequence of upper bounds on mu. We also
present a good candidate for this estimate sequence and detail consistent
empirical results.
ArXiv PREPRINT: 1906.03056
PAPER: Polyak Steps for Adaptive Fast Gradient Methods in pdf
|