Seminar Details
2022-09-20 (14h) : The Polyak-Lojasiewicz condition and overparameterized learning
At Euler building (room a.002)
Organized by Mathematical Engineering
Speakers :
Dávid Terjék (Renyi Institute Budapest)
,
and
Diego González-Sánchez (Renyi Institute Budapest)
Abstract :
We study the convergence of gradient descent on a composition (f ∘ F): G -> R of functions F: G -> H and f: H -> R where G, H are Hilbert spaces. We require f to satisfy the Lipschitz gradient and Polyak-Łojasiewicz conditions, and we find sufficient conditions on F that ensure convergence at a linear rate to a global optimum close to the initial point. We then propose a prototype optimization problem, covering a range of machine learning problems including supervised learning. Applying the results gives us sufficient conditions on the neural network and the data that ensure the global convergence of gradient descent, with the Neural Tangent Kernel making a natural appearance.