Seminar Details
2021-12-07 (14h) : The symbiotic relationship between optimization and deep learning
At Euler building (room a.002)
Organized by Mathematical Engineering
Speaker :
Ablin, Pierre
Abstract :
Optimization is one of the cornerstones of deep learning: most deep neural networks are trained by optimizing a cost function. The purpose of this talk is to cover other fruitful interactions between the fields of optimization and deep learning that are less obvious. First, I will discuss how deep neural networks can be designed to quickly and efficiently solve classical optimization problems, with a focus on the Lasso. I will then argue that the tools of optimization allow us to describe accurately what weights are learned by these networks during the training process. In a second part, I will discuss how classical optimization ideas like momentum acceleration can be translated to deep learning, and allow us to develop novel architectures that perform well and are memory-efficient.