Leonardo Galli

Postdoc Researcher
LMU Munich
RWTH Aachen (transitioning)

I am a researcher working on optimization for machine learning. I am currently focusing on deep learning and its mathematical foundations. My goal is to design reliable methods to create a bridge between theory and practice. I am interested in large-scale optimization, (nonmonotone) line search methods for deep learning, the Edge of Stability phenomenon, Sharpness-Aware-Minimization and all that.

Short bio: I obtained my B.S., M.S. and Ph.D. degrees from University of Florence respectively in 2013, 2016 and 2020. My Ph.D. advisors and mentors in Florence were prof. Marco Sciandrone and prof. Fabio Schoen. To avoid stereotypical comments on italians, I tried to escape my hometown a few times during my studies (University of Würzburg in 2015, UCLA in 2019 and National Taiwan University in 2020, all documented in my photograpic logbook and by my papers). From 2015 to 2017 I collaborated with prof. Christian Kanzow on generalized Nash equilibrium problems and from 2018 to 2020 with prof. Chih-Jen Lin on truncated Newton methods for linear classification. Since 2021 I am working with prof. Holger Rauhut on the mathematical foundations of deep learning, fist at RWTH Aachen and now at LMU Munich. I more recetly started collaborating with prof. Mark Schmidt on line search methods for deep learning.

News

2023-11: Our 5-minutes NeurIPS poster is now online!

2023-10: The weekly MIP seminar is about to restart!

2023-10: I joined prof. Rauhut at LMU Munich.

2023-09: New paper accepted to NeurIPS 2023! ”Don't be so monotone: Relaxing Stochastic Line Search in Over-Parameterized Models”

2023-08: New paper accepted to IJCNN 2023! ”Faster Convergence for Transformer Fine-tuning with Line Search Methods”

2021-11: I won the 2 years KI-starter grant!