2:10 a.m.
Dear Colleagues,
This Friday, in the UTSA MATRIX AI and ECE Seminar Series, I will be hosting Dr. René
Vidal (IEEE, ACM, AIMBE Fellow) from UPenn for an in-person talk at UTSA Main Campus
(please see flyer attached as well as title/abstract/bio below). We will also try to
broadcast the talk on Zoom (https://utsa.zoom.us/j/94807623288) for those that cannot make
it in-person.
Please consider joining us in-person if your schedule permits. Also, please consider
encouraging your students to attend, as well as spreading the word to any interested
colleagues.
All the best,
Panos
Title: “Learning Dynamics and Implicit Bias of Gradient Flow in Overparameterized Linear
Models”
Abstract: Contrary to the common belief that overparameterization may hurt generalization
and optimization, recent work suggests that overparameterization may bias the optimization
algorithm towards solutions that generalize well, a phenomenon known as implicit
regularization or implicit bias, and may also accelerate convergence, a phenomenon known
as implicit acceleration. This talk will provide a detailed analysis of the dynamics of
gradient flow in overparameterized two-layer linear models showing that convergence to
equilibrium depends on the imbalance between input and output weights (which is fixed at
initialization) and the margin of the initial solution. The talk will also provide an
analysis of the implicit bias, showing that large hidden layer width, together with
(properly scaled) random initialization, constrains the network parameters to converge to
a solution which is close to the min-norm solution.
Bio: René Vidal is the Penn Integrates Knowledge and Rachleff University Professor of
Electrical and Systems Engineering & Radiology and the Director of the Center for
Innovation in Data Engineering and Science (IDEAS) at the University of Pennsylvania. He
also directs THEORINET, an NSF-Simons Collaboration on the Mathematical Foundations of
Deep Learning, and the NSF TRIPODS Institute on the Foundations of Graph and Deep
Learning. He is also an Amazon Scholar, an Affiliated Chief Scientist at NORCE, and an
Associate Editor-in-Chief of TPAMI. His current research focuses on the foundations of
deep learning and trustworthy AI and its applications in computer vision and biomedical
data science. He is an ACM Fellow, AIMBE Fellow, IEEE Fellow, IAPR Fellow and Sloan
Fellow, and has received numerous awards for his work, including the IEEE Edward J.
McCluskey Technical Achievement Award, D’Alembert Faculty Award, J.K. Aggarwal Prize, ONR
Young Investigator Award, NSF CAREER Award as well as best paper awards in machine
learning, computer vision, controls, and medical robotics.
Dr. Panagiotis (Panos) Markopoulos, Ph.D.
Associate Professor and Margie and Bill Klesse Endowed Professor
Department of Electrical & Computer Engineering and Department of Computer Science
Chair, Digital Signal Processing Concentration (ECE Dept.)
Founding Director, Machine Learning Optimization and Signal Processing (MELOS) Laboratory
(@ San Pedro I)
Founding Director, Multi-modal Sensing and Signal Processing (MSSP) Laboratory (@ San
Pedro II)
Core Faculty, UTSA MATRIX AI Consortium
The University of Texas at San Antonio
----------------------------------------------
Personal Web: https://www.markopoulos.org/
Lab Web: https://www.meloslab.org/
E-mail: panagiotis.markopoulos(a)utsa.edu (preferred)
Address: One UTSA Circle, San Antonio, TX 78249-3209
