This Friday, in the UTSA MATRIX AI and ECE Seminar Series, I will be hosting Dr. René Vidal (IEEE, ACM, AIMBE Fellow) from UPenn for an in-person talk at UTSA Main Campus (please see flyer attached as well as title/abstract/bio below). We will also try to broadcast the talk on Zoom (https://utsa.zoom.us/j/94807623288) for those that cannot make it in-person.

Please consider joining us in-person if your schedule permits. Also, please consider encouraging your students to attend, as well as spreading the word to any interested colleagues.

Title: “Learning Dynamics and Implicit Bias of Gradient Flow in Overparameterized Linear Models”

Abstract: Contrary to the common belief that overparameterization may hurt generalization and optimization, recent work suggests that overparameterization may bias the optimization algorithm towards solutions that generalize well, a phenomenon known as implicit regularization or implicit bias, and may also accelerate convergence, a phenomenon known as implicit acceleration. This talk will provide a detailed analysis of the dynamics of gradient flow in overparameterized two-layer linear models showing that convergence to equilibrium depends on the imbalance between input and output weights (which is fixed at initialization) and the margin of the initial solution. The talk will also provide an analysis of the implicit bias, showing that large hidden layer width, together with (properly scaled) random initialization, constrains the network parameters to converge to a solution which is close to the min-norm solution.

Bio: René Vidal is the Penn Integrates Knowledge and Rachleff University Professor of Electrical and Systems Engineering & Radiology and the Director of the Center for Innovation in Data Engineering and Science (IDEAS) at the University of Pennsylvania. He also directs THEORINET, an NSF-Simons Collaboration on the Mathematical Foundations of Deep Learning, and the NSF TRIPODS Institute on the Foundations of Graph and Deep Learning. He is also an Amazon Scholar, an Affiliated Chief Scientist at NORCE, and an Associate Editor-in-Chief of TPAMI. His current research focuses on the foundations of deep learning and trustworthy AI and its applications in computer vision and biomedical data science. He is an ACM Fellow, AIMBE Fellow, IEEE Fellow, IAPR Fellow and Sloan Fellow, and has received numerous awards for his work, including the IEEE Edward J. McCluskey Technical Achievement Award, D’Alembert Faculty Award, J.K. Aggarwal Prize, ONR Young Investigator Award, NSF CAREER Award as well as best paper awards in machine learning, computer vision, controls, and medical robotics.

Dr. Panagiotis (Panos) Markopoulos, Ph.D.

Associate Professor and Margie and Bill Klesse Endowed Professor

Department of Electrical & Computer Engineering and Department of Computer Science

Chair, Digital Signal Processing Concentration (ECE Dept.)

Founding Director, Machine Learning Optimization and Signal Processing (MELOS) Laboratory (@ San Pedro I)

Founding Director, Multi-modal Sensing and Signal Processing (MSSP) Laboratory (@ San Pedro II)

Core Faculty, UTSA MATRIX AI Consortium

The University of Texas at San Antonio

----------------------------------------------

Personal Web: https://www.markopoulos.org/

Lab Web: https://www.meloslab.org/

E-mail: panagiotis.markopoulos@utsa.edu (preferred)

Address: One UTSA Circle, San Antonio, TX 78249-3209