Event Details:
Join us at Gunn Rotunda in the Stanford Neurosciences Building to learn about the latest cutting-edge, cross-disciplinary brain research, from biochemistry to behavior and beyond
Join the speaker for coffee, cookies, and conversation before the talk, starting at 11:45am.
The Neural Race Reduction: Feature learning dynamics in deep architectures
Abstract
What is the relationship between task geometry, network architecture, and emergent feature learning dynamics in nonlinear deep networks? I will describe the Gated Deep Linear Network framework, which schematizes how pathways of information flow impact learning dynamics within an architecture. Because of the gating, these networks can compute nonlinear functions of their input. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. The reduction takes the form of a neural race with an implicit bias towards shared representations, which then govern the model’s ability to systematically generalize, multi-task, and transfer. We show how appropriate network architectures can help factorize and abstract knowledge. Together, these results begin to shed light on the links between architecture and network performance.
Andrew Saxe, Ph.D.
University College London
Bio
Dr Andrew Saxe is a Professor of Machine Learning and Theoretical Neuroscience and Joint Group Leader at the Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre at UCL. His research focuses on the theory of deep learning and its applications to phenomena in neuroscience and psychology. His work has been recognised by the Schmidt Science Polymath and UK Blavatnik Finalist in Life Sciences awards.
Hosted by - Julia Costacurta (Linderman Lab)
About the Wu Tsai Neuro Seminar Series
The Wu Tsai Neurosciences Institute seminar series brings together the Stanford neuroscience community to discuss cutting-edge, cross-disciplinary brain research, from biochemistry to behavior and beyond.
Topics include new discoveries in fundamental neurobiology; advances in human and translational neuroscience; insights from computational and theoretical neuroscience; and the development of novel research technologies and neuro-engineering breakthroughs.
Unless otherwise noted, seminars are held Thursdays at 12:00 noon PT.