Weaving together theoretical physics, machine learning and neuroscience (Surya Ganguli, Stanford Applied Physics Colloquium)

Stanford University

APPLIED PHYSICS/PHYSICS COLLOQUIUM

Tuesday, October 5, 2021

4:30 p.m. (PT) via Zoom 

https://stanford.zoom.us/j/96589239251?pwd=T1pVUTlkQXBHa0NkdmRkOXAyYnVBQT09 
    Password: 740805

Surya Ganguli
Stanford University

“Weaving together theoretical physics, machine learning and neuroscience: a tale of neurons, atoms and photons in the service of computation”

We are witnessing an exciting interplay between physics, computation and neurobiology that spans in multiple directions.  In one direction we can use the power of complex systems analysis, developed in theoretical physics and applied mathematics, to elucidate design principles governing how neural networks, both biological and artificial, can learn and function. In another direction, we can exploit novel physics to instantiate and analyze new kinds of quantum neuromorphic computers built using atomic spins and photons. We will give several vignettes in both directions, including:  (1) deriving the detailed structure of the primate retina from first principles by developing optimal neural networks for processing natural movies, (2) using dynamic mean field theory to understand and optimize the training of deep neural networks used in machine learning, (3) understanding the geometry and dynamics of high dimensional optimization in the classical limit of a dissipative many-body quantum optimizer comprised of interacting photons.

References:

1. Y. Bahri, J. Kadmon, J. Pennington, S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annual Reviews of Condensed Matter Physics, 2020.  
2. M. Advani, S. Lahiri and S. Ganguli, Statistical mechanics of complex neural systems and high dimensional data, Journal of Statistical Mechanics Theory and Experiment (2013), P03014. 
3. S. Deny, J. Lindsey, S. Ganguli, S. Ocko, The emergence of multiple retinal cell types through efficient coding of natural movies, Neural Information Processing Systems (NeurIPS) 2018.
4. B. Poole, S. Lahiri, M. Raghu, J. Sohl-Dickstein, and S. Ganguli, Exponential expressivity in deep neural networks through transient chaos, Neural Information Processing Systems (NIPS) 2016.
5. J. Pennington, S. Schloenholz, and S. Ganguli, Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice, Neural Information Processing Systems (NIPS) 2017.
6. Y. Yamamoto, T. Leleu, S. Ganguli and H. Mabuchi, Coherent Ising Machines: quantum optics and neural network perspectives, Applied Physics Letters 2020.
7. B.P. Marsh, Y, Guo, R.M. Kroeze, S. Gopalakrishnan, S. Ganguli, J. Keeling, B.L. Lev, Enhancing associative memory recall and storage capacity using confocal cavity QED, Physical Review X, 2020

For recordings of past colloquia, click on the colloquium title on the AP/Physics colloquium page for recent colloquia or on the archive page for earlier colloquia.