Friday, November 15, 2024, 3:30 – 4:30pm

We demonstrate that fundamental aspects of astrocyte morphology and physiology naturally lead to a dynamic, high-capacity associative memory system. These neuron-astrocyte networks are closely related to two popular machine learning architectures: Transformers and modern Hopfield networks. In the context of associative memory, we show that neuron-astrocyte networks follow superior, supralinear memory scaling laws, outperforming all known biological implementations of modern Hopfield networks. In the context of Transformers, we show that neuron-astrocyte interactions provide a natural biological substrate for building self-attention, the core operation in Transformers.

—

Leo Kozachkov is the 2024-2025 Goldstein Fellow at IBM Research, where he works on theory-driven AI and bio-inspired computing. He earned his Ph.D. in Brain and Cognitive Sciences from MIT in 2022 and his B.S. in Physics from Rutgers University–New Brunswick in 2016. In Fall 2025, he will join Brown University as a tenure-track Assistant Professor of Engineering and an Assistant Professor of Brain Science at the Carney Institute for Brain Science. His research group will focus on understanding dynamics, control, and computation in natural and artificial systems.

In Person Group Viewing and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Baker Hall 340A and Zoom
Speaker's Name: LEO KOZACHKOV
Speaker Website: kozleo.github.io
Speaker's Professional Title: Goldstine Fellow, IBM T.J. Watson Research Research
Talk Title: Building Self-Attention with Tripartite Synapses
For More Information: rdkeller@cs.cmu.edu
Affiliations: Computer Science Department (CSD), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI)
Organization(s): Neuroscience Institute