Dezhe Jin Group

Department of Physics, Penn State University

Research Overview

The brain, composed of a vast number of intricately connected neurons, is one of the most sophisticated dynamical systems in nature. Understanding how the brain computes is at the forefront of contemporary science. Our group studies biological neural networks through theoretical analysis and computational modeling, working in close collaboration with experimental groups.

Computational model of birdsong syntax

Songbirds are accessible model systems for studying vocal communication. Male songbirds learn to sing from their fathers and use their songs to attract females. Birdsong consists of sequences of stereotyped syllables, which can be largely fixed, as in the songs of the zebra finch, or variable, as in the songs of the Bengalese finch. The syllable sequences of variable songs follow probabilistic rules known as birdsong syntax. However, the neural mechanisms underlying birdsong syntax remain poorly understood (PDF).

Single-unit recordings in singing zebra finches have revealed that projection neurons in HVC (used as a proper name), a sensorimotor nucleus of the song system, spike sequentially with precise timing relative to the song. A simple explanation is that the projection neurons form chain networks that support synfire chain activity. We examined this mechanism in detail with computational models and found that such network activity is prone to instability unless individual neurons have active dendritic processes that normalize the spiking activity between successive groups in the chain (PDF). This prediction was later confirmed by intracellular recordings of projection neurons in singing birds, which also strongly supported the chain network hypothesis (PDF). More recently, with experimental collaborators, we showed that local axonal conduction shapes the spatiotemporal properties of the neural sequences in HVC (PDF). We have also studied how the addition of new neurons leads to the emergence of a local circuit for precise timing (PDF), and how the timing network in HVC remains robust to changes in temperature (PDF).

We extended the chain network model to generate variable birdsong sequences. In this model, each chain network encodes a single syllable, and the end of one chain connects to the beginnings of multiple chains, creating branching connectivity. At a branching point, spike activity flows probabilistically into one of the connected chains, with the selection governed by winner-take-all competition mediated by inhibitory interneurons. The transition probabilities between chains are determined by the strengths of the connections to the branches, as well as by external inputs from the thalamic nucleus and by auditory feedback (PDF).

The model predicted that syllable transitions would follow Markovian statistics. To test this prediction, we analyzed the song syntax of the Bengalese finch. The analysis largely supported the prediction but pointed to two significant modifications. First, variable birdsong often contains long runs of repeated syllables, which do not follow Markovian dynamics. Through computational analysis, we identified the source of these non-Markovian repeats as stimulus-specific adaptation of auditory feedback to HVC: as a syllable repeats, the feedback weakens, reducing the probability of further repetition. The model also predicted that deafening would reduce syllable repetition, a prediction confirmed by experimental data (PDF).

Second, the findings suggested that multiple chains could encode the same syllable. Although indirect evidence supports this idea, direct experimental verification is still pending. With these modifications, the network model corresponds to a partially observable Markov model with adaptation, which accurately describes the song syntax of the Bengalese finch (PDF). We later developed statistical tests for inferring partially observable Markov models from song data, which revealed context-dependent syllable transitions in Bengalese finch songs (PDF; code). Experiments manipulating the temperature of HVC further implicate this nucleus in birdsong syntax (PDF).

We expect our computational model of birdsong syntax to contribute to a broader understanding of syntactic structure and of the neural mechanisms underlying vocalizations in other animal species (PDF, PDF).

Dynamics of spiking neural networks

Neurons interact through discrete spikes. In certain regimes, network dynamics can be approximated by rate models, in which interactions between neurons are described in terms of their firing rates. The resulting network equations are continuous; the well-known Hopfield model is one example. However, rate models leave out the possibility that the discreteness of spiking interactions leads to unique network properties. We took up this challenge and analyzed the spiking dynamics of leaky integrate-and-fire neurons in the pulse-coupled regime. We developed a novel nonlinear mapping technique to analyze such networks mathematically. We proved that when the network is dominated by feedback inhibition and the neurons are driven by constant external inputs, the network dynamics flow into spike sequence attractors from any initial condition and for arbitrary connectivity between the neurons, regardless of inhomogeneity in neuron properties and external drives. The attractors are characterized by precise spike timing. In small networks, the spike sequence attractors are periodic spiking patterns, and convergence to them requires only a few transient spikes. Our theory suggests that stable spike sequences are ubiquitous in spiking neural networks (PDF).

A special case of this theory is winner-take-all dynamics between competing neurons. Our analysis showed that winner-take-all dynamics requires very few transient spikes; in certain regimes, whichever neuron spikes first is the winner, with no transient dynamics at all. Winner-take-all dynamics is one of the most important mechanisms for decision-making and object recognition. Although such dynamics also exists in rate models, the transients there are often long, leading to objections that recurrent dynamics cannot explain phenomena such as fast object recognition in the visual system. Our analysis of spiking networks clarified these misconceptions (PDF).

Using our mapping technique, we further proved the existence of winner-take-all competition between chain networks, which is the basis of our computational model of variable birdsong syntax (PDF). The technique also led to an efficient simulation method, with which we demonstrated the formation of chain networks through synaptic plasticity and spontaneous activity (PDF, PDF).

Auditory object recognition and robust speech recognition

Humans and animals can recognize auditory objects such as speech or conspecific vocalizations despite noise and other interfering sounds. This robustness has long been difficult for artificial speech recognition algorithms to match; they often fail in noisy settings such as loud bars. We examined the possibility that sparse coding, often observed in the auditory system and other sensory modalities, contributes to noise robustness. We developed an algorithm for training detectors that respond to features of speech signals within small time windows. Driven by speech, the feature detectors produce sparse spatiotemporal spike patterns, and speech can be recognized by matching these patterns to stored templates. We demonstrated that this scheme outperformed the state-of-the-art speech recognition systems of the time on the standard task of recognizing spoken digits in noise, especially when the noise level was comparable to that of the signal. Our results suggest that sparse spike coding may be crucial for the robustness of the auditory system (PDF). The method is protected by a US patent (Google Patents).

The spike sequences generated by the feature detectors can be recognized by a network of neurons with stable transient plateau potentials, or UP states, which are often observed in the dendrites of pyramidal neurons. The state of the network is defined by which neurons are in the UP state. Transitions between network states are driven by inputs from the feature detectors and by the connectivity between neurons, so different inputs drive the network into different states. An auditory object can thus be recognized by identifying the network state reached in response to the auditory input (PDF, PDF).

Building on these ideas, we recently showed that auditory objects can be recognized from continuous, ultra-sparse spike streams in neuromorphic devices, pointing toward energy-efficient, brain-inspired AI (arXiv).

Neural coding in the basal ganglia

The basal ganglia are critical for motor control and learning and are extensively connected with many areas of the brain. The striatum is the input station of the basal ganglia. It is targeted by dopamine signals, which serve as reward signals for reinforcement learning of implicit motor skills and sensorimotor associations. The striatum is therefore believed to be a key structure for reinforcement learning. Temporal difference learning is a standard reinforcement learning mechanism. It explains how delayed rewards can be credited to the actions or sensory inputs that occurred earlier and eventually led to the rewards. The mechanism requires populations of neurons that fire sequentially in the interval between those actions or inputs and the rewards. However, whether such dynamics exist in the striatum was unknown.

We analyzed thousands of neurons recorded in the striatum and the prefrontal cortex of monkeys performing a simple visually guided saccade task. We used a clustering technique to categorize neuronal response profiles and found that neurons in both structures encoded all aspects of the task, including the visual signals on the screen and the motor signals generated by the subjects. The timing of these responses was dispersed. Most interestingly, we found a subset of neurons in the striatum and the prefrontal cortex that responded with single peaks at different delays after the onset of the visual signals. These neurons thus formed sequential firing patterns that filled the gaps between visual inputs. From the population of neurons in both structures, every time point during the task could be precisely decoded. Our results suggest that time is encoded in the dispersed response profiles of neuronal populations in the prefrontal cortex and the striatum. Furthermore, the sequential firing conjectured by temporal difference learning does exist in the striatum, supporting the possibility that this mechanism guides reinforcement learning in the basal ganglia (PDF).

Population coding in the visual cortex

The tilt aftereffect is a visual illusion: prolonged exposure to gratings or bars in one orientation makes bars in other orientations appear rotated away from the adapting orientation. The neural mechanism of such "repulsive" effects of visual adaptation was unknown. Single-unit recordings in the cat primary visual cortex revealed that adaptation to a single orientation changes the tuning properties of neurons: their preferred orientations shift away from the adapting orientation, and their response magnitudes decrease. At first glance, the repulsive shifts of the preferred orientations seem to explain the tilt aftereffect. We analyzed a population-coding model of the visual cortex and showed that the opposite is usually true: repulsive shifts of the preferred orientations alone lead to an attractive shift in perceived orientation, the opposite of the tilt aftereffect. The repulsive perceptual shift occurs only when the suppression of neural responses near the adapting orientation is strong enough. We quantified the shifts in preferred orientation and the suppression of responses in neurons recorded in the primary visual cortex, and their combined effects quantitatively matched the typical magnitude of the tilt aftereffect. Our analysis revealed the importance of the interplay between shifts in preferred orientation and response suppression, and suggested that these two effects tend to cancel each other, preserving perceptual fidelity under normal conditions. Prolonged exposure to a single orientation breaks this balance, leading to perceptual errors manifested as the tilt aftereffect (PDF).

Reconstruction of neuron morphology from microscopic images

We have developed ShuTu (Chinese for "dendrite"), a software platform for semi-automated reconstruction of neuronal morphology (PDF). It is designed for neurons that are filled with biocytin and stained after patch-clamp recording. With an additional preprocessing step, it can also handle fluorescence images from confocal stacks.

Even when the imaged volume contains only a single neuron, automated reconstruction is difficult because of background noise. ShuTu therefore supports a two-step process: automated reconstruction followed by manual correction. The software uses sophisticated algorithms for the automated step and provides a convenient user interface for efficiently completing the reconstruction through manual annotation.

Neural networks for quantum many-body physics

We also apply ideas from neural networks to quantum physics. Neural-network wave functions are highly expressive representations of quantum many-body states; we developed ways to tame this expressiveness so that the networks converge robustly to quantum many-body states (arXiv). In addition, we contributed to the development of high-resolution angle-resolved photoemission spectroscopy with a tunable magnetic field (Link).

People

Faculty

  • Dezhe Jin, Associate Professor of Physics

Graduate Students

  • Derek Sederman

Undergraduate Students

  • Benjamin Rider
  • Isaiah Manzo-Wendel

Former Members

  • Aayush Khare (PhD, 2026)
  • Autumn Hendricks (PhD, 2025)
  • Kevin Sargent (PhD, 2024)
  • Jiali Lu (PhD, 2023)
  • Leonardo Tavares (PhD, 2022)
  • Yevhen Tupikov (PhD, 2019)
  • Sumithra Surendralal (PhD, 2016)
  • Phillip Schafer (PhD, 2014)
  • Jason Wittenbach (PhD, 2014)
  • Aaron Miller (PhD, 2012)
  • David Fraser (PhD, 2009)
  • Linli Wang (PhD, 2009)
  • Joseph Jun (postdoc, 2004-2006)
  • Wonil Chang (Visitor from KAIST, Korea, 2006-2007)

Former Undergraduates

  • Tristan Gyure (Undergraduate student, 2024-2025)
  • Nikita Kiselov (Undergraduate student, 2024-2025)
  • XiaoMing Zhao (Undergraduate, Lafayette College, Summer 2024)
  • Kaisha Garvin-Darby (Undergraduate student, 2023-2024)
  • Madison Gillner (Undergraduate student, 2023-2024)
  • Zhiruo Zhang (Undergraduate student, Summer 2022)
  • Justin Kim (Undergraduate student, 2022)
  • Austin Marcus (Undergraduate student, 2019-2020)
  • Sarah Greberman (Undergraduate REU student, Summer 2019)
  • Xiaofun Mou (Undergraduate honors thesis, 2018-2019)
  • Guangda Shi (Undergraduate student from Franklin and Marshall College, 2018)
  • Zixin Tang (Undergraduate student, 2016-2017)
  • Collin Van Son (Undergraduate student, 2017)
  • Yosuke Ota (Undergraduate student, 2013)
  • Tarik Salameh (Undergraduate student, 2011-2012)
  • David Vidmar (Undergraduate honors thesis, 2011-2012)
  • David Van Maaden (Undergraduate honors thesis, 2008-2009)
  • Greg Diehl (Undergraduate student, 2006-2007)

Courses

Publications