JM Li

Bio: JM Li is an academic researcher from Max Planck Society. The author has contributed to research in topics: Foraging. The author has an hindex of 1, co-authored 1 publications receiving 1 citations.

Internal State Dynamics Shape Brainwide Activity and Foraging Behavior

Abstract: The brain has persistent internal states that can modulate every aspect of an animal’s mental experience 1 – 4 . In complex tasks such as foraging, the internal state is dynamic 5 – 8 . Caenorhabditis elegans alternate between local search and global dispersal 5 . Rodents and primates exhibit trade-offs between exploitation and exploration 6 , 7 . However, fundamental questions remain about how persistent states are maintained in the brain, which upstream networks drive state transitions and how state-encoding neurons exert neuromodulatory effects on sensory perception and decision-making to govern appropriate behaviour. Here, using tracking microscopy to monitor whole-brain neuronal activity at cellular resolution in freely moving zebrafish larvae 9 , we show that zebrafish spontaneously alternate between two persistent internal states during foraging for live prey ( Paramecia ). In the exploitation state, the animal inhibits locomotion and promotes hunting, generating small, localized trajectories. In the exploration state, the animal promotes locomotion and suppresses hunting, generating long-ranging trajectories that enhance spatial dispersion. We uncover a dorsal raphe subpopulation with persistent activity that robustly encodes the exploitation state. The exploitation-state-encoding neurons, together with a multimodal trigger network that is associated with state transitions, form a stochastically activated nonlinear dynamical system. The activity of this oscillatory network correlates with a global retuning of sensorimotor transformations during foraging that leads to marked changes in both the motivation to hunt for prey and the accuracy of motor sequences during hunting. This work reveals an important hidden variable that shapes the temporal structure of motivation and decision-making. During foraging for live prey, zebrafish larvae alternate between persistent exploitation and exploration behavioural states that correlate with distinct patterns of neuronal activation.

Curiosity and the exploration-exploitation dilemma

Abstract: The exploration-exploitation dilemma is one of a few fundamental problems in the decision and life sciences. It has also been proven to be a mathematically intractable problem, when it is framed in terms of reward. To overcome this we have challenged the basic formulation of the problem itself, as having a single objective, namely reward. In its place we have defined independent objectives for exploration and exploitation. Through theory and numerical experiments we prove that a competition between exploration-as-curiosity and exploitation-for-reward has a tractable solution, based on the classic win-stay, lose-switch strategy from game theory. This strategy is possible because we treat information and reward as if they have equal value, and succeeds because the definition of curiosity we introduce is efficient. Besides offering a mathematical answer, this view of the problem seems more robust than the traditional approach because it succeeds in the difficult conditions where rewards are deceptive, or non-stationary.