David D. Fan

Bio: David D. Fan is an academic researcher from Georgia Institute of Technology. The author has contributed to research in topics: Optimal control & Trajectory optimization. The author has an hindex of 12, co-authored 34 publications receiving 592 citations. Previous affiliations of David D. Fan include Duke University & California Institute of Technology.

A Wireless Multi-Channel Recording System for Freely Behaving Mice and Rats

Abstract: To understand the neural basis of behavior, it is necessary to record brain activity in freely moving animals. Advances in implantable multi-electrode array technology have enabled researchers to record the activity of neuronal ensembles from multiple brain regions. The full potential of this approach is currently limited by reliance on cable tethers, with bundles of wires connecting the implanted electrodes to the data acquisition system while impeding the natural behavior of the animal. To overcome these limitations, here we introduce a multi-channel wireless headstage system designed for small animals such as rats and mice. A variety of single unit and local field potential signals were recorded from the dorsal striatum and substantia nigra in mice and the ventral striatum and prefrontal cortex simultaneously in rats. This wireless system could be interfaced with commercially available data acquisition systems, and the signals obtained were comparable in quality to those acquired using cable tethers. On account of its small size, light weight, and rechargeable battery, this wireless headstage system is suitable for studying the neural basis of natural behavior, eliminating the need for wires, commutators, and other limitations associated with traditional tethered recording systems.

Instrumental uncertainty as a determinant of behavior under interval schedules of reinforcement.

Abstract: Interval schedules of reinforcement are known to generate habitual behavior, the performance of which is less sensitive to revaluation of the earned reward and to alterations in the action-outcome contingency. Here we report results from experiments using different types of interval schedules of reinforcement in mice to assess the effect of uncertainty, in the time of reward availability, on habit formation. After limited training, lever pressing under fixed interval (FI, low interval uncertainty) or random interval schedules (RI, higher interval uncertainty) was sensitive to devaluation, but with more extended training, performance of animals trained under RI schedules became more habitual, i.e. no longer sensitive to devaluation, whereas performance of those trained under FI schedules remained goal-directed. When the press-reward contingency was reversed by omitting reward after pressing but presenting reward in the absence of pressing, lever pressing in mice previously trained under FI decreased more rapidly than that of mice trained under RI schedules. Further analysis revealed that action-reward contiguity is dramatically reduced in lever pressing under RI schedules, whereas action-reward correlation does not vary significantly. These data indicate that the extent of goal-directedness could vary as a function of temporal uncertainty.

Bayesian Learning-Based Adaptive Control for Safety Critical Systems

Abstract: Deep learning has enjoyed much recent success, and applying state-of-the-art model learning methods to controls is an exciting prospect. However, there is a strong reluctance to use these methods on safety-critical systems, which have constraints on safety, stability, and real-time performance. We propose a framework which satisfies these constraints while allowing the use of deep neural networks for learning model uncertainties. Central to our method is the use of Bayesian model learning, which provides an avenue for maintaining appropriate degrees of caution in the face of the unknown. In the proposed approach, we develop an adaptive control framework leveraging the theory of stochastic CLFs (Control Lyapunov Functions) and stochastic CBFs (Control Barrier Functions) along with tractable Bayesian model learning via Gaussian Processes or Bayesian neural networks. Under reasonable assumptions, we guarantee stability and safety while adapting to unknown dynamics with probability 1. We demonstrate this architecture for high-speed terrestrial mobility targeting potential applications in safety-critical high-speed Mars rover missions.

Mechanisms of Action Selection and Timing in Substantia Nigra Neurons

Abstract: The timing of actions is critical for adaptive behavior. In this study we measured neural activity in the substantia nigra as mice learned to change their action duration to earn food rewards. We observed dramatic changes in single unit activity during learning: both dopaminergic and GABAergic neurons changed their activity in relation to behavior to reflect the learned instrumental contingency and the action duration. We found the emergence of "action-on" neurons that increased firing for the duration of the lever press and mirror-image "action-off" neurons that paused at the same time. This pattern is especially common among GABAergic neurons. The activity of many neurons also reflected confidence about the just completed action and the prospect of reward. Being correlated with the relative duration of the completed action, their activity could predict the likelihood of reward collection. Compared with the GABAergic neurons, the activity of dopaminergic neurons was more commonly modulated by the discriminative stimulus signaling the start of each trial, suggesting that their phasic activity reflected sensory salience rather than any reward prediction error found in previous work. In short, these results suggest that (1) nigral activity is highly plastic and modified by the learning of the instrumental contingency; (2) GABAergic output from the substantia nigra can simultaneously inhibit and disinhibit downstream structures, while the dopaminergic output also provide bidirectional modulation of the corticostriatal circuits; (3) dopaminergic and GABAergic neurons show similar task-related activity, although DA neurons are more responsive to the trial start signal.

STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation

Abstract: Although ground robotic autonomy has gained widespread usage in structured and controlled environments, autonomy in unknown and off-road terrain remains a difficult problem. Extreme, off-road, and unstructured environments such as undeveloped wilderness, caves, and rubble pose unique and challenging problems for autonomous navigation. To tackle these problems we propose an approach for assessing traversability and planning a safe, feasible, and fast trajectory in real-time. Our approach, which we name STEP (Stochastic Traversability Evaluation and Planning), relies on: 1) rapid uncertainty-aware mapping and traversability evaluation, 2) tail risk assessment using the Conditional Value-at-Risk (CVaR), and 3) efficient risk and constraint-aware kinodynamic motion planning using sequential quadratic programming-based (SQP) model predictive control (MPC). We analyze our method in simulation and validate its efficacy on wheeled and legged robotic platforms exploring extreme terrains including an abandoned subway and an underground lava tube.

Neuronal Reward and Decision Signals: From Theories to Data

Abstract: Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions.

Abstract: Shifting between goal-directed and habitual actions allows for efficient and flexible decision making. Here we demonstrate a novel, within-subject instrumental lever-pressing paradigm, in which mice shift between goal-directed and habitual actions. We identify a role for orbitofrontal cortex (OFC) in actions following outcome revaluation, and confirm that dorsal medial (DMS) and lateral striatum (DLS) mediate different action strategies. Simultaneous in vivo recordings of OFC, DMS and DLS neuronal ensembles during shifting reveal that the same neurons display different activities depending on whether presses are goal-directed or habitual, with DMS and OFC becoming more and DLS less engaged during goal-directed actions. Importantly, the magnitude of neural activity changes in OFC following changes in outcome value positively correlates with the level of goal-directed behavior. Chemogenetic inhibition of OFC disrupts goal-directed actions, whereas optogenetic activation of OFC specifically increases goal-directed pressing. These results also reveal a role for OFC in action revaluation, which has implications for understanding compulsive behavior.

Animal Intelligence: Experimental Studies

Abstract: ONE of the most remarkable examples of sudden and rapid development of a new scientific method and a new and extensive body of scientific fact is to be seen in the growth of the study of animal psychology during the last ten or a dozen years. As in the case of the general science of psychology, the change came with the introduction of experiment as the fundamental method of investigation, but the transition was accentuated by a craving for objectivity of results, which focussed the attention upon the objective performance or behaviour of the animal under examination, not only to the detriment, but even, in the case of many observers, to the complete neglect of speculation as to its psychical life. If the new psychology claimed to be a psychology without a soul, the new animal psychology threatened, and still threatens, to become an animal psychology without consciousness. Many investigators have indeed openly declared for this ideal-not denying the presence of consciousness, but regarding it as of no importance or value in an explanatory scientific system. Nevertheless signs are not wanting in the most recent work of a healthy reaction from this extreme view, based as much upon observed fact as upon a priori speculation. Animal Intelligence: Experimental Studies. By E. L. Thorndike. Pp. viii + 297. (New York: The Macmillan Co.; London: Macmillan and Co., Ltd., 1911.) Price 7s. net.

Maintenance of persistent activity in a frontal thalamocortical loop

Abstract: Persistent neural activity maintains information that connects past and future events. Models of persistent activity often invoke reverberations within local cortical circuits, but long-range circuits could also contribute. Neurons in the mouse anterior lateral motor cortex (ALM) have been shown to have selective persistent activity that instructs future actions. The ALM is connected bidirectionally with parts of the thalamus, including the ventral medial and ventral anterior-lateral nuclei. We recorded spikes from the ALM and thalamus during tactile discrimination with a delayed directional response. Here we show that, similar to ALM neurons, thalamic neurons exhibited selective persistent delay activity that predicted movement direction. Unilateral photoinhibition of delay activity in the ALM or thalamus produced contralesional neglect. Photoinhibition of the thalamus caused a short-latency and near-complete collapse of ALM activity. Similarly, photoinhibition of the ALM diminished thalamic activity. Our results show that the thalamus is a circuit hub in motor preparation and suggest that persistent activity requires reciprocal excitation across multiple brain areas.