Author
Kevin M. Cury
Bio: Kevin M. Cury is an academic researcher from Columbia University. The author has contributed to research in topics: Component (thermodynamics) & Neuroscience. The author has an hindex of 3, co-authored 3 publications receiving 1463 citations.
Papers
More filters
TL;DR: Using a deep learning approach to track user-defined body parts during various behaviors across multiple species, the authors show that their toolbox, called DeepLabCut, can achieve human accuracy with only a few hundred frames of training data.
Abstract: Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, but markers are intrusive, and the number and location of the markers must be determined a priori. Here we present an efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data. We demonstrate the versatility of this framework by tracking various body parts in multiple species across a broad collection of behaviors. Remarkably, even when only a small number of frames are labeled (~200), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy.
2,303 citations
Posted Content•
TL;DR: This work presents a highly efficient method for markerless tracking based on transfer learning with deep neural networks that achieves excellent results with minimal training data and demonstrates the versatility of this framework by tracking various body parts in a broad collection of experimental settings.
Abstract: Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, yet markers are intrusive (especially for smaller animals), and the number and location of the markers must be determined a priori. Here, we present a highly efficient method for markerless tracking based on transfer learning with deep neural networks that achieves excellent results with minimal training data. We demonstrate the versatility of this framework by tracking various body parts in a broad collection of experimental settings: mice odor trail-tracking, egg-laying behavior in drosophila, and mouse hand articulation in a skilled forelimb task. For example, during the skilled reaching behavior, individual joints can be automatically tracked (and a confidence score is reported). Remarkably, even when a small number of frames are labeled ($\approx 200$), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy.
459 citations
TL;DR: Recent advances in the understanding of insect neural circuits that control when, where and how to lay an egg are reviewed.
Abstract: Egg-laying behavior is one of the most important aspects of female behavior, and has a profound impact on the fitness of a species. As such, it is controlled by several layers of regulation. Here, we review recent advances in our understanding of insect neural circuits that control when, where and how to lay an egg. We also outline outstanding open questions about the control of egg-laying decisions, and speculate on the possible neural underpinnings that can drive the diversification of oviposition behaviors through evolution.
50 citations
TL;DR: In this paper , the structure of the egg-laying behavioral sequence in Drosophila was characterized and significant variability in the transitions between component actions that affords the organism an adaptive flexibility.
Abstract: Innate behaviors are frequently comprised of ordered sequences of component actions that progress to satisfy essential drives. Progression is governed by specialized sensory cues that induce transitions between components within the appropriate context. Here we have characterized the structure of the egg-laying behavioral sequence in Drosophila and found significant variability in the transitions between component actions that affords the organism an adaptive flexibility. We identified distinct classes of interoceptive and exteroceptive sensory neurons that control the timing and direction of transitions between the terminal components of the sequence. We also identified a pair of motor neurons that enact the final transition to egg expulsion. These results provide a logic for the organization of innate behavior in which sensory information processed at critical junctures allows for flexible adjustments in component actions to satisfy drives across varied internal and external environments.
2 citations
Cited by
More filters
TL;DR: Over time, marked improvements in accuracy, productivity, and workflow will likely be actualized, but whether that will be used to improve the patient–doctor relationship or facilitate its erosion remains to be seen.
Abstract: The use of artificial intelligence, and the deep-learning subtype in particular, has been enabled by the use of labeled big data, along with markedly enhanced computing power and cloud storage, across all sectors. In medicine, this is beginning to have an impact at three levels: for clinicians, predominantly via rapid, accurate image interpretation; for health systems, by improving workflow and the potential for reducing medical errors; and for patients, by enabling them to process their own data to promote health. The current limitations, including bias, privacy and security, and lack of transparency, along with the future directions of these applications will be discussed in this article. Over time, marked improvements in accuracy, productivity, and workflow will likely be actualized, but whether that will be used to improve the patient-doctor relationship or facilitate its erosion remains to be seen.
2,574 citations
TL;DR: In this article, the authors observed that spontaneous activity reliably encoded a high-dimensional latent state, which was partially related to the mouse's ongoing behavior and was represented not just in visual cortex but also across the forebrain.
Abstract: Neuronal populations in sensory cortex produce variable responses to sensory stimuli and exhibit intricate spontaneous activity even without external sensory input. Cortical variability and spontaneous activity have been variously proposed to represent random noise, recall of prior experience, or encoding of ongoing behavioral and cognitive variables. Recording more than 10,000 neurons in mouse visual cortex, we observed that spontaneous activity reliably encoded a high-dimensional latent state, which was partially related to the mouse's ongoing behavior and was represented not just in visual cortex but also across the forebrain. Sensory inputs did not interrupt this ongoing signal but added onto it a representation of external stimuli in orthogonal dimensions. Thus, visual cortical population activity, despite its apparently noisy structure, reliably encodes an orthogonal fusion of sensory and multidimensional behavioral information.
844 citations
TL;DR: The Principles of Insect Morphology by R. E. Snodgrass as discussed by the authors is one of the most important works in the field of insect morphology, and it has been widely used in the literature.
Abstract: THE author of this book ranks as the foremost American worker on insect morphology. His contributions on the subject are notable for their clarity and originality of thought, and the appearance of a volume, embodying his ideas in comprehensive form, is sure of a hearty welcome. In its preparation, Mr. Snodgrass has incorporated the results of much first-hand study with those of many recent investigators in the same field. He has produced an outstanding book wherein knowledge of facts is combined with that of function and, at the same time, theoretical conceptions of the origins and relationships of organs and parts are not overlooked. Principles of Insect Morphology
By R. E. Snodgrass. (McGraw-Hill Publications in the Zoological Sciences.) Pp. ix + 667. (New York and London: McGraw-Hill Book Co., Inc., 1935.) 36s. net.
770 citations
TL;DR: The intersection between deep learning and cellular image analysis is reviewed and an overview of both the mathematical mechanics and the programming frameworks of deep learning that are pertinent to life scientists are provided.
Abstract: Recent advances in computer vision and machine learning underpin a collection of algorithms with an impressive ability to decipher the content of images. These deep learning algorithms are being applied to biological images and are transforming the analysis and interpretation of imaging data. These advances are positioned to render difficult analyses routine and to enable researchers to carry out new, previously impossible experiments. Here we review the intersection between deep learning and cellular image analysis and provide an overview of both the mathematical mechanics and the programming frameworks of deep learning that are pertinent to life scientists. We survey the field's progress in four key applications: image classification, image segmentation, object tracking, and augmented microscopy. Last, we relay our labs' experience with three key aspects of implementing deep learning in the laboratory: annotating training data, selecting and training a range of neural network architectures, and deploying solutions. We also highlight existing datasets and implementations for each surveyed application.
714 citations