Computational model of precision grip in Parkinson's disease: a utility based approach.
TL;DR: The model is the first model of PG in PD conditions modeled using Reinforcement Learning with the significant difference that the action selection is performed using utility distribution instead of using purely Value-based distribution, thereby incorporating risk-based decision making.
Abstract: We propose a computational model of Precision Grip (PG) performance in normal subjects and Parkinson’s Disease (PD) patients. Prior studies on grip force generation in PD patients show an increase in grip force during ON medication and an increase in the variability of the grip force during OFF medication (Fellows et al 1998; Ingvarsson et al 1997). Changes in grip force generation in dopamine-deficient PD conditions strongly suggest contribution of the Basal Ganglia, a deep brain system having a crucial role in translating dopamine signals to decision making. The present approach is to treat the problem of modeling grip force generation as a problem of action selection, which is one of the key functions of the Basal Ganglia. The model consists of two components: 1) the sensory-motor loop component, and 2) the Basal Ganglia component. The sensory-motor loop component converts a reference position and a reference grip force, into lift force and grip force profiles, respectively. These two forces cooperate in grip-lifting a load. The sensory-motor loop component also includes a plant model that represents the interaction between two fingers involved in PG, and the object to be lifted. The Basal Ganglia component is modeled using Reinforcement Learning with the significant difference that the action selection is performed using utility distribution instead of using purely Value-based distribution, thereby incorporating risk-based decision making. The proposed model is able to account for the precision grip results from normal and PD patients accurately (Fellows et. al. 1998; Ingvarsson et. al. 1997). To our knowledge the model is the first model of precision grip in PD conditions.
Citations
More filters
TL;DR: It is argued that various motor symptoms in PD reflect dysfunction of neural structures responsible for action selection, motor sequencing, and coordination and execution of movement.
219 citations
Additional excerpts
...The model was able to replicate experimental findings, thereby providing valuable insights about the decision making dynamics involved in grip force selection (Gupta et al., 2013)....
[...]
01 Jan 2016
TL;DR: In this paper, the authors proposed a method to compute the probability of a given node having a negative value for a given value of 0, i.e., a node having no negative value is 0.
Abstract: Для числа ε > 0 и вещественной функции f на отрезке [a, b] обозначим через N(ε, f, [a, b]) супремум множества тех номеров n, для которых в [a, b] существует набор неналегающих отрезков [ai, bi], i = 1, . . . , n, таких, что |f(ai)− f(bi)| > ε для всех i = 1, . . . , n (sup ∅ = 0). Доказана следующая теорема: если {fj} – поточечно ограниченная последовательность вещественных функций на отрезке [a, b] такая, что n(ε) ≡ lim supj→∞N(ε, fj , [a, b]) < ∞ для любого ε > 0, то {fj} содержит подпоследовательность, которая всюду на [a, b] сходится к некоторой функции f такой, что N(ε, f, [a, b]) 6 n(ε) при любом ε > 0. Показано, что основное условие в этой теореме, связанное с верхним пределом, необходимо для равномерно сходящейся последовательности {fj} и “почти” необходимо для всюду сходящейся последовательности измеримых функций и что многие поточечные принципы выбора, обобщающие классическую теорему Хелли, вытекают из этой теоремы, а также приводятся примеры, иллюстрирующие ее точность. Библиография: 16 названий.
188 citations
Additional excerpts
...На сегодняшний день существуют разработки вычислительных моделей движений, затронутых БП, например, походки [3] или жеста захвата [4]....
[...]
TL;DR: A spiking network model to relate exploration to synchrony levels in the BG (which are a neural marker for tremor in Parkinson's disease) is built and shows that exploration levels could be controlled by STN's lateral connection strength which also influenced the synchrony Levels in the STN-GPe circuit.
Abstract: To make an optimal decision we need to weigh all the available options, compare them with the current goal, and choose the most rewarding one. Depending on the situation an optimal decision could be to either ‘explore’ or ‘exploit’ or ‘not to take any action’ for which the Basal Ganglia (BG) is considered to be a key neural substrate. In an attempt to expand this classical picture of BG function, we had earlier hypothesized that the Indirect Pathway (IP) of the BG could be the subcortical substrate for exploration. In this study we build a spiking network model to relate exploration to synchrony levels in the BG (which are a neural marker for tremor in Parkinson’s disease). Key BG nuclei such as the Sub Thalamic Nucleus (STN), Globus Pallidus externus (GPe) and Globus Pallidus internus (GPi) were modeled as Izhikevich spiking neurons whereas the Striatal output was modeled as Poisson spikes. The model is cast in reinforcement learning framework with the dopamine signal representing reward prediction error. We apply the model to two decision making tasks: a binary action selection task (similar to one used by Humphries et al. 2006) and an n-armed bandit task (Bourdaud et al. 2008). The model shows that exploration levels could be controlled by STN’s lateral connection strength which also influenced the synchrony levels in the STN-GPe circuit. An increase in STN’s lateral strength led to a decrease in exploration which can be thought as the possible explanation for reduced exploratory levels in Parkinson’s patients. Our simulations also show that on complete removal of IP, the model exhibits only Go and No-Go behaviors, thereby demonstrating the crucial role of IP in exploration. Our model provides a unified account for synchronization, action section, and explorative behavior.
55 citations
Cites background from "Computational model of precision gr..."
...…various BG functions ranging from action selection in continuous spaces (Krishnan et al., 2011), reaching movements (Magdoom et al., 2011), spatial navigation (Sukumar et al., 2012), precision grip (Gupta et al., 2013), and gait (Muralidharan et al., 2013) in normal and Parkinsonian conditions....
[...]
TL;DR: Computational modeling has the potential to become an invaluable tool to predict the onset of behavioral changes during disease progression and a significant decrease in sensitivity to punishment and risk was crucial for explaining behavioral changes observed in PD-ON ICD patients.
Abstract: Impulsivity, i.e. irresistibility in the execution of actions, may be prominent in Parkinson's disease (PD) patients who are treated with dopamine precursors or dopamine receptor agonists. In this study, we combine clinical investigations with computational modeling to explore whether impulsivity in PD patients on medication may arise as a result of abnormalities in risk, reward and punishment learning. In order to empirically assess learning outcomes involving risk, reward and punishment, four subject groups were examined: healthy controls, ON medication PD patients with impulse control disorder (PD-ON ICD) or without ICD (PD-ON non-ICD), and OFF medication PD patients (PD-OFF). A neural network model of the Basal Ganglia (BG) that has the capacity to predict the dysfunction of both the dopaminergic (DA) and the serotonergic (5HT) neuromodulator systems was developed and used to facilitate the interpretation of experimental results. In the model, the BG action selection dynamics were mimicked using a utility function based decision making framework, with DA controlling reward prediction and 5HT controlling punishment and risk predictions. The striatal model included three pools of Medium Spiny Neurons (MSNs), with D1 receptor (R) alone, D2R alone and co-expressing D1R-D2R. Empirical studies showed that reward optimality was increased in PD-ON ICD patients while punishment optimality was increased in PD-OFF patients. Empirical studies also revealed that PD-ON ICD subjects had lower reaction times (RT) compared to that of the PD-ON non-ICD patients. Computational modeling suggested that PD-OFF patients have higher punishment sensitivity, while healthy controls showed comparatively higher risk sensitivity. A significant decrease in sensitivity to punishment and risk was crucial for explaining behavioral changes observed in PD-ON ICD patients. Our results highlight the power of computational modelling for identifying neuronal circuitry implicated in learning, and its impairment in PD. The results presented here not only show that computational modelling can be used as a valuable tool for understanding and interpreting clinical data, but they also show that computational modeling has the potential to become an invaluable tool to predict the onset of behavioral changes during disease progression.
21 citations
TL;DR: A review of behavioral and neural studies on the associations between motor symptoms and cognitive deficits in PD, specifically akinesia/bradykinesia, tremor, gait, handwriting, precision grip, and speech production paves the way for providing a framework for understanding how treatment of cognitive dysfunction, for example cognitive rehabilitation programs, may in turn influence the motor symptoms of PD.
Abstract: Parkinson's disease (PD) is characterized by a range of motor symptoms. Besides the cardinal symptoms (tremor, bradykinesia/akinesia, and rigidity), PD patients also show other motor deficits, including gait disturbance, speech deficits, and impaired handwriting. However, along with these key motor symptoms, PD patients also experience cognitive deficits in attention, executive function, working memory, and learning. Recent evidence suggests that these motor and cognitive deficits of PD are not completely dissociable, as aspects of cognitive dysfunction can impact motor performance in PD. In this article, we provide a review of behavioral and neural studies on the associations between motor symptoms and cognitive deficits in PD, specifically akinesia/bradykinesia, tremor, gait, handwriting, precision grip, and speech production. This review paves the way for providing a framework for understanding how treatment of cognitive dysfunction, for example cognitive rehabilitation programs, may in turn influence the motor symptoms of PD.
20 citations
References
More filters
Book•
01 Sep 1988
TL;DR: In this article, the authors present the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields, including computer programming and mathematics.
Abstract: From the Publisher:
This book brings together - in an informal and tutorial fashion - the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields
Major concepts are illustrated with running examples, and major algorithms are illustrated by Pascal computer programs No prior knowledge of GAs or genetics is assumed, and only a minimum of computer programming and mathematics background is required
52,797 citations
Book•
01 Jan 1988TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
37,989 citations
"Computational model of precision gr..." refers background in this paper
...Striatal dopamine levels are thought to switch between DP and IP, since the DP (IP) is selected for higher (lower) levels of dopamine (Sutton and Barto, 1998; Frank, 2005; Wu et al., 2009) (Figure 5)....
[...]
TL;DR: In this paper, the authors present a critique of expected utility theory as a descriptive model of decision making under risk, and develop an alternative model, called prospect theory, in which value is assigned to gains and losses rather than to final assets and in which probabilities are replaced by decision weights.
Abstract: This paper presents a critique of expected utility theory as a descriptive model of decision making under risk, and develops an alternative model, called prospect theory. Choices among risky prospects exhibit several pervasive effects that are inconsistent with the basic tenets of utility theory. In particular, people underweight outcomes that are merely probable in comparison with outcomes that are obtained with certainty. This tendency, called the certainty effect, contributes to risk aversion in choices involving sure gains and to risk seeking in choices involving sure losses. In addition, people generally discard components that are shared by all prospects under consideration. This tendency, called the isolation effect, leads to inconsistent preferences when the same choice is presented in different forms. An alternative theory of choice is developed, in which value is assigned to gains and losses rather than to final assets and in which probabilities are replaced by decision weights. The value function is normally concave for gains, commonly convex for losses, and is generally steeper for losses than for gains. Decision weights are generally lower than the corresponding probabilities, except in the range of low prob- abilities. Overweighting of low probabilities may contribute to the attractiveness of both insurance and gambling. EXPECTED UTILITY THEORY has dominated the analysis of decision making under risk. It has been generally accepted as a normative model of rational choice (24), and widely applied as a descriptive model of economic behavior, e.g. (15, 4). Thus, it is assumed that all reasonable people would wish to obey the axioms of the theory (47, 36), and that most people actually do, most of the time. The present paper describes several classes of choice problems in which preferences systematically violate the axioms of expected utility theory. In the light of these observations we argue that utility theory, as it is commonly interpreted and applied, is not an adequate descriptive model and we propose an alternative account of choice under risk. 2. CRITIQUE
35,067 citations
Book•
01 Jan 1970
TL;DR: This comprehensive treatment of the analysis and design of continuous-time control systems provides a gradual development of control theory and shows how to solve all computational problems with MATLAB.
Abstract: From the Publisher:
This comprehensive treatment of the analysis and design of continuous-time control systems provides a gradual development of control theoryand shows how to solve all computational problems with MATLAB. It avoids highly mathematical arguments, and features an abundance of examples and worked problems throughout the book. Chapter topics include the Laplace transform; mathematical modeling of mechanical systems, electrical systems, fluid systems, and thermal systems; transient and steady-state-response analyses, root-locus analysis and control systems design by the root-locus method; frequency-response analysis and control systems design by the frequency-response; two-degrees-of-freedom control; state space analysis of control systems and design of control systems in state space.
6,634 citations
TL;DR: A model in which specific types of basal ganglia disorders are associated with changes in the function of subpopulations of striatal projection neurons is proposed, which suggests that the activity of sub Populations of Striatal projections neurons is differentially regulated by striatal afferents and that different striatal projections may mediate different aspects of motor control.
5,094 citations
"Computational model of precision gr..." refers background in this paper
...Striatal dopamine levels are thought to switch between Go and NoGo regimes (Albin et al., 1989)....
[...]