Showing papers by "Giovanni Pezzulo published in 2016"

Top-down models in biology: explanation and control of complex living systems above the molecular level.

Abstract: It is widely assumed in developmental biology and bioengineering that optimal understanding and control of complex living systems follows from models of molecular events. The success of reductionism has overshadowed attempts at top-down models and control policies in biological systems. However, other fields, including physics, engineering and neuroscience, have successfully used the explanations and models at higher levels of organization, including least-action principles in physics and control-theoretic models in computational neuroscience. Exploiting the dynamic regulation of pattern formation in embryogenesis and regeneration requires new approaches to understand how cells cooperate towards large-scale anatomical goal states. Here, we argue that top-down models of pattern homeostasis serve as proof of principle for extending the current paradigm beyond emergence and molecule-level rules. We define top-down control in a biological context, discuss the examples of how cognitive neuroscience and physics exploit these strategies, and illustrate areas in which they may offer significant advantages as complements to the mainstream paradigm. By targeting system controls at multiple levels of organization and demystifying goal-directed (cybernetic) processes, top-down strategies represent a roadmap for using the deep insights of other fields for transformative advances in regenerative medicine and systems bioengineering.

Active inference and robot control: a case study

Abstract: Active inference is a general framework for perception and action that is gaining prominence in computational and systems neuroscience but is less known outside these fields. Here, we discuss a proof-of-principle implementation of the active inference scheme for the control or the 7-DoF arm of a (simulated) PR2 robot. By manipulating visual and proprioceptive noise levels, we show under which conditions robot control under the active inference scheme is accurate. Besides accurate control, our analysis of the internal system dynamics (e.g. the dynamics of the hidden states that are inferred during the inference) sheds light on key aspects of the framework such as the quintessentially multimodal nature of control and the differential roles of proprioception and vision. In the discussion, we consider the potential importance of being able to implement active inference in robots. In particular, we briefly review the opportunities for modelling psychophysiological phenomena such as sensory attenuation and related failures of gain control, of the sort seen in Parkinson's disease. We also consider the fundamental difference between active inference and optimal control formulations, showing that in the former the heavy lifting shifts from solving a dynamical inverse problem to creating deep forward or generative models with dynamics, whose attracting sets prescribe desired behaviours.

Active Inference, epistemic value, and vicarious trial and error

Abstract: Balancing habitual and deliberate forms of choice entails a comparison of their respective merits-the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are "forward sweeps" of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)-the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.

Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi

Abstract: How do humans and other animals face novel problems for which predefined solutions are not available? Human problem solving links to flexible reasoning and inference rather than to slow trial-and-error learning. It has received considerable attention since the early days of cognitive science, giving rise to well known cognitive architectures such as SOAR and ACT-R, but its computational and brain mechanisms remain incompletely known. Furthermore, it is still unclear whether problem solving is a “specialized” domain or module of cognition, in the sense that it requires computations that are fundamentally different from those supporting perception and action systems. Here we advance a novel view of human problem solving as probabilistic inference with subgoaling. In this perspective, key insights from cognitive architectures are retained such as the importance of using subgoals to split problems into subproblems. However, here the underlying computations use probabilistic inference methods analogous to those that are increasingly popular in the study of perception and action systems. To test our model we focus on the widely used Tower of Hanoi (ToH) task, and show that our proposed method can reproduce characteristic idiosyncrasies of human problem solvers: their sensitivity to the “community structure” of the ToH and their difficulties in executing so-called “counterintuitive” movements. Our analysis reveals that subgoals have two key roles in probabilistic inference and problem solving. First, prior beliefs on (likely) useful subgoals carve the problem space and define an implicit metric for the problem at hand—a metric to which humans are sensitive. Second, subgoals are used as waypoints in the probabilistic problem solving inference and permit to find effective solutions that, when unavailable, lead to problem solving deficits. Our study thus suggests that a probabilistic inference scheme enhanced with subgoals provides a comprehensive framework to study problem solving and its deficits.

Prefrontal goal codes emerge as latent states in probabilistic value learning

Abstract: The prefrontal cortex PFC supports goal-directed actions and exerts cognitive control over behavior, but the underlying coding and mechanism are heavily debated. We present evidence for the role of goal coding in PFC from two converging perspectives: computational modeling and neuronal-level analysis of monkey data. We show that neural representations of prospective goals emerge by combining a categorization process that extracts relevant behavioral abstractions from the input data and a reward-driven process that selects candidate categories depending on their adaptive value; both forms of learning have a plausible neural implementation in PFC. Our analyses demonstrate a fundamental principle: goal coding represents an efficient solution to cognitive control problems, analogous to efficient coding principles in other e.g., visual brain areas. The novel analytical-computational approach is of general interest because it applies to a variety of neurophysiological studies.

Tracking and simulating dynamics of implicit stereotypes: A situated social cognition perspective.

Abstract: Adopting a situated social cognition perspective, we relied on different methodologies-1 computational and 3 empirical studies-to investigate social group-related specificities pertaining to implicit gender-domain stereotypes, as measured by a mouse-tracking adapted Implicit Association Test (IAT) and IAT(-like) tasks. We tested whether the emergence of implicit stereotypes was partially determined by associations congruent with the self, by visuospatial features of the task and subsequent competition at both sensorimotor and abstract levels. We tracked human and simulated artificial participants' hand movements among gender stereotypical (e.g., male engineers) and counterstereotypical (e.g., female engineers) social groups. In the computational study, data were simulated by a novel generative connectionist model integrating strengths from recent developments in embodied models of decision-making. Results support the self-congruency hypothesis and suggest the presence of competition at both levels. Discussion focuses on the generalizability of the self-congruency hypothesis and on the relevance of a situated perspective for implicit social cognition. (PsycINFO Database Record

Nonparametric Problem-Space Clustering: Learning Efficient Codes for Cognitive Control Tasks

Abstract: We present an information-theoretic method permitting one to find structure in a problem space (here, in a spatial navigation domain) and cluster it in ways that are convenient to solve different classes of control problems, which include planning a path to a goal from a known or an unknown location, achieving multiple goals and exploring a novel environment. Our generative nonparametric approach, called the generative embedded Chinese restaurant process (geCRP), extends the family of Chinese restaurant process (CRP) models by introducing a parameterizable notion of distance (or kernel) between the states to be clustered together. By using different kernels, such as the the conditional probability or joint probability of two states, the same geCRP method clusters the environment in ways that are more sensitive to different control-related information, such as goal, sub-goal and path information. We perform a series of simulations in three scenarios—an open space, a grid world with four rooms and a maze having the same structure as the Hanoi Tower—in order to illustrate the characteristics of the different clusters (obtained using different kernels) and their relative benefits for solving planning and control problems.

Differential effects of visual uncertainty and contextual guidance on perceptual decisions: Evidence from eye and mouse tracking in visual search.

Abstract: Visual search can be seen as a decision-making process that aims to assess whether a target is present or absent from a scene. In this perspective, eye movements collect evidence related to target detection and verification to guide the decision. We investigated whether, in real-world scenes, target detection and verification are differentially recruited in the decision-making process in the presence of prior information (expectations about target location) and perceptual uncertainty (noise). We used a mouse-tracking methodology with which mouse trajectories unveil components of decision-making and eye-tracking measures reflect target detection and verification. Indoor scenes were presented, including a target in usual or unusual locations or no target, and were degraded with additive noise (or no noise). Participants had to respond to the target's presence or absence. Degrading the scene delayed the decision due to increased verification times and reduced mouse velocity. Targets in unusual locations delayed the decision and deviated mouse trajectories toward the target-absent response. Detection times played a major role in these effects. Thus, target detection and verification processes influence decision-making by integrating the available sources of information differently and lead to an accumulation of evidence toward both the presence of a target and its absence.

Learning programs is better than learning dynamics

Abstract: Distributed and hierarchical models of control are nowadays popular in computational modeling and robotics. In the artificial neural network literature, complex behaviors can be produced by composing elementary building blocks or motor primitives, possibly organized in a layered structure. However, it is still unknown how the brain learns and encodes multiple motor primitives, and how it rapidly reassembles, sequences and switches them by exerting cognitive control. In this paper we advance a novel proposal, a hierarchical programmable neural network architecture, based on the notion of programmability and an interpreter-programmer computational scheme. In this approach, complex and novel behaviors can be acquired by embedding multiple modules motor primitives in a single, multi-purpose neural network. This is supported by recent theories of brain functioning in which skilled behaviors can be generated by combining functional different primitives embedded in ?reusable? areas of ?recycled? neurons. Such neuronal substrate supports flexible cognitive control, too. Modules are seen as interpreters of behaviors having controlling input parameters, or programs that encode structures of networks to be interpreted. Flexible cognitive control can be exerted by a programmer module feeding the interpreters with appropriate input parameters, without modifying connectivity. Our results in a multiple T -maze robotic scenario show how this computational framework provides a robust, scalable and flexible scheme that can be iterated at different hierarchical layers permitting to learn, encode and control multiple qualitatively different behaviors.

The effects of phonological similarity on the semantic categorisation of pictorial and lexical stimuli: evidence from continuous behavioural measures

Abstract: Behavioural and neuroimaging studies provide evidence of automatic activation of phonology (e.g., covert speech) during the recognition of lexical stimuli. Implicit processing of phonological information was investigated in a kinematic study of semantic categorisation of pictures (session A) or words (session B). Participants clicked one of two items (target and distractor) based on their semantic congruency (artefact or natural) with a cued-word. Phonological similarity between cued-word and distractor was varied. The presence of the phonological distractor produced trajectories with greater curvature towards the competing semantic category than did the presence of a distractor not phonologically related. This suggests that the semantic categorisation of pictorial and lexical stimuli is influenced by the automatic activation of phonological information. Trajectories’ curvature reveals competition between partially activated phonological and semantic representations suggesting that phonological co...

Acting up: An approach to the study of cognitive development

Abstract: Despite decades of research, we lack a comprehensive framework to study and explain cognitive development. The emerging " paradigm " of action-based cognition implies that cognitive development is an active rather than a passive, automatic, and self-paced maturational process. Importantly, " active " refers to both sensorimotor activity (in the narrow sense) as well as to autonomous exploration (e.g., as found in active perception or active learning). How does this emphasis on action affect our understanding of cog-nitive development? Can an action-based approach provide a much-needed integrative theory of cognitive development? This chapter reviews key factors that infl uence development (including sensorimo-tor skills as well as genetic, social, and cultural factors) and their associated brain mechanisms. Discussion focuses on how these factors can be incorporated into a comprehensive action-based framework. Challenges are highlighted for future research (e.g., problems associated with explaining higher-level cognitive abilities and devising novel experimental methodologies). Although still in its infancy, an action-based approach to cognitive development holds promise to improve scientifi c understanding of cognitive development and to impact education and technology.

Predictive Technologies: Can Smart Tools Augment the Brain's Predictive Abilities?

Abstract: The ability of “looking into the future” – namely, the capacity of anticipating future states of the environment or of the body – represents a fundamental function of human (and animal) brains. A goalkeeper who tries to guess the ball’s direction; a chess player who attempts to anticipate the opponent’s next move; or a man-in-love who tries to calculate what are the chances of her saying yes – in all these cases, people are simulating possible future states of the world, in order to maximize the success of their decisions or actions. Research in neuroscience is showing that our ability to predict the behaviour of physical or social phenomena is largely dependent on the brain’s ability to integrate current and past information to generate (probabilistic) simulations of the future. But could predictive processing be augmented using advanced technologies? In this contribution, we discuss how computational technologies may be used to support, facilitate or enhance the prediction of future events, by considering exemplificative scenarios across different domains, from simpler sensorimotor decisions to more complex cognitive tasks. We also examine the key scientific and technical challenges that must be faced to turn this vision into reality.

Toward mechanistic models of action-oriented and detached cognition.

Abstract: To be successful, the research agenda for a novel control view of cognition should foresee more detailed, computationally specified process models of cognitive operations including higher cognition. These models should cover all domains of cognition, including those cognitive abilities that can be characterized as online interactive loops and detached forms of cognition that depend on internally generated neuronal processing.

Efficacy of pulsed low-intensity electric neuromuscular stimulation in reducing pain and disability in patients with myofascial syndrome.

Abstract: Myofascial pain syndrome (MPS) is characterized by chronic pain in multiple myofascial trigger points and fascial constrictions. In recent years, the scientific literature has recognized the need to include the patient with MPS in a multidimensional rehabilitation project. At the moment, the most widely recognized therapeutic methods for the treatment of myofascial syndrome include the stretch and spray pressure massage. Microcurrent electric neuromuscular stimulation was proposed in pain management for its effects on normalizing bioelectricity of cells and for its sub-sensory application. In this study, we tested the efficacy of low-intensity pulsed electric neuromuscular stimulus (PENS) on pain in patients with MPS of cervical spine muscles. We carried out a prospective-analytic longitudinal study at an outpatient clinic during two weeks. Forty subjects (mean age 42±13 years) were divided into two groups: treatment (TrGr, n=20) and control group (CtrlGr, n=20). Visual-analog scale (VAS) values, concerning the spontaneous and movement-related pain in the cervical-dorsal region at baseline (T0) and at the end of the study (T1), showed a reduction from 7 to 3.81 (p < 0.001) in TrGr. In the CtrlGr, VAS was reduced from 8.2 to 7.2 (n.s.). Moreover, the pressure pain threshold at T0 was 2.1 vs 4.2 at T1 (p < 0.001) in TrG. In the CtrlGR we observed no significant changes. Modulated low-intensity PENS is an innovative therapy permitting to act on the transmission of pain and on the restoration of tissue homeostasis. It seems to affect the transmission of pain through the stimulation of A-beta fibers. The above results show that low-intensity PENS can be considered as an effective treatment to reduce pain and disability in patients with MPS.