Digital twin can be defined as a virtual representation of a physical asset enabled through data and simulators for real-time prediction, optimization, monitoring, controlling, and improved decision making. Recent advances in computational pipelines, multiphysics solvers, artificial intelligence, big data cybernetics, data processing and management tools bring the promise of digital twins and their impact on society closer to reality. Digital twinning is now an important and emerging trend in many applications. Also referred to as a computational megamodel, device shadow, mirrored system, avatar or a synchronized virtual prototype, there can be no doubt that a digital twin plays a transformative role not only in how we design and operate cyber-physical intelligent systems, but also in how we advance the modularity of multi-disciplinary systems to tackle fundamental barriers not addressed by the current, evolutionary modeling practices. In this work, we review the recent status of methodologies and techniques related to the construction of digital twins mostly from a modeling perspective. Our aim is to provide a detailed coverage of the current challenges and enabling technologies along with recommendations and reflections for various stakeholders.

/pdf/digital-twin-values-challenges-and-enablers-from-a-modeling-3hpoelaogn.pdf

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

(2007). Stochastic Ageing and Dependence for Reliability. Technometrics: Vol. 49, No. 2, pp. 222-222.

Stochastic Ageing and Dependence for Reliability

Reliability engineering dates back to reliability studies in the 20th century; since then, various models have been defined and used. Software engineering plays a key role from several viewpoints, but the main concern is that we're moving toward a more connected world, including enterprises and mobile devices. The three articles in this special issue illustrate current trends in this domain.

Reliability Engineering

3 Non-linear Objective functions 4 3.1 Production problem with set-up costs . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.2 Piecewise linear cost function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3 Piecewise linear convex cost function . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.4 Disjunctive constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Integer Programming and Combinatorial Optimization

Energy systems undergo major transitions to facilitate the large-scale penetration of renewable energy technologies and improve efficiencies, leading to the integration of many sectors into the energy system domain. As the complexities in this domain increase, it becomes challenging to control energy flows using existing techniques based on physical models. Moreover, although data-driven models, such as reinforcement learning (RL), have gained considerable attention in many fields, a direct shift into RL is not feasible in the energy domain irrespective of the ongoing complexities. To this end, a top-down approach is used to understand this behavior by reviewing the current state of the art. We classified RL papers in the literature into seven categories based on their area of application. Subsequently, publications under each category were further examined relative to problem diversity, RL technique employed, performance improvement (compared with other white and gray box models), verification, and reproducibility; many of the articles reported a 10–20% performance improvement with the use of RL. In most studies, however, deep learning techniques and state-of-the-art actor-critic methods (e.g., twin delayed deep deterministic policy gradient and soft actor-critic) were not applied. This has remarkably hindered performance improvements and problems related to complex energy flows have not been considered. Approximately half of the publications reported the use of Q-learning. Furthermore, despite the availability of historical data in the energy system domain, batch RL algorithms have not been exploited. Emerging multi-agent RL applications may be considered as a positive development that can enable the management of complex interactions among multiple parties. Most studies lack proper benchmarking compared to model-based approaches or gray-box models, and a majority cover energy dispatch problems and building energy management. Although RL can adequately solve problems that are considerably integrated in several sectors, only a limited number of publications have discussed its broad application. The present study clearly demonstrates that even without the full utilization of RL capacity, this technique has a considerable potential in resolving the continuously increasing complexity within the energy system domain.

Applications of reinforcement learning in energy systems

https://re.public.polimi.it/bitstream/11311/1122896/1/A_Reinforcement_Learning_Framework_for_Optimal_Operation_and_Maintenance_of_Power_Grids1.pdf

A reinforcement learning framework for optimal operation and maintenance of power grids

https://hal.archives-ouvertes.fr/hal-01652191/document

Reliability model of a component equipped with PHM capabilities

Optimal allocation of prognostics and health management capabilities to improve the reliability of a power transmission network

A variety of prognostic and health management (PHM) algorithms have been developed in the last years and some metrics have been proposed to evaluate their performances. However, a general framework that allows us to quantify the benefit of PHM depending on these metrics is still lacking. We propose a general, time-variant, analytical model that conservatively evaluates the increase in system availability achievable when a component is equipped with a PHM system of known performance metrics. The availability model builds on metrics of literature and is applicable to different contexts. A simulated case study is presented concerning crack propagation in a mechanical component. A simplified cost model is used to compare the performance of predictive maintenance based on PHM with corrective and scheduled maintenance.

/pdf/availability-model-of-a-phm-equipped-component-12xs8a7lek.pdf

Availability Model of a PHM-Equipped Component

For maintenance of gas turbines (GTs) in oil and gas applications, capital parts are removed and replaced by parts of the same type taken from the warehouse When the removed parts are found not broken, they are repaired at the workshop and returned to the warehouse, ready to be used in future maintenance The management of this flow is of great importance for the profitability of a GT plant In this paper, we adopt a previously developed formalized framework of the part flow and reinforcement learning (RL) to optimize part flow management The formal framework and RL algorithm are extended to account for the stochastic failure process of the involved parts An application to a scaled-down case study derived from an industrial application is illustrated

/pdf/reinforcement-learning-based-flow-management-of-gas-turbine-3qp5h80p3t.pdf

Luca Bellani

Papers

A reinforcement learning framework for optimal operation and maintenance of power grids

Reliability model of a component equipped with PHM capabilities

Optimal allocation of prognostics and health management capabilities to improve the reliability of a power transmission network

Availability Model of a PHM-Equipped Component

Reinforcement learning-based flow management of gas turbine parts under stochastic failures