The New Encyclopaedia Britannica

The realization of military capability as a System of Systems (SoS), presents significant development challenges across technical, operational and programmatic dimensions. In particular, tools for deciding how to form and evolve SoS which consider performance and risk are lacking. This research leverages tools from financial engineering and operations research perspectives in portfolio optimization to assist decision making in this setting. Our approach facilitates evolutions of SoS architecture through a framework that supports architecture selection at a given decision-epoch of the evolutionary process. The approach models hierarchies of interdependent systems as generic nodes on a network that, subject to connectivity and compatibility constraints, work cohesively to fulfill overarching capability objectives. A robust portfolio algorithm is employed to address inherent real world issues of data uncertainty, inter-nodal performance and developmental risk. A naval warfare scenario illustrates application of the method to find “portfoliosˮ of systems from a candidate list of available systems. Results show how the framework effectively reduces the combinatorial complexity of tradespace exploration (e.g., connectivity rules, feasibility of solutions, optimality of solutions) by allowing the optimization problem to handle the mathematically intensive aspects of the decision-making process. As a result, human decision-makers can focus on choosing the appropriate weights for risk aversion in making final decisions.

A Robust Portfolio Optimization Approach to System of System Architectures

While system complexity is on the rise across many product lines, the resources required to successfully design and implement complex systems remain constrained. Because financiers of complex systems development efforts actively monitor project implementation cost, project performance models are needed to help project managers predict their cost compliance and avoid cost overruns. This article describes recent research conducted by the authors to develop a cost overrun predictive model using five known drivers of complex systems development cost. The study identifies schedule and reliability, as the key determinants of whether or not a large complex systems development project will experience cost overrun.

Developing a Cost Overrun Predictive Model for Complex Systems Development Projects

Systems of systems exhibit characteristics that pose difficulty in modelling and predicting their overall performance capabilities, including the presence of operational independence, emergent behaviour, and evolutionary development. When considering systems of systems within the autonomous defence systems context, these aspects become increasingly critical, as constraints on the performance of the final system are typically driven by hard constraints on space, weight and power. System execution modelling languages and tools permit early prediction of the performance of model-driven systems; however, the focus to date has been on understanding the performance of a model rather than determining whether it meets performance requirements, and only subsequently carrying out analysis to reveal the causes of any requirement violations. Moreover, such an analysis is even more difficult when applied to several systems cooperating to achieve a common goal—a system of systems. In this article, we propose an integrated approach to performance prediction of model-driven real-time embedded defence systems and systems of systems. Our architectural prototyping system supports a scenario-driven experimental platform for evaluating model suitability within a set of deployment and real-time performance constraints. We present an overview of our performance prediction system, demonstrating the integration of modelling, execution and performance analysis, and discuss a case study to illustrate our approach.

Model-driven performance prediction of systems of systems

System of systems uncertainty quantification using machine learning techniques with smart grid application

Technical Performance Measures TPMs historically are used to help the Program Manager in predicting if a program is on a path to achieve required performance. When extended over a developmental timeline, TPMs have provided a deterministic approach for predicting expected operational performance. However, TPMs are difficult to derive for a complex acknowledged system of systems SoS. An SoS Performance Measure SPM has been developed that is equivalent in purpose to TPMs and demonstrated for a deterministic state similar to TPM usage. However, reality indicates that many of the SoS component variables have significant uncertainty associated with them during the SoS development. Therefore, to more accurately account for the ability of an SoS to achieve its desired performance, this variability needs to be accounted for. This paper extends the deterministic SPM concept to a stochastic SPM to account for this uncertainty. An Antisubmarine Warfare ASW mission example is used, and demonstrates how this extension improves the effectiveness of the SPM methodology.

Incorporating a Measure of Uncertainty into Systems of Systems Development Performance Measures

This paper addresses the need for predicting performance in a system of systems (SoS) during development. Historically, technical performance measures (TPMs) along with modelling and simulation have been used by senior decision makers to predict if a system under development will meet the required performance needs. This methodology does not appear to be directly translatable to SoS’s for several reasons including the inherent complexity of the SoS and the operational flexibility the end user has in employing the SoS. An approach to dealing with the SoS performance prediction need is presented that addresses the use and integration of multiple technologies into a SoS and the decision maker’s options in the use of these technologies. This approach is used to develop a metric defined as a ‘SoS performance measure (SPM)’, an equivalent to a TPM for a SoS. An anti-submarine warfare mission construct is used to demonstrate this new metric.

A framework for performance prediction during development of systems of systems

: This paper addresses the need for predicting performance in a system of systems (SoS) during incremental development and for dealing with the inherent variability associated with predicting performance. Historically, senior decision-makers have used technical performance measures (TPM), along with modeling and simulation, to predict whether a system under development will meet performance requirements. This methodology does not appear to be directly translatable to SoS for several reasons, including the inherent complexity of the SoS and the operational flexibility the end user may have in employing the SoS. An approach for dealing with the SoS performance prediction has been presented previously. It laid out a notional approach to dealing with this issue. This approach has been generalized to address the use and integration of multiple technologies into an SoS and into the decision-maker's options in the use of these technologies that is rooted in using subject matter expert input and historical data. This methodology is used to develop a metric defined as an SoS performance measure (SPM), which serves as an equivalent in functionality to a TPM for a SoS. Similar to TPMs, an approach to developing tolerance bands is presented to be used for predicting the status of development as a function of time. The methodology is first presented as a deterministic method for predicting SoS performance during development. This method is then demonstrated using an example case to illustrate the methodology. However, many of the component variables have significant uncertainty associated with them during SoS development and integration into the SoS. The paper provides an approach for expanding the SPM concept to account for this uncertainty using a stochastic approach to address this issue.

https://calhoun.nps.edu/bitstream/10945/33862/3/NPS-AM-12-C9P02Z02-101.pdf

Richard Volkert

Papers

Incorporating a Measure of Uncertainty into Systems of Systems Development Performance Measures

A framework for performance prediction during development of systems of systems

Development and Extension of a Deterministic System of Systems Performance Prediction Methodology for an Acknowledged System of Systems