A study of the relationship between decision model naturalness and performance

TLDR: The results suggest that naturalness and performance are differentially sensitive to task contingencies and that conceptual ease of use may be an unreliable predictor of a DSS's effect on performance.

Abstract: Two objectives in the design of decision support systems (DSS) are to improve decision-making performance and to use DSS modeling forms that are natural, that is, to adopt modeling paradigms that are congruent with decision makers' conceptual models of decision tasks. By accomplishing the latter objective, a DSS should enjoy better conceptual ease of use and face validity. However, past research finds that DSS deemed natural for a task by decision makers, DSS designers, and researchers alike, often do not improve (or even hinder) performance; the inverse also occurs. Further decision-making behavior seems quite sensitive to minor task differences. How reliably are decision model natural ness and performance related? This study utilizes the bootstrapping paradigm of psychological research to help answer this question. In assessing the naturalness and performance of differing model paradigms over time and across levels of task complexity, no single, systematic pattern emerges. But the results suggest that naturalness and performance are differentially sensitive to task contingencies. For example, while relative performance is stable over time only in the low complexity condition, relative naturalness is stable over time only int the intermediate complexity condition. One implication of the results is that conceptual ease of use may be an unreliable predictor of a DSS's effect on performance. DSS mechanisms may help decision makers better analyze model naturalness and performance.