scispace - formally typeset
SciSpace - Your AI assistant to discover and understand research papers | Product Hunt

Journal ArticleDOI

Modelling for Prediction vs. Modelling for Understanding: Commentary on Musso et al. (2013)

19 Dec 2013-Vol. 1, Iss: 2, pp 99-101

TL;DR: It is concluded that ANNs have high potential for theoretical and practical improvements in learning sciences and researchers in the learning sciences should prefer more theory-driven and parsimonious modelling techniques over ANNs whenever possible.

AbstractMusso et al. (2013) predict students’ academic achievement with high accuracy one year in advance from cognitive and demographic variables, using artificial neural networks (ANNs). They conclude that ANNs have high potential for theoretical and practical improvements in learning sciences. ANNs are powerful statistical modelling tools but they can mainly be used for exploratory modelling. Moreover, the output generated from ANNs cannot be fully translated into a meaningful set of rules because they store information about input-output relations in a complex, distributed, and implicit way. These problems hamper systematic theory-building as well as communication and justification of model predictions in practical contexts. Modern-day regression techniques, including (Bayesian) structural equation models, have advantages similar to those of ANNs but without the drawbacks. They are able to handle numerous variables, non-linear effects, multi-way interactions, and incomplete data. Thus, researchers in the learning sciences should prefer more theory-driven and parsimonious modelling techniques over ANNs whenever possible.

...read more

Content maybe subject to copyright    Report

Citations
More filters

Proceedings ArticleDOI
08 Jul 2018
TL;DR: This paper investigates the use of a hybrid model comprising multiple artificial neural networks with a final C4.5 decision tree classifier to investigate the potential of explaining the classification decision through production rules and the significant tree size questions the rule transparency to a human.
Abstract: The Artificial Neural Network is generally considered to be an effective classifier, but also a “Black Box” component whose internal behavior cannot be understood by human users. This lack of transparency forms a barrier to acceptance in high-stakes applications by the general public. This paper investigates the use of a hybrid model comprising multiple artificial neural networks with a final C4.5 decision tree classifier to investigate the potential of explaining the classification decision through production rules. Two large datasets collected from comprehension studies are used to investigate the value of the C4.5 decision tree as the overall comprehension classifier in terms of accuracy and decision transparency. Empirical trials show that higher accuracies are achieved through using a decision tree classifier, but the significant tree size questions the rule transparency to a human.

6 citations


Cites background from "Modelling for Prediction vs. Modell..."

  • ...[11] observed that the basic problems of communicating how they reach their conclusions in meaningful terms has yet to be solved....

    [...]


DissertationDOI
01 Jan 2017
Abstract: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I Zusammenfassung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III 1 General Introduction 1 1.1 History of Research on Scientific Thinking . . . . . . . . . . . . . . . . . . . . . . 4 1.2 The Present Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3 Methodological Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2 Psychometric Issues in Research on Scientific Reasoning 35 2.

3 citations


JournalDOI
30 Jan 2015
TL;DR: Two articles, Edelsbrunner and, Schneider (2013), and Nokelainen and Silander (2014) comment on Musso, Kyndt, Cascallar, and Dochy, with a perspective on its place among other predictive approaches.
Abstract: Two articles, Edelsbrunner and, Schneider (2013), and Nokelainen and Silander (2014) comment on Musso, Kyndt, Cascallar, and Dochy (2013). Several relevant issues are raised and some important clarifications are made in response to both commentaries. Predictive systems based on artificial neural networks continue to be the focus of current research and several advances have improved the model building and the interpretation of the resulting neural network models. What is needed is the courage and open-mindedness to actually explore new paths and rigorously apply new methodologies which can perhaps, sometimes unexpectedly, provide new conceptualisations and tools for theoretical advancement and practical applied research. This is particularly true in the fields of educational science and social sciences, where the complexity of the problems to be solved requires the exploration of proven methods and new methods, the latter usually not among the common arsenal of tools of neither practitioners nor researchers in these fields. This response will enrich the understanding of the predictive systems methodology proposed by the authors and clarify the application of the procedure, as well as give a perspective on its place among other predictive approaches.

3 citations


Cites background or methods or result from "Modelling for Prediction vs. Modell..."

  • ...Edelsbrunner and Schneider (2013) in their commentary on Musso, Kyndt, Cascallar and Dochy (2013) argue that artificial neural networks (ANNs) should only be used as exploratory modelling techniques, in spite of being powerful statistical modelling tools with demonstrated ability to improve…...

    [...]

  • ...The reasons Edelsbrunner and Schneider (2013) argue for their rather strong position are centred on two main arguments: (a) that the output from ANNs cannot be fully translated into a meaningful set of rules because of a lack of accessibility to the input-output relationships, and (b) that there is…...

    [...]

  • ...Therefore, contrary to what has been pointed out by Edelsbrunner and Schneider (2013) and quoted by Golino and Gomes (2014), the ANN approach offers the potential to examine the complex relationships amongst its components....

    [...]

  • ...Now it 69 | F L R expressed in Edelsbrunner & Schneider (2013)....

    [...]

  • ...The second main argument regarding problems associated with the ANN methodology, as claimed by Edelsbrunner and Schneider (2013), has to do with the lack of some statistical parameters in ANNs....

    [...]


Journal ArticleDOI
28 Apr 2014
Abstract: This commentary to the recent article by Musso et al. (2013) discusses issues related to model fitting, comparison of classification accuracy of generative and discriminative models, and two (or more) cultures of data modeling. We start by questioning the extremely high classification accuracy with an empirical data from a complex domain. There is a risk that we model perfect nonsense perfectly. Our second concern is related to the relevance of comparing multilayer perceptron neural networks and linear discriminant analysis classification accuracy indices. We find this problematic, as it is like comparing apples and oranges. It would have been easier to interpret the model and the variable (group) importance’s if the authors would have compared MLP to some discriminative classifier, such as group lasso logistic regression. Finally, we conclude our commentary with a discussion about the predictive properties of the adopted data modeling approach.

1 citations


01 Jan 2014
TL;DR: This commentary to the recent article by Musso et al. (2013) discusses issues related to model fitting, comparison of classification accuracy of generative and discriminative models, and two (or more) cultures of data modeling.
Abstract: This commentary to the recent article by Musso et al. (2013) discusses issues related to model fitting, comparison of classification accuracy of generative and discriminative models, and two (or more) cultures of data modeling. We start by questioning the extremely high classification accuracy with an empirical data from a complex domain. There is a risk that we model perfect nonsense perfectly. Our second concern is related to the relevance of comparing multilayer perceptron neural networks and linear discriminant analysis classification accuracy indices. We find this problematic, as it is like comparing apples and oranges. It would have been easier to interpret the model and the variable (group) importance’s if the authors would have compared MLP to some discriminative classifier, such as group lasso logistic regression. Finally, we conclude our commentary with a discussion about the predictive properties of the adopted data modeling approach.

1 citations


References
More filters

Book
01 Jan 1991
Abstract: Introduction Interactions between Continuous Predictors in Multiple Regression The Effects of Predictor Scaling on Coefficients of Regression Equations Testing and Probing Three-Way Interactions Structuring Regression Equations to Reflect Higher Order Relationships Model and Effect Testing with Higher Order Terms Interactions between Categorical and Continuous Variables Reliability and Statistical Power Conclusion Some Contrasts Between ANOVA and MR in Practice

27,846 citations


Journal ArticleDOI
Abstract: (1994). Multiple Regression: Testing and Interpreting Interactions. Journal of the Operational Research Society: Vol. 45, No. 1, pp. 119-120.

11,971 citations


"Modelling for Prediction vs. Modell..." refers methods in this paper

  • ...Like ANNs, modern regression techniques can account for non-linear relations (Bates & Watts, 2007) and complex interactions between variables (Aiken & West, 1991)....

    [...]


Book
01 Aug 1988
TL;DR: This book offers a balanced presentation of the theoretical, practical, and computational aspects of nonlinear regression and provides background material on linear regression, including the geometrical development for linear and nonlinear least squares.
Abstract: Wiley-Interscience Paperback Series The Wiley-Interscience Paperback Series consists of selected books that have been made more accessible to consumers in an effort to increase global appeal and general circulation. With these new unabridged softcover volumes, Wiley hopes to extend the lives of these works by making them available to future generations of statisticians, mathematicians, and scientists. "The authors have put together an extraordinary presentation of concepts and methods concerning the use and analysis of nonlinear regression models ...highly recommend[ed] ...for anyone needing to use and/or understand issues concerning the analysis of nonlinear regression models." -Technometrics "[This book] provides a good balance of relevant theory and application with many examples ...[and it] provides the most balanced approach to theory and application appropriate for a first course in nonlinear regression modeling for graduate statistics students." -Mathematical Reviews "[This book] joins a distinguished list of publications with a reputation for balancing technical rigor with readability, and theory with application. [It] upholds tradition ...[and is] a worthwhile reference for the marketing researcher with a serious interest in linear models. " -Journal of Marketing Research This book offers a balanced presentation of the theoretical, practical, and computational aspects of nonlinear regression and provides background material on linear regression, including the geometrical development for linear and nonlinear least squares. The authors employ real data sets throughout, and their extensive use of geometric constructs and continuing examples makes the progression of ideas appear very natural. The book also includes pseudocode for computing algorithms.

3,063 citations


"Modelling for Prediction vs. Modell..." refers methods in this paper

  • ...Like ANNs, modern regression techniques can account for non-linear relations (Bates & Watts, 2007) and complex interactions between variables (Aiken & West, 1991)....

    [...]


Book
01 Jan 1989
Abstract: From the Publisher: Combines the theoretical foundations of intelligent problem-solving with he data structures and algorithms needed for its implementation. The book presents logic, rule, object and agent-based architectures, along with example programs written in LISP and PROLOG. The practical applications of AI have been kept within the context of its broader goal: understanding the patterns of intelligence as it operates in this world of uncertainty, complexity and change. The introductory and concluding chapters take a new look at the potentials and challenges facing artificial intelligence and cognitive science. An extended treatment of knowledge-based problem-solving is given including model-based and case-based reasoning. Includes new material on: Fundamentals of search, inference and knowledge representation AI algorithms and data structures in LISP and PROLOG Production systems, blackboards, and meta-interpreters including planers, rule-based reasoners, and inheritance systems. Machine-learning including ID3 with bagging and boosting, explanation based learning, PAC learning, and other forms of induction Neural networks, including perceptrons, back propogation, Kohonen networks, Hopfield networks, Grossberg learning, and counterpropagation. Emergent and social methods of learning and adaptation, including genetic algorithms, genetic programming and artificial life. Object and agent-based problem solving and other forms of advanced knowledge representation

1,164 citations


Book
01 Jan 2012
TL;DR: This work focuses on the implementation of Structural Equation Modeling in R with the sem and OpenMx Packages and on the development of scale construction and development models for this and other applications.
Abstract: Part 1. Background. R. Hoyle, Introduction and Overview. R. Matsueda, Key Advances in the History of Structural Equation Modeling. M. Ho, S. Stark, O. Chernyshenko, Graphical Representation of Structural Equation Models Using Path Diagrams. K. Bollen, R. Hoyle, Latent Variables in Structural Equation Modeling. J. Pearl, The Causal Foundations of Structural Equation Modeling. D. Bandalos, P. Gagne, Simulation Methods in Structural Equation Modeling. Part 2. Fundamentals. R. Kline, Assumptions in Structural Equation Modeling. R. Hoyle, Model Specification in Structural Equation Modeling. D. Kenny, S. Milan, Identification: A Nontechnical Discussion of a Technical Issue. P. Lei, Q. Wu, Estimation in Structural Equation Modeling. T. Lee, L. Cai, R. MacCallum, Power Analysis for Tests of Structural Equation Models. M. Edwards, R. Wirth, C. Houts, N. Xi, Categorical Data in the Structural Equation Modeling Framework. S. West, A. Taylor, W. Wu, Model Fit and Model Selection in Structural Equation Modeling. C. Chou, J. Huh, Model Modification in Structural Equation Modeling. L. Williams, Equivalent Models: Concepts, Problems, Alternatives. Part 3. Implementation. P. Malone, J. Lubansky, Preparing Data for Structural Equation Modeling: Doing Your Homework. J. Graham, D. Coffman, Structural Equation Modeling with Missing Data. G. Hancock, M. Liu, Bootstrapping Standard Errors and Data-Model Fit Statistics in Structural Equation Modeling. B. Byrne, Choosing Structural Equation Modeling Computer Software: Snapshots of LISREL, EQS, Amos, and Mplus. J. Fox, J. Byrnes, S. Boker, M. Neale, Structural Equation Modeling in R with the sem and OpenMx Packages. A. Boomsma, R. Hoyle, A. Panter, The Structural Equation Modeling Research Report. Part 4. Basic Applications. T. Brown, M. Moore, Confirmatory Factor Analysis. R. Millsap, M. Olivera-Aguilar, Investigating Measurement Invariance Using Confirmatory Factor Analysis. S. Green, M. Thompson, A Flexible Structural Equation Modeling Approach for Analyzing Means. J. Cheong, D. MacKinnon, Mediation/Indirect Effects in Structural Equation Modeling. H. Marsh, Z. Wen, B. Nagengast, K. Hau, Structural Equation Models of Latent Interaction. J. Biesanz, Autoregressive Longitudinal Models. T. Raykov, Scale Construction and Development Using Structural Equation Modeling. Part 5. Advanced Applications. J. Bovaird, N. Koziol, Measurement Models for Ordered-Categorical Indicators. S. Rabe-Hesketh, A. Skrondal, X. Zheng, Multilevel Structural Equation Modeling. M. Shiyko, N. Ram, K. Grimm, An Overview of Growth Mixture Modeling: A Simple Nonlinear Application in OpenMx. J. McArdle, Latent Curve Modeling of Longitudinal Growth Data. P. Wood, Dynamic Factor Models for Longitudinally Intensive Data: Description and Estimation via Parallel Factor Models of Cholesky Decomposition. D. Cole, Latent Trait-State Models. E. Ferrer, H. Song, Longitudinal Structural Models for Assessing Dynamics in Dyadic Interactions. S. Franic, C. Dolan, D. Borsboom, D. Boomsma, Structural Equation Modeling in Genetics. A. McIntosh, A. Protzner, Structural Equation Models of Imaging Data. D.Kaplan, S. Depaoli, Bayesian Structural Equation Modeling. M. Wall, Spatial Structural Equation Modeling. G. Marcoulides, M. Ing, Automated Structural Equation Modeling Strategies.

759 citations