There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

Why most published research findings are false

Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), path analysis, and structural equation modeling (SEM) have long histories in clinical research. Although CFA has largely superseded EFA, CFAs of multidimensional constructs typically fail to meet standards of good measurement: goodness of fit, measurement invariance, lack of differential item functioning, and well-differentiated factors in support of discriminant validity. Part of the problem is undue reliance on overly restrictive CFAs in which each item loads on only one factor. Exploratory SEM (ESEM), an overarching integration of the best aspects of CFA/SEM and traditional EFA, provides confirmatory tests of a priori factor structures, relations between latent factors and multigroup/multioccasion tests of full (mean structure) measurement invariance. It incorporates all combinations of CFA factors, ESEM factors, covariates, grouping/multiple-indicator multiple-cause (MIMIC) variables, latent growth, and complex structures that typically have required CFA/SEM. ESEM has broad applicability to clinical studies that are not appropriately addressed either by traditional EFA or CFA/SEM.

/pdf/exploratory-structural-equation-modeling-an-integration-of-ycnhyyhfme.pdf

Exploratory Structural Equation Modeling: An Integration of the Best Features of Exploratory and Confirmatory Factor Analysis

The introduction of connected and autonomous vehicles will bring changes to the highway driving environment. Connected vehicle technology provides real-time information about the surrounding traffic condition and the traffic management center’s decisions. Such information is expected to improve drivers’ efficiency, response, and comfort while enhancing safety and mobility. Connected vehicle technology can also further increase efficiency and reliability of autonomous vehicles, though these vehicles could be operated solely with their on-board sensors, without communication. While several studies have examined the possible effects of connected and autonomous vehicles on the driving environment, most of the modeling approaches in the literature do not distinguish between connectivity and automation, leaving many questions unanswered regarding the implications of different contemplated deployment scenarios. There is need for a comprehensive acceleration framework that distinguishes between these two technologies while modeling the new connected environment. This study presents a framework that utilizes different models with technology-appropriate assumptions to simulate different vehicle types with distinct communication capabilities. The stability analysis of the resulting traffic stream behavior using this framework is presented for different market penetration rates of connected and autonomous vehicles. The analysis reveals that connected and autonomous vehicles can improve string stability. Moreover, automation is found to be more effective in preventing shockwave formation and propagation under the model’s assumptions. In addition to stability, the effects of these technologies on throughput are explored, suggesting substantial potential throughput increases under certain penetration scenarios.

Influence of connected and autonomous vehicles on traffic flow stability and throughput

/pdf/multiple-factor-analysis-1xkff91jny.pdf

Multiple factor analysis.

Researchers must make numerous choices when conducting factor analyses, each of which can have significant ramifications on the model results. They must decide on an appropriate sample size to achieve accurate parameter estimates and adequate power, a factor model and estimation method, a method for determining the number of factors and evaluating model fit, and a rotation criterion. Unfortunately, researchers continue to use outdated methods in each of these areas. The present article provides a current overview of these areas in an effort to provide researchers with up-to-date methods and considerations in both exploratory and confirmatory factor analysis. A demonstration was provided to illustrate current approaches. Choosing between confirmatory and exploratory methods is also discussed, as researchers often make incorrect assumptions about the application of each.

/pdf/current-methodological-considerations-in-exploratory-and-5pwgjsk3rb.pdf

Current Methodological Considerations in Exploratory and Confirmatory Factor Analysis

Exploratory factor analysis (EFA) is generally regarded as a technique for large sample sizes (N), with N = 50 as a reasonable absolute minimum. This study offers a comprehensive overview of the conditions in which EFA can yield good quality results for N below 50. Simulations were carried out to estimate the minimum required N for different levels of loadings (λ), number of factors (f), and number of variables (p) and to examine the extent to which a small N solution can sustain the presence of small distortions such as interfactor correlations, model error, secondary loadings, unequal loadings, and unequal p/f. Factor recovery was assessed in terms of pattern congruence coefficients, factor score correlations, Heywood cases, and the gap size between eigenvalues. A subsampling study was also conducted on a psychological dataset of individuals who filled in a Big Five Inventory via the Internet. Results showed that when data are well conditioned (i.e., high λ, low f, high p), EFA can yield reliable results for N well below 50, even in the presence of small distortions. Such conditions may be uncommon but should certainly not be ruled out in behavioral research data. ∗ These authors contributed equally to this work.

Exploratory Factor Analysis with Small Sample Sizes.

This study investigated user acceptance, concerns, and willingness to buy partially, highly, and fully automated vehicles. By means of a 63-question Internet-based survey, we collected 5000 responses from 109 countries (40 countries with at least 25 respondents). We determined cross-national differences, and assessed correlations with personal variables, such as age, gender, and personality traits as measured with a short version of the Big Five Inventory. Results showed that respondents, on average, found manual driving the most enjoyable mode of driving. Responses were diverse: 22% of the respondents did not want to pay more than $0 for a fully automated driving system, whereas 5% indicated they would be willing to pay more than $30,000, and 33% indicated that fully automated driving would be highly enjoyable. 69% of respondents estimated that fully automated driving will reach a 50% market share between now and 2050. Respondents were found to be most concerned about software hacking/misuse, and were also concerned about legal issues and safety. Respondents scoring higher on neuroticism were slightly less comfortable about data transmitting, whereas respondents scoring higher on agreeableness were slightly more comfortable with this. Respondents from more developed countries (in terms of lower accident statistics, higher education, and higher income) were less comfortable with their vehicle transmitting data, with cross-national correlations between ρ = −0.80 and ρ = −0.90. The present results indicate the major areas of promise and concern among the international public, and could be useful for vehicle developers and other stakeholders.

/pdf/public-opinion-on-automated-driving-results-of-an-4z5lglcxwz.pdf

Public opinion on automated driving: results of an international questionnaire among 5000 respondents

The Driver Behaviour Questionnaire as a predictor of accidents: A meta-analysis

Principal axis factoring (PAF) and maximum likelihood factor analysis (MLFA) are two of the most popular estimation methods in exploratory factor analysis. It is known that PAF is better able to recover weak factors and that the maximum likelihood estimator is asymptotically efficient. However, there is almost no evidence regarding which method should be preferred for different types of factor patterns and sample sizes. Simulations were conducted to investigate factor recovery by PAF and MLFA for distortions of ideal simple structure and sample sizes between 25 and 5000. Results showed that PAF is preferred for population solutions with few indicators per factor and for overextraction. MLFA outperformed PAF in cases of unequal loadings within factors and for underextraction. It was further shown that PAF and MLFA do not always converge with increasing sample size. The simulation findings were confirmed by an empirical study as well as by a classic plasmode, Thurstone's box problem. The present results are...

J.C.F. de Winter

Papers

Exploratory Factor Analysis with Small Sample Sizes.

Public opinion on automated driving: results of an international questionnaire among 5000 respondents

The Driver Behaviour Questionnaire as a predictor of accidents: A meta-analysis

Factor recovery by principal axis factoring and maximum likelihood factor analysis as a function of factor pattern and sample size

Social desirability is the same in offline, online, and paper surveys