Interest in the problem of method biases has a long history in the behavioral sciences. Despite this, a comprehensive summary of the potential sources of method biases and how to control for them does not exist. Therefore, the purpose of this article is to examine the extent to which method biases influence behavioral research results, identify potential sources of method biases, discuss the cognitive processes through which method biases influence responses to measures, evaluate the many different procedural and statistical techniques that can be used to control method biases, and provide recommendations for how to select appropriate procedural and statistical remedies for different types of research settings.

/pdf/common-method-biases-in-behavioral-research-a-critical-53ocysaemj.pdf

Common method biases in behavioral research: a critical review of the literature and recommended remedies.

Most medical researchers, whether clinical or non-clinical, receive some background in statistics as undergraduates. However, it is most often brief, a long time ago, and largely forgotten by the time it is needed. Furthermore, many introductory texts fall short of adequately explaining the underlying concepts of statistics, and often are divorced from the reality of conducting and assessing medical research.

Practical Statistics for Medical Research is a problem-based text for medical researchers, medical students, and others in the medical arena who need to use statistics but have no specialized mathematics background. 

The author draws on twenty years of experience as a consulting medical statistician to provide clear explanations to key statistical concepts, with a firm emphasis on practical aspects of designing and analyzing medical research. The text gives special attention to the presentation and interpretation of results and the many real problems that arise in medical research

/pdf/practical-statistics-for-medical-research-1ijulrzqv3.pdf

Practical statistics for medical research

Silhouettes: a graphical aid to the interpretation and validation of cluster analysis

This book describes the new generation of discrete choice methods, focusing on the many advances that are made possible by simulation. Researchers use these statistical methods to examine the choices that consumers, households, firms, and other agents make. Each of the major models is covered: logit, generalized extreme value, or GEV (including nested and cross-nested logits), probit, and mixed logit, plus a variety of specifications that build on these basics. Simulation-assisted estimation procedures are investigated and compared, including maximum simulated likelihood, method of simulated moments, and method of simulated scores. Procedures for drawing from densities are described, including variance reduction techniques such as anithetics and Halton draws. Recent advances in Bayesian procedures are explored, including the use of the Metropolis-Hastings algorithm and its variant Gibbs sampling. No other book incorporates all these fields, which have arisen in the past 20 years. The procedures are applicable in many fields, including energy, transportation, environmental studies, health, labor, and marketing.

/pdf/discrete-choice-methods-with-simulation-l1x4cghjtt.pdf

Discrete Choice Methods with Simulation

In 1974 an article appeared in Science magazine with the dry-sounding title “Judgment Under Uncertainty: Heuristics and Biases” by a pair of psychologists who were not well known outside their discipline of decision theory. In it Amos Tversky and Daniel Kahneman introduced the world to Prospect Theory, which mapped out how humans actually behave when faced with decisions about gains and losses, in contrast to how economists assumed that people behave. Prospect Theory turned Economics on its head by demonstrating through a series of ingenious experiments that people are much more concerned with losses than they are with gains, and that framing a choice from one perspective or the other will result in decisions that are exactly the opposite of each other, even if the outcomes are monetarily the same. Prospect Theory led cognitive psychology in a new direction that began to uncover other human biases in thinking that are probably not learned but are part of our brain’s wiring.

Thinking fast and slow.

Differential Item Functioning.

Contents: C.V. Bunderson, Foreword to the First Edition. D.H. Gitomer, Foreword to the Second Edition. H. Wainer, Preface to the First Edition. H. Wainer, Preface to the Second Edition. H. Wainer, Introduction and History. B.F. Green, System Design and Operation. R. Flaugher, Item Pools. H. Wainer, R.J. Mislevy, Item Response Theory, Item Calibration, and Proficiency Estimation. D. Thissen, R.J. Mislevy, Testing Algorithms. N.J. Dorans, Scaling and Equating. D. Thissen, Reliability and Measurement Precision. L. Steinberg, D. Thissen, H. Wainer, Validity. H. Wainer, N.J. Dorans, B.F. Green, R.J. Mislevy, L. Steinberg, D. Thissen, Future Challenges. H. Wainer, D. Eignor, Caveats, Pitfalls, and Unexpected Consequences of Implementing Large-Scale Computerized Testing.

Computerized Adaptive Testing: A Primer

It is observed that many sorts of difficulties may preclude the uneventful construction of tests by a computerized algorithm, such as those currently in favor in Computerized Adaptive Testing (CAT). In this essay we discuss a number of these problems, as well as some possible avenues of solution. We conclude with the development of the "testlet," a bundle of items that can be arranged either hierarchically or linearly, thus maintaining the efficiency of an adaptive test while keeping the quality control of test construction that is possible currently only with careful expert scrutiny. Performance on the separate testlets is aggregated to yield ability estimates. In the old days of testing a wise examiner confronted a nervous examinee and asked questions. After getting some initial idea of the examinee's ability or knowledge level, the examiner would not waste time asking questions that were too difficult or too easy. Instead, he or she would focus questions on the area around the examinee's proficiency level. This made for a challenging and efficient test. It was also subjective and expensive. This state of affairs changed during the First World War with the Army Alpha. The mass processing of recruits into the armed services demonstrated that standardized tests could make the training and accessioning process more efficient. Substituting for the wise examiner was a broad-range test made up of many multiple-choice questions. This had several advantages, among which were its vastly reduced cost per unit and its objectivity-everyone took the same test, and it was scored in the same way for everyone. It also had some disadvantages. As a written document, a test form could be stolen, and because of its broad audience it had to contain items that would tax the least able of the prospective

Item Clusters and Computerized Adaptive Testing: A Case for Testlets

It is proved that under very general circumstances coefficients in multiple regression models can be replaced with equal weights with almost no loss in accuracy on the original data sample. It is then shown that these equal weights will have greater robustness than least squares regression coefficients. The implications for problems of prediction are discussed. In the two decades since Meehl's (1954) book on the respective accuracy of clinical versus clerical prediction, little practical consequence has been observed. Diagnoses are still made by clinicians, not by clerks; college admissions are still done by committee, not by computer. This is true despite the considerable strength of Meehl's argument that humans are very poor at combining information optimally and that regression models evidently combine information rather well. These points were underlined in some recent work by Dawes and Corrigan (1974), in which they found again that human predictors do poorly when compared with regression models. Strikingly, they found that for some reason, linear models with random regression weights also do better than do humans. Even more striking, when all regression weights were set equal to one another they found still higher correlation with criterion on a validating sample. The obvious question here is Why? Is it because humans are so terrible at combining information that almost any rule works better, or is it some artifact of linear regression?

Howard Wainer

Papers

Differential Item Functioning.

Computerized Adaptive Testing: A Primer

Item Clusters and Computerized Adaptive Testing: A Case for Testlets

Estimating Coefficients in Linear Models: It Don't Make No Nevermind

Detection of differential item functioning using the parameters of item response models.