Showing papers in "Behavior Research Methods in 2007"

G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences

Abstract: G*Power (Erdfelder, Faul, & Buchner, 1996) was designed as a general stand-alone power analysis program for statistical tests commonly used in social and behavioral research. G*Power 3 is a major extension of, and improvement over, the previous versions. It runs on widely used computer platforms (i.e., Windows XP, Windows Vista, and Mac OS X 10.4) and covers many different statistical tests of thet, F, and χ2 test families. In addition, it includes power analyses forz tests and some exact tests. G*Power 3 provides improved effect size calculators and graphic options, supports both distribution-based and design-based input modes, and offers all types of power analyses in which users might be interested. Like its predecessors, G*Power 3 is free.

The English Lexicon Project.

Abstract: The English Lexicon Project is a multiuniversity effort to provide a standardized behavioral and descriptive data set for 40,481 words and 40,481 nonwords. It is available via the Internet at elexicon.wustl.edu. Data from 816 participants across six universities were collected in a lexical decision task (approximately 3400 responses per participant), and data from 444 participants were collected in a speeded naming task (approximately 2500 responses per participant). The present paper describes the motivation for this project, the methods used to collect the data, and the search engine that affords access to the behavioral measures and descriptive lexical statistics for these stimuli.

Distribution of the product confidence limits for the indirect effect: Program PRODCLIN

Abstract: This article describes a program, PRODCLIN (distribution of the PRODuct Confidence Limits for INdirect effects), written for SAS, SPSS, and R, that computes confidence limits for the product of two normal random variables. The program is important because it can be used to obtain more accurate confidence limits for the indirect effect, as demonstrated in several recent articles (MacKinnon, Lockwood, & Williams, 2004; Pituch, Whittaker, & Stapleton, 2005). Tests of the significance of and confidence limits for indirect effects based on the distribution of the product method have more accurate Type I error rates and more power than other, more commonly used tests. Values for the two paths involved in the indirect effect and their standard errors are entered in the PRODCLIN program, and distribution of the product confidence limits are computed. Several examples are used to illustrate the PRODCLIN program. The PRODCLIN programs in rich text format may be downloaded from www.psychonomic.org/archive.

Using heteroskedasticity-consistent standard error estimators in OLS regression: an introduction and software implementation.

Abstract: Homoskedasticity is an important assumption in ordinary least squares (OLS) regression. Although the estimator of the regression parameters in OLS regression is unbiased when the homoskedasticity assumption is violated, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity, which can produce significance tests and confidence intervals that can be liberal or conservative. After a brief description of heteroskedasticity and its effects on inference in OLS regression, we discuss a family of heteroskedasticity-consistent standard error estimators for OLS regression and argue investigators should routinely use one of these estimators when conducting hypothesis tests using OLS regression. To facilitate the adoption of this recommendation, we provide easy-to-use SPSS and SAS macros to implement the procedures discussed here.

Extracting semantic representations from word co-occurrence statistics: a computational study.

Abstract: The idea that at least some aspects of word meaning can be induced from patterns of word co-occurrence is becoming increasingly popular. However, there is less agreement about the precise computations involved, and the appropriate tests to distinguish between the various possibilities. It is important that the effect of the relevant design choices and parameter values are understood if psychological models using these methods are to be reliably evaluated and compared. In this article, we present a systematic exploration of the principal computational possibilities for formulating and validating representations of word meanings from word co-occurrence statistics. We find that, once we have identified the best procedures, a very simple approach is surprisingly successful and robust over a range of psychologically relevant evaluation measures.

Fast-dm: a free program for efficient diffusion model analysis.

Abstract: In the present article, a flexible and fast computer program, calledfast-dm, for diffusion model data analysis is introduced. Fast-dm is free software that can be downloaded from the authors’ websites. The program allows estimating all parameters of Ratcliff ’s (1978) diffusion model from the empirical response time distributions of any binary classification task. Fast-dm is easy to use: it reads input data from simple text files, while program settings are specified by command0s in a control file. With fast-dm, complex models can be fitted, where some parameters may vary between experimental conditions, while other parameters are constrained to be equal across conditions. Detailed directions for use of fast-dm are presented, as well as results from three short simulation studies exemplifying the utility of fast-dm.

Check Vocal: A program to facilitate checking the accuracy and response time of vocal responses from DMDX

Abstract: CheckVocal is a Windows application that facilitates checking the accuracy and response time of recorded vocal responses in naming and other experimental tasks using the DMDX display and response collection software. CheckVocal handles all keeping-track and presents each recorded response audiovisually (as waveform, spectrogram, and sound played out) along with the corresponding printed correct response and registered response time. The user simply decides whether the response was correct, wrong, or missing, with a single mouse click advancing to the next response. Response time correction can be done manually or automatically (retriggering by a power threshold). Data safety and integrity is ensured by cross-checking and status saving, so that interrupted sessions can be resumed later. Check Vocal is freely available to the DMDX community via a dedicated Web page.

The Spanish adaptation of ANEW (affective norms for English words).

Abstract: This article presents the Spanish adaptation of the Affective Norms for English Words (ANEW; Bradley & Lang, 1999). The norms are based on 720 participants’ assessments of the translation into Spanish of the 1,034 words included in the ANEW. The evaluations were done in the dimensions of valence, arousal and dominance using the Self-Assessment Manikin (SAM). Apart from these dimensions, five objective (number of letters, number of syllables, grammatical class, frequency and number of orthographic neighbors) and three subjective (familiarity, concreteness and imageability) psycholinguistic indexes are included. The Spanish adaptation of ANEW can be downloaded at www.psychonomic.org.

Multilevel models for the experimental psychologist: foundations and illustrative examples.

Abstract: Although common in the educational and developmental areas, multilevel models are not often utilized in the analysis of data from experimental designs. This article illustrates how multilevel models can be useful with two examples from experimental designs with repeated measurements not involving time. One example demonstrates how to properly examine independent variables for experimental stimuli or individuals that are categorical, continuous, or semicontinuous in the presence of missing data. The second example demonstrates how response times and error rates can be modeled simultaneously within a multivariate model in order to examine speed—accuracy trade-offs at the experimental-condition and individual levels, as well as to examine differences in the magnitude of effects across outcomes. SPSS and SAS syntax for the examples are available electronically.

Characterization of the Affective Norms for English Words by discrete emotional categories

Abstract: The Affective Norms for English Words (ANEW) are a commonly used set of 1,034 words characterized on the affective dimensions of valence, arousal, and dominance. Traditionally, studies of affect have used stimuli characterized along either affective dimensions or discrete emotional categories, but much current research draws on both of these perspectives. As such, stimuli that have been thoroughly characterized according to both of these approaches are exceptionally useful. In an effort to provide researchers with such a characterization of stimuli, we have collected descriptive data on the ANEW to identify which discrete emotions are elicited by each word in the set. Our data, coupled with previous characterizations of the dimensional aspects of these words, will allow researchers to control for or manipulate stimulus properties in accordance with both dimensional and discrete emotional views, and provide an avenue for further integration of these two perspectives. Our data have been archived at www.psychonomic.org/archive/.

Testing the race model inequality: an algorithm and computer programs.

Abstract: In divided-attention tasks, responses are faster when two target stimuli are presented, and thus one is redundant, than when only a single target stimulus is presented. Raab (1962) suggested an account of this redundanttargets effect in terms of a race model in which the response to redundant target stimuli is initiated by the faster of two separate target detection processes. Such models make a prediction about the probability distributions of reaction times that is often calledthe race model inequality, and it is often of interest to test this prediction. In this article, we describe a precise algorithm that can be used to test the race model inequality and present MATLAB routines and a Pascal program that implement this algorithm.

Methods for the behavioral, educational, and social sciences: an R package.

Abstract: Methods for the Behavioral, Educational, and Social Sciences (MBESS; Kelley, 2007b) is an open source package for R (R Development Core Team, 2007b), an open source statistical programming language and environment. MBESS implements methods that are not widely available elsewhere, yet are especially helpful for the idiosyncratic techniques used within the behavioral, educational, and social sciences. The major categories of functions are those that relate to confidence interval formation for noncentral t, F, and chi2 parameters, confidence intervals for standardized effect sizes (which require noncentral distributions), and sample size planning issues from the power analytic and accuracy in parameter estimation perspectives. In addition, MBESS contains collections of other functions that should be helpful to substantive researchers and methodologists. MBESS is a long-term project that will continue to be updated and expanded so that important methods can continue to be made available to researchers in the behavioral, educational, and social sciences.

Multidimensional normative ratings for the International Affective Picture System.

Abstract: The purpose of the present investigation was to replicate and extend the International Affective Picture System norms (Ito, Cacioppo, & Lang, 1998; Lang, Bradley, & Cuthbert, 1999). These norms were developed to provide researchers with photographic slides that varied in emotional evocation, especially arousal and valence. In addition to collecting rating data on the dimensions of arousal and valence, we collected data on the dimensions of consequentiality, meaningfulness, familiarity, distinctiveness, and memorability. Furthermore, we collected ratings on the primary emotions of happiness, surprise, sadness, anger, disgust, and fear. A total of 1,302 participants were tested in small groups. The participants in each group rated a subset of 18 slides on 14 dimensions. Ratings were obtained on 703 slides. The means and standard deviations for all of the ratings are provided. We found our valence ratings to be similar to the previous norms. In contrast, our participants were more likely to rate the slides as less arousing than in the previous norms. The mean ratings on the remaining 12 dimensions were all below the midpoint of the 9-point Likert scale. However, sufficient variability in ratings across the slides indicates that selecting slides on the basis of these variables is feasible. Overall, the present ratings should allow investigators to use these norms for research purposes, especially in research dealing with the interrelationships among emotion and cognition. The means and standard deviations for emotions may be downloaded as an Excel spreadsheet from www.psychonomic.org/archive.

Development and validation of a World-Wide-Web-based neurocognitive assessment battery: WebNeuro

Abstract: Assessment of neurocognitive functioning is a critical task in many clinical, educational, service, and industrial settings. We report on descriptive and validation data of a new, World-Wide-Web-based, comprehensive battery of neurocognitive functioning, WebNeuro, that can be used in both applied and research contexts. Fifty healthy control participants completed both WebNeuro, and an established non-Internet-based computerized cognitive assessment battery, IntegNeuro, that uses a touchscreen platform. Results indicated comparability across the two batteries, in terms of critical single test scores, factor analysis derived indices, overall performance scores, and sex differences. These results support the validity of WebNeuro as a neurocognitive assessment measure. Advantages of its use in applied and research settings are discussed.

EMuJoy: Software for continuous measurement of perceived emotions in music

Abstract: An adequate study of emotions in music and film should be based on the real-time measurement of selfreported data using a continuous-response method The recording system discussed in this article reflects two important aspects of such research: First, for a better comparison of results, experimental and technical standards for continuous measurement should be taken into account, and second, the recording system should be open to the inclusion of multimodal stimuli In light of these two considerations, our article addresses four basic principles of the continuous measurement of emotions: (1) the dimensionality of the emotion space, (2) data acquisition (eg, the synchronization of media and the self-reported data), (3) interface construction for emotional responses, and (4) the use of multiple stimulus modalities Researcher-developed software (EMuJoy) is presented as a freeware solution for the continuous measurement of responses to different media, along with empirical data from the self-reports of 38 subjects listening to emotional music and viewing affective pictures Behavior Research Methods

Manulex-infra: distributional characteristics of grapheme-phoneme mappings, and infralexical and lexical units in child-directed written material.

Abstract: It is well known that the statistical characteristics of a language, such as word frequency or the consistency of the relationships between orthography and phonology, influence literacy acquisition. Accordingly, linguistic databases play a central role by compiling quantitative and objective estimates about the principal variables that affect reading and writing acquisition. We describe a new set of Web-accessible databases of French orthography whose main characteristic is that they are based on frequency analyses of words occurring in reading books used in the elementary school grades. Quantitative estimates were made for several infralexical variables (syllable, grapheme-to-phoneme mappings, bigrams) and lexical variables (lexical neighborhood, homophony and homography). These analyses should permit quantitative descriptions of the written language in beginning readers, the manipulation and control of variables based on objective data in empirical studies, and the development of instructional methods in keeping with the distributional characteristics of the orthography.

Methodological considerations in performing semantic- and translation-priming experiments across languages.

Abstract: Research in the field of bilingualism has had as its principal aim to describe the structure and function of memory for bilingual speakers. A primary technique that has been used to examine bilingual memory is an examination of cross-language word priming (semantic and translation), using the lexical decision and pronunciation tasks. Although studies have, on occasion, revealed greater degrees of word priming from a dominant to a subordinate language, in comparison with the reverse, a careful review of the methodology that has been used reveals a number of issues that render conclusions such as this quite problematic. Parameters of concern include language proficiency, cognate status, masking, control conditions, word frequency and length, stimulus onset asynchrony, relatedness proportion, and nonword ratio. These factors are discussed, as well as recommendations for conducting future empirical research in this area of investigation.

Word naming and psycholinguistic norms: Chinese.

Abstract: In this article, we present normative data for 2,423 Chinese single-character words. For each word, we report values for the following 15 variables: word frequency, cumulative frequency, homophone density, phonological frequency, age of learning, age of acquisition, number of word formations, number of meanings, number of components, number of strokes, familiarity, concreteness, imageability, regularity, and initial phoneme. To validate the norms, we collected word-naming latencies. Factor analysis and multiple regression analysis show that naming latencies of Chinese single-character words are predicted by frequency, semantics, visual features, and consistency, but not by phonology. These analyses show distinct patterns in word naming between Chinese and alphabetic languages and demonstrate the utility of normative data in the study of nonalphabetic orthographic processing.

SAS and SPSS macros to calculate standardized Cronbach's alpha using the upper bound of the phi coefficient for dichotomous items.

Abstract: Cronbach’s α is widely used in social science research to estimate the internal consistency of reliability of a measurement scale. However, when items are not strictly parallel, the Cronbach’s α coefficient provides a lower-bound estimate of true reliability, and this estimate may be further biased downward when items are dichotomous. The estimation of standardized Cronbach’s α for a scale with dichotomous items can be improved by using the upper bound of coefficient ϕ. SAS and SPSS macros have been developed in this article to obtain standardized Cronbach’s α via this method. The simulation analysis showed that Cronbach’s α from upper-bound ϕ might be appropriate for estimating the real reliability when standardized Cronbach’s α is problematic.

Sample size planning for the coefficient of variation from the accuracy in parameter estimation approach.

Abstract: The accuracy in parameter estimation approach to sample size planning is developed for the coefficient of variation, where the goal of the method is to obtain an accurate parameter estimate by achieving a sufficiently narrow confidence interval. The first method allows researchers to plan sample size so that the expected width of the confidence interval for the population coefficient of variation is sufficiently narrow. A modification allows a desired degree of assurance to be incorporated into the method, so that the obtained confidence interval will be sufficiently narrow with some specified probability (e.g., 85% assurance that the 95% confidence interval width will be no wider than ω units). Tables of necessary sample size are provided for a variety of scenarios that may help researchers planning a study where the coefficient of variation is of interest plan an appropriate sample size in order to have a sufficiently narrow confidence interval, optionally with some specified assurance of the confidence interval being sufficiently narrow. Freely available computer routines have been developed that allow researchers to easily implement all of the methods discussed in the article.

Match: a program to assist in matching the conditions of factorial experiments.

Abstract: In most experiments that involve between-subjects or between-items factorial designs, the items and/or the participants in the various experimental groups differ on one or more variables, but need to be matched on all other factors that can affect the outcome measure. Matching large groups of items or participants on multiple dimensions is a difficult and time-consuming task, yet failure to match conditions will lead to suboptimal experiments. We describe a computer program, “Match,” that automates this process by selecting the best-matching items from larger sets of candidate items. In most cases, the program produces near-optimal solutions in a matter of minutes and selects matches that are typically superior to those obtained using hand matching or other semiautomated processes. We report the results of a case study in which Match was used to generate matched sets of experimental items (words varying in length and frequency) for a published study on language processing. The program was able to come up with better-matching item sets than those hand-selected by the authors of the original study, and in a fraction of the time originally taken up with stimulus matching.

On the use of multilevel modeling as an alternative to items analysis in psycholinguistic research.

Abstract: The use of multilevel modeling is presented as an alternative to separate item and subject ANOVAs (F1�F 2) in psycholinguistic research. Multilevel modeling is commonly utilized to model variability arising from the nesting of lower level observations within higher level units (e.g., students within schools, repeated measures within individuals). However, multilevel models can also be used when two random factors are crossed at the same level, rather than nested. The current work illustrates the use of the multilevel model for crossed random effects within the context of a psycholinguistic experimental study, in which both subjects and items are modeled as random effects within the same analysis, thus avoiding some of the problems plaguing current approaches.

HMMTree: A computer program for latent-class hierarchical multinomial processing tree models

Abstract: Latent-class hierarchical multinomial models are an important extension of the widely used family of multinomial processing tree models, in that they allow for testing the parameter homogeneity assumption and provide a framework for modeling parameter heterogeneity. In this article, the computer program HMMTree is introduced as a means of implementing latent-class hierarchical multinomial processing tree models. HMMTree computes parameter estimates, confidence intervals, and goodness-of-fit statistics for such models, as well as the Fisher information, expected category means and variances, and posterior probabilities for class membership. A brief guide to using the program is provided.

Adobe Flash as a medium for online experimentation: a test of reaction time measurement capabilities

Abstract: Adobe Flash can be used to run complex psychological experiments over the Web. We examined the reliability of using Flash to measure reaction times (RTs) using a simple binary-choice task implemented both in Flash and in a Linux-based system known to record RTs with millisecond accuracy. Twenty-four participants were tested in the laboratory using both implementations; they also completed the Flash version on computers of their own choice outside the lab. RTs from the trials run on Flash outside the lab were approximately 20 msec slower than those from trials run on Flash in the lab, which in turn were approximately 10 msec slower than RTs from the trials run on the Linux-based system (baseline condition). RT SDs were similar in all conditions, suggesting that although Flash may overestimate RTs slightly, it does not appear to add significant noise to the data recorded.

PyEPL: A cross-platform experiment-programming library

Abstract: PyEPL (the Python Experiment-Programming Library) is a Python library which allows cross-platform and object-oriented coding of behavioral experiments. It provides functions for displaying text and images onscreen, as well as playing and recording sound, and is capable of rendering 3-D virtual environments for spatial-navigation tasks. It is currently tested for Mac OS X and Linux. It interfaces with Activewire USB cards (on Mac OS X) and the parallel port (on Linux) for synchronization of experimental events with physiological recordings. In this article, we first present two sample programs which illustrate core PyEPL features. The examples demonstrate visual stimulus presentation, keyboard input, and simulation and exploration of a simple 3-D environment. We then describe the components and strategies used in implementing PyEPL.

Variations of the flanker paradigm: Assessing selective attention in young children

Abstract: The development of selective attention and associated self-regulatory processes was assessed in young children, ages 4, 5, and 6, through the use of three alternative versions of the flanker paradigm utilizing colors, shapes, and fish. These variations were used to examine the influence of task differences on children’s performance. The presence of cognitive self-regulatory strategies in young children was also assessed. Significant flanker interference effects, marked by significant task-linked response time differences, were found across all three versions of the paradigm. Although a significant portion of children demonstrated self-regulatory abilities, not every participant demonstrated the specific strategies of self-monitoring and response control. Furthermore, these differences were evident across all age groups. The implications of these results are discussed within the theoretical context of task development, taking into consideration the need to modify computerized attention paradigms for use with young children in order to reliably measure cognitive constructs across children and adults.

Internet-delivered assessment and manipulation of anxiety-linked attentional bias: Validation of a free-access attentional probe software package.

Abstract: It recently has been established within the laboratory that the attentional probe methodology not only can sensitively assess the threat attentional bias associated with anxiety vulnerability, but also can be configured to yield training tasks capable of modifying emotional vulnerability by manipulating such attentional selectivity In order to appraise and exploit the potential practical applications of such procedures, it is desirable that clinical investigators without specialist equipment should become able to deliver such tasks within field settings The present research program involved development of a fully customizable attentional probe software package, that delivers assessment and training versions of the probe task across the Internet Two experimental studies served to validate the assessment and training efficacy of resulting probe task variants, completed remotely by GAD sufferers and nonclinical populations using their own computers We advise fellow researchers how to freely download this software package for use within their own investigations

Translation norms for English and Spanish: The role of lexical variables, word class, and L2 proficiency in negotiating translation ambiguity

Abstract: We present a set of translation norms for 670 English and 760 Spanish nouns, verbs and class ambiguous items that varied in their lexical properties in both languages, collected from 80 bilingual participants. Half of the words in each language received more than a single translation across participants. Cue word frequency and imageability were both negatively correlated with number of translations. Word class predicted number of translations: Nouns had fewer translations than did verbs, which had fewer translations than class-ambiguous items. The translation probability of specific responses was positively correlated with target word frequency and imageability, and with its form overlap with the cue word. Translation choice was modulated by L2 proficiency: Less proficient bilinguals tended to produce lower probability translations than more proficient bilinguals, but only in forward translation, from L1 to L2. These findings highlight the importance of translation ambiguity as a factor influencing bilingual representation and performance. The norms can also provide an important resource to assist researchers in the selection of experimental materials for studies of bilingual and monolingual language performance. These norms may be downloaded from www.psychonomic.org/archive.

jTRACE: A reimplementation and extension of the TRACE model of speech perception and spoken word recognition

Abstract: This article describes jTRACE, a freely available, cross-platform Java reimplementation of the TRACE model of spoken word recognition. The goal of the reimplementation is to facilitate the use of simulations by researchers who may not have the skills or time necessary to use or extend the original C implementation. In this article, we report a large-scale validation project, in which we have replicated a number of important previous simulations, and then we describe several new features in jTRACE designed to help researchers conduct original TRACE research, as well as to replicate earlier findings. These features include visualization tools, powerful scripting, built-in data graphing, adjustable levels of external and internal noise, and adjustable lexical characteristics, such as frequency of occurrence. Functions for saving and reloading entire simulations facilitate archiving, sharing, and replication and also make jTRACE ideal for educational use, since it comes bundled with several important simulations. jTRACE can be downloaded from magnuson.psy.uconn.edu/jtrace.

Observer agreement for event sequences: methods and software for sequence alignment and reliability estimates.

Abstract: When sequences of discrete events, or other units, are independently coded by two coders using a set of mutually exclusive and exhaustive codes, but the onset times for the codes are not preserved, it is often unclear how pairs of protocols should be aligned. Yet such alignment is required before Cohen’s kappa, a common agreement statistic, can be computed. Here we describe a method—based on the Needleman and Wunsch (1970) algorithm originally devised for aligning nucleotide sequences—for optimally aligning such sequences; we also offer the results of a simulation study of the behavior of alignment kappa with a number of variables, including number of codes, varying degrees of observer accuracy, sequence length, code variability, and parameters governing the alignment algorithm. We conclude that (1) under most reasonable circumstances, observer accuracies of 90% or better result in alignment kappas of .60 or better; (2) generally, alignment kappas are not strongly affected by sequence length, the number of codes, or the variability in the codes’ probability; (3) alignment kappas are adversely affected when missed events and false alarms are possible; and (4) cost matrices and priority orders used in the algorithm should favor substitutions (i.e., disagreements) over insertions and deletions (i.e., missed events and false alarms). Two computer programs were developed: Global Sequence Alignment, or GSA, for carrying out the simulation study, and Event Alignment, or ELign, a user-oriented program that computes alignment kappa and provides the optimal alignment given a pair of event sequences.