Many researchers have pointed out that it is necessary to ensure measurement equivalence in cross-national comparative research. Three aspects of measurement equivalence, translation, metric, and calibration equivalence, are necessary to establish the cross-national reliability and validity of items used to measure theoretical constructs. This paper discusses these issues and proposes two recently developed empirical techniques, Multiple Group LISREL and Optimal Scaling, for use in diagnosing cross-national measurement equivalence. These techniques are illustrated by reanalysis of a pioneering U.S. and Japanese study. The two techniques yield convergent results, indicating measurement equivalence for some, but not all, ordinal-level items under consideration. The findings demonstrate that the proposed methods are useful diagnostic tools for exploring measurement equivalence. Several suggestions for reducing the likelihood of problems with measurement equivalence and a number of methods for dealing with items where lack of equivalence persists are also discussed.

http://www.rcmewhu.com/upload/file/20150528/20150528101814_7576.pdf

Diagnosing Measurement Equivalence in Cross-National Research

The effect of rating scale format on response styles: The number of response categories and response category labels

Questionnaires using Likert-type rating scales are an important source of data in marketing research. Researchers use different rating scale formats with varying number of response categories and varying label formats (e.g., seven point rating scales labeled at the endpoints, fully labeled five point scales…), but have few guidelines when selecting a specific format. Drawing from the response style literature, we formulate hypotheses on the effect of the labeling of response categories and the number of response categories on net acquiescence response style, extreme response style and misresponse to reversed items. We test the hypotheses in an online survey (N=1207) with eight experimental conditions and a follow-up study with two experimental conditions (N = 226). We find evidence of strong effects of scale format on response distributions and misresponse to reversed items and formulate recommendations on scale format choice

Marketing researchers frequently split (dichotomize) continuous predictor variables into two groups, such as with a median split, before performing data analysis. The practice is prevalent but its effects are not well understood. In this paper, we present historic results on the effects of dichotomization of normal predictor variables rederived in a regression context that may be more relevant to marketing researchers. Then, we present new results on the effect of dichotomizing continuous predictor variables having a variety of non-normal distributions and examine the effects of dichotomization on model specification and fit in multiple regression. We conclude that dichotomization has only negative consequences and should be avoided.

Negative Consequences of Dichotomizing Continuous Predictor Variables

Marketing researchers frequently split (dichotomize) continuous predictor variables into two groups, as with a median split, before performing data analysis. The practice is prevalent, but its effects are not well understood. In this article, the authors present historical results on the effects of dichotomization of normal predictor variables rederived in a regression context that may be more relevant to marketing researchers. The authors then present new results on the effect of dichotomizing continuous predictor variables with various nonnormal distributions and examine the effects of dichotomization on model specification and fit in multiple regression. The authors conclude that dichotomization has only negative consequences and should be avoided.

We quantify the information loss incurred by categorizing an unobserved continuous variable X into an ordered categorical scale Z. The continuous variable is conceptualized as true score τ which varies across individuals plus random error e, with both components assumed to be normally distributed. The index of metric quality is operationalized as r2Z, τ/r2X, τ, where r2, the squared correlation coefficient, is a descriptive measure of the power of X or Z to predict τ. The index is useful in defining limits on explanatory power population R2 in multiple regression models in which an ordered categorical variable is regressed against a set of predictors. The index can also be used to correct correlations for the effects of ordered categorical measurement.

The index of metric quality is extended to the case when several ordered categorical scales are averaged as in the multi-item measurement of a construct. We prove theoretically that as long as the error variance is “large,” the index of metric quality for the average ZI of ordered categorical scales goes to 1 as the number of scales becomes “large.” The index for averaged data is useful in answering questions such as whether the measurement of a construct by averaging three 5-point scales is better or worse than the measurement obtained by averaging five 3-point scales. The results indicate that the loss of information by marketing researchers' ad hoc use of Z as opposed to the more refined X is small <10% when the Z scale is well designed with at least five categories. The loss would be even smaller when a multi-item based ZI is employed.

Amiya K. Basu

Papers

The Metric Quality of Ordered Categorical Data

On the relative performance of linear versus nonlinear compensation plans