Publishing in AMJ—Part 2: Research Design

TL;DR: This paper conducted an in-depth review and content analysis of what variables, and why such variables are controlled for, in 10 of the most popular research domains (task performance, organizational citizenship behaviors, turnover, job satisfaction, organizational commitment, employee burnout, personality, leader member exchange, organizational justice, and affect) in organizational behavior/human resource management (OB/HRM) and applied psychology.

...read moreread less

Abstract: The use of control variables plays a central role in organizational research due to practical difficulties associated with the implementation of experimental and quasi-experimental designs. As such, we conducted an in-depth review and content analysis of what variables, and why such variables are controlled for, in 10 of the most popular research domains (task performance, organizational citizenship behaviors, turnover, job satisfaction, organizational commitment, employee burnout, personality, leader‒member exchange, organizational justice, and affect) in organizational behavior/human resource management (OB/HRM) and applied psychology. Specifically, we examined 580 articles published from 2003 to 2012 in AMJ, ASQ, JAP, JOM, and PPsych. Results indicate that, across research domains with clearly distinct theoretical bases, the overwhelming majority of the more than 3,500 controls identified in our review converge around the same simple demographic factors (i.e., gender, age, tenure), very little effort is made to explain why and how controls relate to focal variables of interest, and control variable practices have not changed much over the past decade. To address these results, we offer best-practice recommendations in the form of a sequence of questions and subsequent steps that can be followed to make decisions on the appropriateness of including a specific control variable within a particular theoretical framework, research domain, and empirical study. Our recommendations can be used by authors as well as journal editors and reviewers to improve the transparency and appropriateness of practices regarding control variable usage.

...read moreread less

855 citations

Cites background from "Publishing in AMJ—Part 2: Research ..."

...…each of these assumptions relies on rather large inferential leaps, perhaps the biggest and potentially most frequently violated assumption is that control variables hold theoretically meaningful relationships with predictors and criteria (Bono & McNamara, 2011; see results by Carlson & Wu, 2012)....
[...]

Journal Article•DOI•

Servant leadership: A systematic review and call for future research.

[...]

Nathan Eva, Mulyadi Robin, Sen Sendjaya¹, Dirk van Dierendonck², Robert C. Liden³ - Show less +1 more•Institutions (3)

Swinburne University of Technology¹, Erasmus University Rotterdam², University of Illinois at Urbana–Champaign³

01 Feb 2019-Leadership Quarterly

TL;DR: In this article, an integrative and comprehensive review of the 285 articles on servant leadership spanning 20 years (1998-2018) is presented. But, a lack of coherence and clarity around the construct has impeded its theory development.

...read moreread less

Abstract: Notwithstanding the proliferation of servant leadership studies with over 100 articles published in the last four years alone, a lack of coherence and clarity around the construct has impeded its theory development. We provide an integrative and comprehensive review of the 285 articles on servant leadership spanning 20 years (1998–2018), and in so doing extend the field in four different ways. First, we provide a conceptual clarity of servant leadership vis-a-vis other value-based leadership approaches and offer a new definition of servant leadership. Second, we evaluate 16 existing measures of servant leadership in light of their respective rigor of scale construction and validation. Third, we map the theoretical and nomological network of servant leadership in relation to its antecedents, outcomes, moderators, mediators. We finally conclude by presenting a detailed future research agenda to bring the field forward encompassing both theoretical and empirical advancement. All in all, our review paints a holistic picture of where the literature has been and where it should go into the future.

...read moreread less

689 citations

Cites background or methods from "Publishing in AMJ—Part 2: Research ..."

...It is not appropriate to develop one's own short version, because when items are added or dropped from a scale, the psychometric properties change (see Bono and McNamara (2011) and Keller and Dansereau (2001)). Both the SL-7 and SLBS-6 underwent the necessary psychometric work verifying that these short versions accurately capture the essence of the full measures. For both the Liden et al. and Sendjaya et al. scales, the short versions are only recommended for research examining overall/global servant leadership. Researchers who plan to test servant leadership dimensions separately should use the full scales (SL-28 and SLBS-35) so that each dimension's reliability can be estimated and so that validity is enhanced. Specifically, it is not appropriate to conduct a dimensional analysis using single items from the short version of the scale that represents a dimension(s) of interest. Dimensional analyses may also be conducted using van Dierendonck and Nuijten's (2011) scale described below. van Dierendonck and Nuijten's (2011) Servant Leadership Survey consists of 30 items that represent eight dimensions (standing back, forgiveness, courage, empowerment, accountability, authenticity, humility, and stewardship). While it is longer than the first two, it also has sound theorizing that takes into consideration the juxtaposition between the ‘servant’-side and ‘leader’-side of servant leadership. The underlying model emphasizes that servant leaders empower and develop people while holding people accountable for the outcomes of their work; they work with a humble attitude reflected in an openness to learn and a willingness to admit mistakes, they are willing to stand for their innate values, and their focus is the good of the whole. The original developmental article confirmed its construct validity (van Dierendonck & Nuijten, 2011). More recently, a short version has been introduced, an 18-items version that reflects cross-cultural factorial stability (van Dierendonck et al., 2017). Similar to the leadership field in general, servant leadership measures have an inherent focus on the hierarchical leader-follower dyad. However, some modern organizations are embracing alternate structures, giving rise to non-traditional leader-follower situations. For example, the utilization of shared leadership in education and medicine and in non-for-profit and volunteer organizations. Recent studies show promise that existing servant leadership measures can be reformulated to reflect this different way of organizing in a valid and reliable way. For the shared leadership context, Sousa and Van Dierendonck (2016) constructed a 15-item shared servant leadership measure derived from van Dierendonck and Nuijten's (2011) SLS. This measure changes the focus from rating the leader, to rating the other members within the team and the items that are retained are team-member specific rather than leader-follower specific. Secondly, there have been a number of examples in the literature where the researchers have changed the referent of the items from manager to club president (and volunteers) (Schneider & George, 2011), principal (Cerit, 2009), and nurse manager (Neubert et al., 2016). For example, Liden et al.'s (2015) “I would seek help from my leader if I had a personal problem”might be changed to “I would seek help from my team mate/co-worker/shift supervisor/principal/scout leader, if I had a personal problem”. In order to accurately capture unique settings/situations (e.g., a specific sample or non-traditional forms of leadership) researchers may shift the referent of the servant leadership measure. In doing so, we suggest a close reading of Chan's (1998) work on referent-shift consensus models as well as re-validating the servant leadership measure within that given context (exploratory and confirmatory factor analyses)....
[...]
...It is not appropriate to develop one's own short version, because when items are added or dropped from a scale, the psychometric properties change (see Bono and McNamara (2011) and Keller and Dansereau (2001))....
[...]
...It is not appropriate to develop one's own short version, because when items are added or dropped from a scale, the psychometric properties change (see Bono and McNamara (2011) and Keller and Dansereau (2001)). Both the SL-7 and SLBS-6 underwent the necessary psychometric work verifying that these short versions accurately capture the essence of the full measures. For both the Liden et al. and Sendjaya et al. scales, the short versions are only recommended for research examining overall/global servant leadership. Researchers who plan to test servant leadership dimensions separately should use the full scales (SL-28 and SLBS-35) so that each dimension's reliability can be estimated and so that validity is enhanced. Specifically, it is not appropriate to conduct a dimensional analysis using single items from the short version of the scale that represents a dimension(s) of interest. Dimensional analyses may also be conducted using van Dierendonck and Nuijten's (2011) scale described below....
[...]
...It is not appropriate to develop one's own short version, because when items are added or dropped from a scale, the psychometric properties change (see Bono and McNamara (2011) and Keller and Dansereau (2001)). Both the SL-7 and SLBS-6 underwent the necessary psychometric work verifying that these short versions accurately capture the essence of the full measures. For both the Liden et al. and Sendjaya et al. scales, the short versions are only recommended for research examining overall/global servant leadership. Researchers who plan to test servant leadership dimensions separately should use the full scales (SL-28 and SLBS-35) so that each dimension's reliability can be estimated and so that validity is enhanced. Specifically, it is not appropriate to conduct a dimensional analysis using single items from the short version of the scale that represents a dimension(s) of interest. Dimensional analyses may also be conducted using van Dierendonck and Nuijten's (2011) scale described below. van Dierendonck and Nuijten's (2011) Servant Leadership Survey consists of 30 items that represent eight dimensions (standing back, forgiveness, courage, empowerment, accountability, authenticity, humility, and stewardship). While it is longer than the first two, it also has sound theorizing that takes into consideration the juxtaposition between the ‘servant’-side and ‘leader’-side of servant leadership. The underlying model emphasizes that servant leaders empower and develop people while holding people accountable for the outcomes of their work; they work with a humble attitude reflected in an openness to learn and a willingness to admit mistakes, they are willing to stand for their innate values, and their focus is the good of the whole. The original developmental article confirmed its construct validity (van Dierendonck & Nuijten, 2011). More recently, a short version has been introduced, an 18-items version that reflects cross-cultural factorial stability (van Dierendonck et al., 2017). Similar to the leadership field in general, servant leadership measures have an inherent focus on the hierarchical leader-follower dyad. However, some modern organizations are embracing alternate structures, giving rise to non-traditional leader-follower situations. For example, the utilization of shared leadership in education and medicine and in non-for-profit and volunteer organizations. Recent studies show promise that existing servant leadership measures can be reformulated to reflect this different way of organizing in a valid and reliable way. For the shared leadership context, Sousa and Van Dierendonck (2016) constructed a 15-item shared servant leadership measure derived from van Dierendonck and Nuijten's (2011) SLS....
[...]
...It is not appropriate to develop one's own short version, because when items are added or dropped from a scale, the psychometric properties change (see Bono and McNamara (2011) and Keller and Dansereau (2001)). Both the SL-7 and SLBS-6 underwent the necessary psychometric work verifying that these short versions accurately capture the essence of the full measures. For both the Liden et al. and Sendjaya et al. scales, the short versions are only recommended for research examining overall/global servant leadership. Researchers who plan to test servant leadership dimensions separately should use the full scales (SL-28 and SLBS-35) so that each dimension's reliability can be estimated and so that validity is enhanced. Specifically, it is not appropriate to conduct a dimensional analysis using single items from the short version of the scale that represents a dimension(s) of interest. Dimensional analyses may also be conducted using van Dierendonck and Nuijten's (2011) scale described below. van Dierendonck and Nuijten's (2011) Servant Leadership Survey consists of 30 items that represent eight dimensions (standing back, forgiveness, courage, empowerment, accountability, authenticity, humility, and stewardship). While it is longer than the first two, it also has sound theorizing that takes into consideration the juxtaposition between the ‘servant’-side and ‘leader’-side of servant leadership. The underlying model emphasizes that servant leaders empower and develop people while holding people accountable for the outcomes of their work; they work with a humble attitude reflected in an openness to learn and a willingness to admit mistakes, they are willing to stand for their innate values, and their focus is the good of the whole. The original developmental article confirmed its construct validity (van Dierendonck & Nuijten, 2011). More recently, a short version has been introduced, an 18-items version that reflects cross-cultural factorial stability (van Dierendonck et al., 2017). Similar to the leadership field in general, servant leadership measures have an inherent focus on the hierarchical leader-follower dyad. However, some modern organizations are embracing alternate structures, giving rise to non-traditional leader-follower situations. For example, the utilization of shared leadership in education and medicine and in non-for-profit and volunteer organizations. Recent studies show promise that existing servant leadership measures can be reformulated to reflect this different way of organizing in a valid and reliable way. For the shared leadership context, Sousa and Van Dierendonck (2016) constructed a 15-item shared servant leadership measure derived from van Dierendonck and Nuijten's (2011) SLS. This measure changes the focus from rating the leader, to rating the other members within the team and the items that are retained are team-member specific rather than leader-follower specific. Secondly, there have been a number of examples in the literature where the researchers have changed the referent of the items from manager to club president (and volunteers) (Schneider & George, 2011), principal (Cerit, 2009), and nurse manager (Neubert et al., 2016). For example, Liden et al.'s (2015) “I would seek help from my leader if I had a personal problem”might be changed to “I would seek help from my team mate/co-worker/shift supervisor/principal/scout leader, if I had a personal problem”....
[...]

Journal Article•DOI•

Statistical control in correlational studies: 10 essential recommendations for organizational researchers.

[...]

Thomas E. Becker¹, Guclu Atinc², James A. Breaugh³, Kevin D. Carlson⁴, Jeffrey R. Edwards⁵, Paul E. Spector⁶ - Show less +2 more•Institutions (6)

University of South Florida Sarasota–Manatee¹, Texas A&M University–Commerce², University of Missouri–St. Louis³, Pamplin College of Business⁴, University of North Carolina at Chapel Hill⁵, University of South Florida⁶

01 Feb 2016-Journal of Organizational Behavior

TL;DR: The use of statistical control is widely used in correlational studies with the intent of providing more accurate estimates of relationships among variables, more conservative tests of hypotheses, or ruling out alternative explanations for empirical findings as discussed by the authors.

...read moreread less

Abstract: Summary Statistical control is widely used in correlational studies with the intent of providing more accurate estimates of relationships among variables, more conservative tests of hypotheses, or ruling out alternative explanations for empirical findings. However, the use of control variables can produce uninterpretable parameter estimates, erroneous inferences, irreplicable results, and other barriers to scientific progress. As a result, methodologists have provided a great deal of advice regarding the use of statistical control, to the point that researchers might have difficulties sifting through and prioritizing the available suggestions. We integrate and condense this literature into a set of 10 essential recommendations that are generally applicable and which, if followed, would substantially enhance the quality of published organizational research. We provide explanations, qualifications, and examples following each recommendation. Copyright © 2015 John Wiley & Sons, Ltd.

...read moreread less

541 citations

Cites background from "Publishing in AMJ—Part 2: Research ..."

...Where a strong foundation in theory is not available, researchers should at least provide a logical explanation for selecting a given CV and explain why the CV is a biasing rather than substantive variable (Becker, 2005; Bono & McNamara, 2011)....
[...]
...These articles have been cited more than 700 times, and the corresponding recommendations have been incorporated into the norms of the Academy of Management Journal and the Journal of Organizational Behavior (Bono & McNamara, 2011; Edwards, 2008)....
[...]
...Copyright © 2015 John Wiley & Sons, Ltd. Received 02 September 2015, Accepted 05 September 2015 Journal of Organizational Behavior, J. Organiz....
[...]
...Jeffrey R. Edwards is the Belk Distinguished Professor of Organizational Behavior at the Kenan-Flagler Business School, University of North Carolina....
[...]
...He is Point/Counterpoint editor for Journal of Organizational Behavior and Associate Editor for Work & Stress and is on the editorial board of Journal of Applied Psychology....
[...]

Journal Article•DOI•

Making sense of conflicting empirical findings: A meta-analytic review of the relationship between corporate environmental and financial performance

[...]

Jan Endrikat¹, Edeltraud Guenther¹, Holger Hoppe•Institutions (1)

Dresden University of Technology¹

01 Oct 2014-European Management Journal

TL;DR: In this article, a meta-analytic analysis of 149 studies on the relationship between corporate environmental performance (CEP) and corporate financial performance (CFP) is presented, showing that there is a positive and partially bidirectional relationship between CEP and CFP.

...read moreread less

379 citations

Journal Article•DOI•

Servant leadership: Validation of a short form of the SL-28

[...]

Robert C. Liden¹, Sandy J. Wayne¹, Jeremy D. Meuser¹, Jia Hu², Junfeng Wu³, Chenwei Liao³ - Show less +2 more•Institutions (3)

University of Illinois at Chicago¹, University of Notre Dame², Michigan State University³

01 Apr 2015-Leadership Quarterly

TL;DR: A 7-item measure of global servant leadership, based on Liden, Wayne, Zhao, and Henderson's (2008) 28-item servant leadership measure (SL-28), is introduced in this article.

...read moreread less

Abstract: Although research on servant leadership has been expanding over the past several years, a concise, valid scale for assessing global servant leadership has been lacking. In the current investigation a 7-item measure of global servant leadership (SL-7), based on Liden, Wayne, Zhao, and Henderson's (2008) 28-item servant leadership measure (SL-28), is introduced. Psychometric properties of the SL-7 were assessed at the individual level with data collected from 729 undergraduate students, 218 graduate students, and 552 leader–follower dyads from 11 organizations, and at the team level with a study consisting of a total of 71 ongoing intact work teams. Results across three independent studies with six samples showed correlations between the SL-7 and SL-28 scales ranging from .78 to .97, internal consistency reliabilities over .80 in all samples, and significant criterion-related validities for the SL-7 that parallel those found with the SL-28.

...read moreread less

365 citations

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

Common method biases in behavioral research: a critical review of the literature and recommended remedies.

[...]

Philip M. Podsakoff¹, Scott B. MacKenzie¹, Jeong-Yeon Lee¹, Nathan P. Podsakoff²•Institutions (2)

Indiana University¹, University of Florida²

01 Oct 2003-Journal of Applied Psychology

TL;DR: The extent to which method biases influence behavioral research results is examined, potential sources of method biases are identified, the cognitive processes through which method bias influence responses to measures are discussed, the many different procedural and statistical techniques that can be used to control method biases is evaluated, and recommendations for how to select appropriate procedural and Statistical remedies are provided.

...read moreread less

Abstract: Interest in the problem of method biases has a long history in the behavioral sciences. Despite this, a comprehensive summary of the potential sources of method biases and how to control for them does not exist. Therefore, the purpose of this article is to examine the extent to which method biases influence behavioral research results, identify potential sources of method biases, discuss the cognitive processes through which method biases influence responses to measures, evaluate the many different procedural and statistical techniques that can be used to control method biases, and provide recommendations for how to select appropriate procedural and statistical remedies for different types of research settings.

...read moreread less

52,531 citations

Journal Article•DOI•

What Reviewers Should Expect from Authors Regarding Common Method Bias in Organizational Research

[...]

James M. Conway¹, Charles E. Lance²•Institutions (2)

Central Connecticut State University¹, University of Georgia²

22 May 2010-Journal of Business and Psychology

TL;DR: The authors argue that journal reviewers (as well as editors and dissertation or thesis committee members) have to some extent perpetuated misconceptions about common method bias in self-report measures, including (a) relationships between self-reported variables are necessarily and routinely upwardly biased, (b) other-reports (or other methods) are superior to self-reports, and (c) rating sources (e.g., self, other) constitute measurement methods.

...read moreread less

Abstract: We believe that journal reviewers (as well as editors and dissertation or thesis committee members) have to some extent perpetuated misconceptions about common method bias in self-report measures, including (a) that relationships between self-reported variables are necessarily and routinely upwardly biased, (b) other-reports (or other methods) are superior to self-reports, and (c) rating sources (e.g., self, other) constitute measurement methods. We argue against these misconceptions and make recommendations for what reviewers (and others) should reasonably expect from authors regarding common method bias. We believe it is reasonable to expect (a) an argument for why self-reports are appropriate, (b) construct validity evidence, (c) lack of overlap in items for different constructs, and (d) evidence that authors took proactive design steps to mitigate threats of method effects. We specifically do not recommend post hoc statistical control strategies; while some statistical strategies are promising, all have significant drawbacks and some have shown poor empirical results.

...read moreread less

1,916 citations

"Publishing in AMJ—Part 2: Research ..." refers background in this paper

...Podsakoff , MacKenzie, Lee, and Podsakoff (2003) discussed common method variance in detail and also suggested ways to reduce its biasing effects (see also Conway & Lance, 2010)....
[...]

Journal Article•DOI•

Potential Problems in the Statistical Control of Variables in Organizational Research: A Qualitative Analysis With Recommendations

[...]

Thomas E. Becker¹•Institutions (1)

University of Delaware¹

01 Jul 2005-Organizational Research Methods

TL;DR: Analysis of statistical control in a random sample of 60 articles published in four top journals during 2000 to 2002 found potential problems included a lack of explanations for inclusion, unclear descriptions of measures and methods, incomplete reporting, and other flaws.

...read moreread less

Abstract: The author examines statistical control in a random sample of 60 articles published in four top journals during 2000 to 2002. Authors’bases for including control variables, clarity regarding measur...

...read moreread less

1,749 citations

"Publishing in AMJ—Part 2: Research ..." refers background or methods in this paper

...Control variables should meet three conditions for inclusion in a study (Becker, 2005; James, 1980)....
[...]
...However, if control variables are included that don't meet these three tests, they may hamper the study by unnecessarily soaking up degrees of freedom or bias the findings related to the hypothesized variables (increasing either type I or type II error) (Becker, 2005)....
[...]

Journal Article•DOI•

A contrarian view of the five-factor approach to personality description.

[...]

Jack Block

01 Mar 1995-Psychological Bulletin

TL;DR: In this article, various misgivings about the FFA are delineated and implications of these problems are drawn.

...read moreread less

Abstract: The 5-factor approach (FFA) to personality description has been represented as a comprehensive and compelling rubric for assessment. In this article, various misgivings about the FFA are delineated. The algorithmic method of factor analysis may not provide dimensions that are incisive. The "discovery" of the five factors may be influenced by unrecognized constraints on the variable sets analyzed. Lexical analyses are based on questionable conceptual and methodological assumptions, and have achieved uncertain results. The questionnaire version of the FFA has not demonstrated the special merits and sufficiencies of the five factors settled upon. Serious uncertainties have arisen in regard to the claimed 5-factor structure and the substantive meanings of the factors. Some implications of these problems are drawn.

...read moreread less

1,551 citations

"Publishing in AMJ—Part 2: Research ..." refers background in this paper

...…the definition and boundaries of the new construct , map its association with existing constructs, and avoid assumptions that scales with the same name reflect the same construct and that scales with different names reflect different constructs (i.e., jingle jangle fallacies [Block, 1995])....
[...]

Journal Article•DOI•

The two faces of transformational leadership: Empowerment and dependency.

[...]

Ronit Kark¹, Boas Shamir, Gilad Chen²•Institutions (2)

Bar-Ilan University¹, Georgia Institute of Technology²

01 Apr 2003-Journal of Applied Psychology

TL;DR: Using a sample of 888 bank employees working under 76 branch manages, the authors found that transformational leadership was positively related to both followers' dependence and their empowerment and that personal identification mediated the relationship betweentransformational leadership and followers' dependent on the leader.

...read moreread less

Abstract: Followers' identification with the leader and the organizational unit, dependence on the leader, and empowerment by the leader are often attributed to transformational leadership in organizations. However, these hypothesized outcomes have received very little attention in empirical studies. Using a sample of 888 bank employees working under 76 branch manages, the authors tested the relationships between transformational leadership and these outcomes. They found that transformational leadership was positively related to both followers' dependence and their empowerment and that personal identification mediated the relationship between transformational leadership and followers' dependence on the leader, whereas social identification mediated the relationship between transformational leadership and followers' empowerment. The authors discuss the implications of these findings to both theory and practice.

...read moreread less

1,287 citations