scispace - formally typeset

Journal ArticleDOI

Using Biodata to Select Air Traffic Controllers

01 Sep 2013-Vol. 57, Iss: 1, pp 823-827

AbstractBiodata factors were examined as predictors of training performance for candidate air traffic control specialists (ATCSs). These factors, which have been shown to predict controller training performance in previous research, were highest educational degree achieved, grade point average both in high school overall and in high school math courses, aviation operations experience, pilot licenses held, and age. Results from logistic regression analyses were only partially supportive of previous research. Age was the most consistent (inverse) predictor of training success. Most of the other factors did not predict training success. Differences between these results and previous research might be attributed to differences in the criterion measures, samples, and generational differences. Overall, the evidence for using the assessed biodata factors for selection was weak. We suggested that a new biodata instrument be developed to assess and identify experiences to predict performance of the next generation of cont...

Topics: Biodata (65%)

Summary (3 min read)

Jump to: [INTRODUCTION][Sample][Measures][Analyses][En Route][Terminal][DISCUSSION][CTI Graduates][Option][Limitation] and [CONCLUSIONS]

INTRODUCTION

  • The selection of air traffic control specialists (ATCSs; referred to as controllers) has been the subject of on-going research by the Federal Aviation Administration (FAA) from its beginnings in 1958.
  • Two recent events within the FAA have led to renewed interest in biodata as a predictor of controller training performance.
  • The IRP recommended that the FAA incorporate these factors into its air traffic controller recruitment and selection processes.
  • AT-SAT has been found to be a hiring “barrier” for African-American, Hispanic/Latino, and female applicants (Outz & Hanges, 2012).

Sample

  • The sample for this study was a group of candidate air traffic controllers hired by the FAA and entering the FAA Academy from February 2007 through December 2011 (N =2,662).
  • Controllers within terminal facilities organize the flow of air traffic into and out of airports.
  • As indicated in the FAA’s (2014) most recent controller workforce plan, the training target time by size of terminal facility is 17 months for small (levels 4-6), 24 months for medium (levels 7-9), and 29 months for large (levels 10-12) facilities.
  • There are many reasons why trainees may request a transfer.
  • The unsuccessful category is for trainees that did not complete training; however, lack of success was not due to poor performance but rather some issue, such as failing a medical exam or security screening, retirement, or even death.

Measures

  • There were seven predictors and one criterion measure used in this study.
  • For their criterion measure, the authors used training outcome data from both the FAA Academy and the NTD to determine the training status (successful or unsuccessful) of the trainees in their sample as of June 2014.
  • If the authors excluded age from the analyses, one or more of the biodata items may predict training status merely due to its association with age.
  • BQ items selected for analysis were those shown to either predict the training performance of candidate controllers in the past and/or to assess a subset of the proposed quality ranking factors.
  • Trainees assigned to an en route center were successful less often than expected (652/690) and unsuccessful more often than expected (284/247).

Analyses

  • Logistic regression was used to model the relationship of ATSAT score, controller age upon entering the FAA Academy, and the five-biodata items to training status of trainees assigned to an en route center or terminal facility.
  • For both analyses, AT-SAT score and age were continuous variables.
  • The authors excluded trainees with missing data on any of the variables entered into the model.
  • The final number of trainees included in the analyses was 1,900 (841 en route center trainees; 1,059 terminal facility trainees).
  • Shown in Table 3 are the values and frequency counts for the biodata items entered into the logistic regression.

En Route

  • For trainees assigned to an en route center, the initial logistic regression model of AT-SAT score, age at entry on duty, and biodata items on training status resulted in correct classifications of 64.6% of the trainees.
  • Thus, the authors interpreted the original model with the outliers.
  • A significant relationship was found between two of the assessed biodata items and training status after accounting for AT-SAT score and age, as indicated by the chi-square for the third block (χ2 (2) = 21.48, p < .001).
  • To test the linearity of the logit, the authors ran the logistic regression analysis again, including predictors that were the interaction of each continuous variables and the log of itself (Field, 2009).
  • In their test for multicollinearity, all tolerance values were greater than .1, and VIF values were less than 10.

Terminal

  • For trainees assigned to a terminal facility, the initial logistic regression model of AT-SAT score, age at entry on duty, and biodata items on training status resulted in correct classifications of 61.3% of the terminal trainees.
  • An analysis of the data, with 13 outliers (studentized residuals greater than 2.58 or less than -2.58) removed, resulted in a model in which 64.1% of the trainees were correctly classified.
  • In reviewing both regression models (with or without the outliers), the authors found that the same variables were included and excluded in the models.
  • Nevertheless, as shown in Table 5, the logistic regression coefficients for AT-SAT score and age were statistically significant.
  • There was no relationship between any of the biodata items and training status for the trainees assigned to a terminal facility.

DISCUSSION

  • In the current research, the authors investigated the validity of five biodata items as predictors of controller training status after accounting for aptitude (e.g., AT-SAT score) and age at entry on duty for trainees assigned to en route centers and terminal facilities.
  • As expected, the authors found AT-SAT score and age were related to training status for trainees assigned to both en route centers and terminal facilities.
  • It was somewhat surprising that the authors did not find high school GPA or having a degree from a CTI school to predict training status.
  • It is possible that by including HS math GPA in this study, which the authors found to predict training status of en route trainees, they obscured the relationship between training status and high school GPA seen in previous research.
  • It is also possible that the criterion measure used in this study, which differed from previous research studies, had an impact.

CTI Graduates

  • One potential explanation for these findings is that CTI graduates were more likely than non-CTI graduates to be assigned to more complex facilities (e.g., en route centers and higher-level terminal facilities) and thus, less likely to succeed in training than non-CTI graduates.
  • The proportion of CTI graduates assigned to an en route center, rather than a terminal facility, was 47.9%.
  • With large sample sizes, such as the authors have here, even small differences may be statistically significant.
  • There are 36 schools in the program, and except for a common core in aviation-education supplied and required by the FAA, the schools vary in what they teach and even how the information is taught.
  • Thus, it is possible that training performance of CTI graduates was influenced by differences in the CTI programs and the processes used to select from the CTI applicants.

Option

  • Results for the other biodata items, specifically HS math GPA and holding any pilot’s certificate, differed between options.
  • A brief description extracted from the FAA’s controller workforce plan (FAA, 2014) is provided in Table 6 (also see Broach, 2013).
  • Controllers in a combination tower and TRACON facility use both visual means and radar to control traffic within their terminal airspace.
  • Results from job analyses have indicated that the aptitudes (work requirements) required to control air traffic are similar across option (en route and terminal) and facility types within the terminal option (Nickels, Bobko, Blair, Sands, & Tartak, 1995).

Limitation

  • A limitation of this research was the use of only one criterion measure related to training status.
  • There is a need to consider multiple outcome measures, especially measures related to onthe-job performance of air traffic controllers.
  • Ultimately, the goal is to select, place, and train successful air traffic controllers, not merely identify candidates likely to succeed in training.

CONCLUSIONS

  • The authors findings are somewhat consistent with previous research, especially their findings regarding predictors of success in training as an en route controller.
  • The authors findings also provide additional support for the utility and validity of AT-SAT in predicting training status of trainees (Broach et al., 2013).
  • The authors results demonstrated yet again an inverse relationship between age at entry on duty and training performance.
  • Younger controllers performed better in training than did older controllers.
  • The FAA makes exceptions to the age policy for retired military controllers and previous FAA controllers.

Did you find this useful? Give us your feedback

...read more

Content maybe subject to copyright    Report

Using Biodata to Select
Air Traffic Controllers
Linda G. Pierce
Dana Broach
Cristina L. Byrne
M. Kathryn Bleckley
Civil Aerospace Medical Institute
Federal Aviation Administration
Oklahoma City, OK 73125
October 2014
Final Report
DOT/FAA/AM-14/8
Ofce of Aerospace Medicine
Washington, DC 20591
Federal Aviation
Administration

NOTICE
This document is disseminated under the sponsorship
of the U.S. Department of Transportation in the interest
of information exchange. The United States Government
assumes no liability for the contents thereof.
___________
This publication and all Office of Aerospace Medicine
technical reports are available in full-text from the
Federal Aviation Administration website.

i
Technical Report Documentation Page
1. Report No.
2. Government Accession No.
3. Recipient's Catalog No.
DOT/FAA/AM-14/8
4. Title and Subtitle
5. Report Date
Using Biodata to Select Air Traffic Controllers
October 2014
6. Performing Organization Code
7. Author(s)
8. Performing Organization Report No.
Pierce LG, Broach D, Byrne C, Bleckley MK
10. Work Unit No. (TRAIS)
P.O. Box 25082
Oklahoma City, OK 73125
11. Contract or Grant No.
13. Type of Report and Period Covered
Office of Aerospace Medicine
Federal Aviation Administration
800 Independence Ave., S.W.
Washington, DC 20591
14. Sponsoring Agency Code
15. Supplemental Notes
Work was accomplished under approved task AM-B-11-HRR-523
16. Abstract
We examined biographical data (biodata) as predictors of training status (successful or unsuccessful) for
candidate air traffic control specialists (ATCSs): self-reported high school grade point average (GPA), high
school GPA in mathematics, highest educational degree achieved, completing an aviation program from a
school in the FAA’s collegiate training initiative program, and holding any pilot certificate. These factors
have been shown to predict controller training success in previous research or are being considered for use
as quality rating factors in controller selection.
Method. We computed separate logistic regression equations for en route and terminal trainees. Score on
the Air Traffic-Selection and Training (AT-SAT) test battery and age at entry on duty was entered first and
second into the equations. Finally, we entered the biodata items using a forward stepwise selection method.
Success in training, first at the FAA Academy and subsequently at the trainee’s first facility, was the
criterion measure.
Results. Results were only partially supported by previous research. As expected, AT-SAT score was a
significant predictor of training success in both regression models. Trainees with higher AT-SAT scores
were more likely to complete training successfully than trainees with lower AT-SAT scores. Also, and as
expected, age was inversely related to training success in both models. Younger trainees were more likely to
complete training successfully than older trainees were. En route trainees with a self-reported high school
math GPA of A and those with any type of pilot certificate were more likely to succeed in training than
trainees with a high school math GPA less than an A and/or without any type of pilot certificate. For
terminal trainees, no biodata items added to AT-SAT score and age in predicting training success.
Discussion. Based on an analysis of the relationship between selected biodata items and training success,
we conclude that the evidence for using these biodata items for controller selection is weak. We recommend
that if biodata are used to select ATCSs, additional research is needed to identify and validate items
predictive of success in training. We also recommend that a criterion measure representative of job
performance of air traffic controllers be developed and validated for use in future research on the selection
of air traffic controllers.
17. Key Words
18. Distribution Statement
ATCS Selection, Air Traffic Control, Biodata,
Biographical Data
Document is available to the public
through the Internet:
www.faa.gov/go/oamtechreports
19. Security Classif. (of this report)
20. Security Classif. (of this page)
21. No. of Pages
22. Price
Unclassified
Unclassified
13
Form DOT F 1700.7 (8-72) Reproduction of completed page authorized


iii
ACKNOWLEDGMENTS
Research reported in this paper was conducted under the Air Trac Program Directive/Level of Eort Agreement
between the Human Factors Division (ANG-C1), FAA Headquarters, and the Aerospace Human Factors Research Division
(AAM-500) of the Civil Aerospace Medical Institute.
e opinions expressed are those of the authors alone, and do not necessarily reect those of the Federal Aviation
Administration, the Department of Transportation, or federal government of the United States.
Address correspondence concerning this report to Linda Pierce, Aerospace Human Factors Research Division (AAM-
500), P.O. Box 25082, Oklahoma City, OK 73125. Email: linda.pierce@faa.gov.

Citations
More filters


Journal ArticleDOI
TL;DR: This paper makes advances in computational methodology for assessment of physiological features of stress resilience, and investigates the predictive power of the obtained feature space in a binary classification problem: prediction of high- vs. low-performance on the developed ATC simulator.
Abstract: In this paper, we investigate the potential of generic physiological features of stress resilience in predicting air traffic control (ATC) candidates’ performance in a highly-stressful low-fidelity ATC simulator scenario. Stress resilience is highlighted as an important occupational factor that influences the performance and well-being of air traffic control officers (ATCO). Poor stress management, besides the lack of skills, can be a direct cause of poor performance under stress, both in the selection process of ATCOs and later in the workplace. 40 ATC candidates, within the final stages of their selection process, underwent a stimulation paradigm for elicitation and assessment of various generic task-unrelated physiological features, related to resting heart rate variability (HRV) and respiratory sinus arrhythmia (RSA), acoustic startle response (ASR) and the physiological allostatic response, which are all recognized as relevant psychophysiological markers of stress resilience. The multimodal approach included analysis of electrocardiography, electromyography, electrodermal activity and respiration. We make advances in computational methodology for assessment of physiological features of stress resilience, and investigate the predictive power of the obtained feature space in a binary classification problem: prediction of high- vs. low-performance on the developed ATC simulator. Our novel approach yields a relatively high 78.16% classification accuracy. These results are discussed in the context of prior work, while considering study limitations and proposing directions for future work.

3 citations


Cites background from "Using Biodata to Select Air Traffic..."

  • ...didates’ advancement in the selection process include age, intelligence, attention and multitasking ability, as well as general motivation [12], [13]....

    [...]


Journal ArticleDOI
01 Sep 2015
Abstract: An assessment was completed of training outcomes for air traffic control (ATC) trainees allowed to transfer from their first, higher-level ATC facility to a less complex lower-level ATC facility. Transfers followed failure in field qualification training at the first facility. We found that training outcomes at the second facility were related to the types and complexity levels of the ATC facilities involved. Trainees succeeded significantly more often if transferred to small or medium towers than if transferred to a facility that combined tower and radar. We considered an inability of trainees to acquire radar skills to separate air traffic and age as contributing factors and suggested that additional research examining age upon entry and training success at facilities of varying complexity should be undertaken. This effort supports a Federal Aviation Administration (FAA) strategic priority to build the workforce of the future by retaining trainees capable of controlling air traffic.

Cites background from "Using Biodata to Select Air Traffic..."

  • ...As mentioned previously, a significant amount of research has suggested that age has a substantial impact on success rates (see Pierce et al., 2014)....

    [...]

  • ...Researchers have found age at entry to be a consistent and powerful (inverse) predictor of training performance (see Pierce et al., 2014, for a review)....

    [...]



Journal ArticleDOI
01 Sep 2018
Abstract: Training a developmental (air traffic controller in training) to certification is a lengthy and expensive process. To limit losses due to poor performance in training, the FAA has implemented a policy to reassign some developmentals to less complex air traffic control (ATC) facilities following voluntary or involuntary removal from training at more complex facilities. The National Employee Service Team (NEST) is implementing the policy. We reviewed the NEST processes and assessed effectiveness based on the extent to which the NEST recommendations are accepted and the likelihood that developmentals reassigned in accordance with NEST recommendations succeed in training at the new facility. We found that NEST recommendations to retain or terminate developmentals and type and level facility for reassignment are accepted by the FAA at a high level. Furthermore, the success rate of developmentals at a new facility is greater than 90%.

References
More filters

Book
01 Jan 2000
TL;DR: Suitable for those new to statistics as well as students on intermediate and more advanced courses, the book walks students through from basic to advanced level concepts, all the while reinforcing knowledge through the use of SAS(R).
Abstract: Hot on the heels of the 3rd edition of Andy Field's award-winning Discovering Statistics Using SPSS comes this brand new version for students using SAS(R). Andy has teamed up with a co-author, Jeremy Miles, to adapt the book with all the most up-to-date commands and programming language from SAS(R) 9.2. If you're using SAS(R), this is the only book on statistics that you will need! The book provides a comprehensive collection of statistical methods, tests and procedures, covering everything you're likely to need to know for your course, all presented in Andy's accessible and humourous writing style. Suitable for those new to statistics as well as students on intermediate and more advanced courses, the book walks students through from basic to advanced level concepts, all the while reinforcing knowledge through the use of SAS(R). A 'cast of characters' supports the learning process throughout the book, from providing tips on how to enter data in SAS(R) properly to testing knowledge covered in chapters interactively, and 'real world' and invented examples illustrate the concepts and make the techniques come alive. The book's companion website (see link above) provides students with a wide range of invented and real published research datasets. Lecturers can find multiple choice questions and PowerPoint slides for each chapter to support their teaching.

24,385 citations


Journal ArticleDOI
Abstract: This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and Ihe 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed. From the point of view of practical value, the most important property of a personnel assessment method is predictive validity: the ability to predict future job performance, job-related learning (such as amount of learning in training and development programs), and other criteria. The predictive validity coefficient is directly proportional to the practical economic value (utility) of the assessment method (Brogden, 1949; Schmidt, Hunter, McKenzie, & Muldrow, 1979). Use of hiring methods with increased predictive validity leads to substantial increases in employee performance as measured in percentage increases in output, increased monetary value of output, and increased learning of job-related skills (Hunter, Schmidt, & Judiesch, 1990). Today, the validity of different personnel measures can be determined with the aid of 85 years of research. The most wellknown conclusion from this research is that for hiring employees without previous experience in the job the most valid predictor of future performance and learning is general mental ability ([GMA], i.e., intelligence or general cognitive ability; Hunter & Hunter, 1984; Ree & Earles, 1992). GMA can be measured using commercially available tests. However, many other measures can also contribute to the overall validity of the selection process. These include, for example, measures of

3,582 citations


Journal ArticleDOI
Abstract: Mean subgroup (gender, ethnic/cultural, and age) differences are summarized across studies for several predictor domains – cognitive ability, personality and physical ability – at both broadly and more narrowly defined construct levels, with some surprising results. Research clearly indicates that the setting, the sample, the construct and the level of construct specificity can all, either individually or in combination, moderate the magnitude of differences between groups. Employers using tests in employment settings need to assess accurately the requirements of work. When the exact nature of the work is specified, the appropriate predictors may or may not have adverse impact against some groups. The possible causes and remedies for adverse impact (measurement method, culture, test coaching, test-taker perceptions, stereotype threat and criterion conceptualization) are also summarized. Each of these factors can contribute to subgroup differences, and some appear to contribute significantly to subgroup differences on cognitive ability tests, where Black–White mean differences are most pronounced. Statistical methods for detecting differential prediction, test fairness and construct equivalence are described and evaluated, as are statistical/mathematical strategies for reducing adverse impact (test-score banding and predictor/criterion weighting strategies).

355 citations


"Using Biodata to Select Air Traffic..." refers background in this paper

  • ...However, it is probably the type of job assessed, as well as the items used to predict performance, that will influence the likelihood and degree of subgroup differences in biodata (Hough et al., 2001; Imus et al., 2011)....

    [...]


BookDOI
29 Mar 2012
Abstract: Part One: Introduction and Overview Conceptual Outline of the Handbook of Assessment and Selection Neal Schmitt Part Two: Historical and Social Context of Selection and the Nature of Individual Differences A History of Personnel Selection and Assessment Andrew J. Vinchur and Laura L. Koppes Bryan Individual Differences Kevin R. Murphy The Social and Organizational Context of Personnel Selection Robert E. Ployhart and Benjamin Schneider Employee Recruitment: Current Knowledge and Suggestions for Future Research James A. Breaugh Part Three: Research Strategies The Concept of Validity and the Process of Validation Paul R. Sackett, Dan J. Putka, and Rodney A. McCloy Job Analysis for KSAOs, Predictor Measures, and Performance Outcomes Michael T. Brannick, Adrienne Cadle, and Edward L. Levine Organizational Strategy and Staffing John P. Hausknecht and Patrick M. Wright Meta-Analysis as a Validity Summary Tool George C. Banks and Michael A. McDaniel Part Four: Individual Difference Construct Cognitive Abilities Deniz S. Ones, Stephan Dilchert, and Chockalingam Viswesvaran Nature and Use of Personality in Selection Murray R. Barrick and Michael K. Mount Person-Environment Fit in the Selection Process Cheri Ostroff and Yujie Zhan The Assessment of Physical Capabilities in the Workplace Todd A. Baker and Deborah L. Gebhardt Using Composite Predictors in Personnel Selection Kate Hattrup Part Five: Measures of Predictor Construct The Selection Interview from the Interviewer and Applicant Perspectives: Can't Have One without the Other Robert L. Dipboye, Therese Macan, and Comila Shahani-Denning Background data: Use of experiential knowledge in personnel selection Michael D. Mumford, Jamie D. Barrett, and Kimberly S. Hester Simulations Filip Lievens and Britt De Soete Individual Psychological Assessment S. Morton McPhail and P. Richard Jeanneret Self-Reports For Employee Selection Paul E. Spector Predictive Bias in Work and Educational Settings Nathan R. Kuncel and David M. Klieger Web-Based Assessments John C. Scott and Daniel V. Lezotte Part Six: Performance and Outcomes Assessment Supervisory Performance Ratings David J. Woehr and Sylvia Roch The Use of Objective Measures as Criteria in I/O Psychology Walter C. Borman and Tiffany N. Smith A Review of Citizenship and Counterproductive Behaviors in Organizational Decision-Making Brian J. Hoffman and Stephan Dilchert Assessment of Voluntary Turnover in Organizations: Answering the Questions of Why, Who, and How Much Sang Eun Woo and Carl P. Maertz, Jr. Adaptive Performance and Trainability as Criteria in Selection Research Elaine D. Pulakos, Rose A. Mueller-Hanson, and Johnathan K. Nelson Occupational Safety J. Craig Wallace, Jeffrey B. Paul, Ronald S. Landis, and Stephen J. Vodanovich Part Seven: Societal and Organizational Constraints on Selection Applicant Reactions to Testing and Selection Stephen W. Gilliland and Dirk D. Steiner Multilevel Selection and the Paradox of Sustained Competitive Advantage Robert E. Ployhart Legal Constraints on Personnel Selection Decisions Arthur Gutman Time in Personnel Selection Margaret E. Beier and Phillip L. Ackerman Personnel Selection across the Globe Dirk D. Steiner Employee Value: Combining Utility Analysis with Strategic Human Resource Management Research to Yield Strong Theory Michael C. Sturman "Retooling" Evidence-Based Staffing: Extending the Validation Paradigm Using Management Mental Models John W. Boudreau Workplace Diversity Ann Marie Ryan and Charlotte Powers Team Selection Frederick P. Morgeson, Stephen E. Humphrey, and Matthew C. Reeder Selection Out: How Firms Choose Workers to Lay Off Daniel C. Feldman and Thomas W.H. Ng Contingent Workers: Who Are They and How Can We Select Them for Success? Talya N. Bauer, Donald M. Truxillo, Layla R. Mansfield, and Berrin Erdogan Part Eight: Implementation and Sustainability of Selection Systems Implementation Issues in Employee Selection Testing Nancy T. Tippins The Life Cycle of Successful Selection Programs Jerard Kehoe, Steven Brown, and Calvin Hoffman Part Nine: Conclusion and Future Directions Theoretical and Practical Issues: Research Needs Neal Schmitt and Catherine Ott-Holland

133 citations


Journal ArticleDOI
Abstract: Although biodata has been shown to be one of the best predictors of employee performance and turnover, a number of important issues remain unresolved (e.g., how broadly or narrowly should biodata be defined?). This paper has three main objectives. The first objective is to provide a selective but representative review of the research that has been conducted on the use of biodata for employee selection. The second objective is to constructively critique this research. This critique is intended to highlight deficiencies of this research that may limit the conclusions that should be drawn. The paper's third objective is to stimulate important future research on biodata that avoids the limitations of past research.

52 citations


Frequently Asked Questions (1)
Q1. What are the contributions mentioned in the paper "Using biodata to select air traffic controllers" ?

In a recent study, Broach et al. this paper used the CBAS as a predictor of controller training performance.