Using Biodata to Select Air Traffic Controllers

TL;DR: In this article, the authors examined the use of biodata factors as predictors of training performance for candidate air traffic control specialists (ATCSs) and found that age was the most consistent predictor of training success.

Abstract: Biodata factors were examined as predictors of training performance for candidate air traffic control specialists (ATCSs). These factors, which have been shown to predict controller training performance in previous research, were highest educational degree achieved, grade point average both in high school overall and in high school math courses, aviation operations experience, pilot licenses held, and age. Results from logistic regression analyses were only partially supportive of previous research. Age was the most consistent (inverse) predictor of training success. Most of the other factors did not predict training success. Differences between these results and previous research might be attributed to differences in the criterion measures, samples, and generational differences. Overall, the evidence for using the assessed biodata factors for selection was weak. We suggested that a new biodata instrument be developed to assess and identify experiences to predict performance of the next generation of cont...

Summary

INTRODUCTION

• The selection of air traffic control specialists (ATCSs; referred to as controllers) has been the subject of on-going research by the Federal Aviation Administration (FAA) from its beginnings in 1958.
• Two recent events within the FAA have led to renewed interest in biodata as a predictor of controller training performance.
• The IRP recommended that the FAA incorporate these factors into its air traffic controller recruitment and selection processes.
• AT-SAT has been found to be a hiring “barrier” for African-American, Hispanic/Latino, and female applicants (Outz & Hanges, 2012).

Sample

• The sample for this study was a group of candidate air traffic controllers hired by the FAA and entering the FAA Academy from February 2007 through December 2011 (N =2,662).
• Controllers within terminal facilities organize the flow of air traffic into and out of airports.
• As indicated in the FAA’s (2014) most recent controller workforce plan, the training target time by size of terminal facility is 17 months for small (levels 4-6), 24 months for medium (levels 7-9), and 29 months for large (levels 10-12) facilities.
• There are many reasons why trainees may request a transfer.
• The unsuccessful category is for trainees that did not complete training; however, lack of success was not due to poor performance but rather some issue, such as failing a medical exam or security screening, retirement, or even death.

Measures

• There were seven predictors and one criterion measure used in this study.
• For their criterion measure, the authors used training outcome data from both the FAA Academy and the NTD to determine the training status (successful or unsuccessful) of the trainees in their sample as of June 2014.
• If the authors excluded age from the analyses, one or more of the biodata items may predict training status merely due to its association with age.
• BQ items selected for analysis were those shown to either predict the training performance of candidate controllers in the past and/or to assess a subset of the proposed quality ranking factors.
• Trainees assigned to an en route center were successful less often than expected (652/690) and unsuccessful more often than expected (284/247).

Analyses

• Logistic regression was used to model the relationship of ATSAT score, controller age upon entering the FAA Academy, and the five-biodata items to training status of trainees assigned to an en route center or terminal facility.
• For both analyses, AT-SAT score and age were continuous variables.
• The authors excluded trainees with missing data on any of the variables entered into the model.
• The final number of trainees included in the analyses was 1,900 (841 en route center trainees; 1,059 terminal facility trainees).
• Shown in Table 3 are the values and frequency counts for the biodata items entered into the logistic regression.

En Route

• For trainees assigned to an en route center, the initial logistic regression model of AT-SAT score, age at entry on duty, and biodata items on training status resulted in correct classifications of 64.6% of the trainees.
• Thus, the authors interpreted the original model with the outliers.
• A significant relationship was found between two of the assessed biodata items and training status after accounting for AT-SAT score and age, as indicated by the chi-square for the third block (χ2 (2) = 21.48, p < .001).
• To test the linearity of the logit, the authors ran the logistic regression analysis again, including predictors that were the interaction of each continuous variables and the log of itself (Field, 2009).
• In their test for multicollinearity, all tolerance values were greater than .1, and VIF values were less than 10.

Terminal

• For trainees assigned to a terminal facility, the initial logistic regression model of AT-SAT score, age at entry on duty, and biodata items on training status resulted in correct classifications of 61.3% of the terminal trainees.
• An analysis of the data, with 13 outliers (studentized residuals greater than 2.58 or less than -2.58) removed, resulted in a model in which 64.1% of the trainees were correctly classified.
• In reviewing both regression models (with or without the outliers), the authors found that the same variables were included and excluded in the models.
• Nevertheless, as shown in Table 5, the logistic regression coefficients for AT-SAT score and age were statistically significant.
• There was no relationship between any of the biodata items and training status for the trainees assigned to a terminal facility.

DISCUSSION

• In the current research, the authors investigated the validity of five biodata items as predictors of controller training status after accounting for aptitude (e.g., AT-SAT score) and age at entry on duty for trainees assigned to en route centers and terminal facilities.
• As expected, the authors found AT-SAT score and age were related to training status for trainees assigned to both en route centers and terminal facilities.
• It was somewhat surprising that the authors did not find high school GPA or having a degree from a CTI school to predict training status.
• It is possible that by including HS math GPA in this study, which the authors found to predict training status of en route trainees, they obscured the relationship between training status and high school GPA seen in previous research.
• It is also possible that the criterion measure used in this study, which differed from previous research studies, had an impact.

CTI Graduates

• One potential explanation for these findings is that CTI graduates were more likely than non-CTI graduates to be assigned to more complex facilities (e.g., en route centers and higher-level terminal facilities) and thus, less likely to succeed in training than non-CTI graduates.
• The proportion of CTI graduates assigned to an en route center, rather than a terminal facility, was 47.9%.
• With large sample sizes, such as the authors have here, even small differences may be statistically significant.
• There are 36 schools in the program, and except for a common core in aviation-education supplied and required by the FAA, the schools vary in what they teach and even how the information is taught.
• Thus, it is possible that training performance of CTI graduates was influenced by differences in the CTI programs and the processes used to select from the CTI applicants.

Option

• Results for the other biodata items, specifically HS math GPA and holding any pilot’s certificate, differed between options.
• A brief description extracted from the FAA’s controller workforce plan (FAA, 2014) is provided in Table 6 (also see Broach, 2013).
• Controllers in a combination tower and TRACON facility use both visual means and radar to control traffic within their terminal airspace.
• Results from job analyses have indicated that the aptitudes (work requirements) required to control air traffic are similar across option (en route and terminal) and facility types within the terminal option (Nickels, Bobko, Blair, Sands, & Tartak, 1995).

Limitation

• A limitation of this research was the use of only one criterion measure related to training status.
• There is a need to consider multiple outcome measures, especially measures related to onthe-job performance of air traffic controllers.
• Ultimately, the goal is to select, place, and train successful air traffic controllers, not merely identify candidates likely to succeed in training.

CONCLUSIONS

• The authors findings are somewhat consistent with previous research, especially their findings regarding predictors of success in training as an en route controller.
• The authors findings also provide additional support for the utility and validity of AT-SAT in predicting training status of trainees (Broach et al., 2013).
• The authors results demonstrated yet again an inverse relationship between age at entry on duty and training performance.
• Younger controllers performed better in training than did older controllers.
• The FAA makes exceptions to the age policy for retired military controllers and previous FAA controllers.

Figures

Table 4. Predictors of Status in En Route Training

Table 1. Training Status as of March 2014

Table 5. Predictors of Status in Terminal Training

Table 3. Values and Frequency Counts for Predictor Variables by Facility Option

Table 2. Training Status by Option

Table 6. Types of Terminal Facilities

Table 7. Training Status by Terminal Facility Type

Full text

Using Biodata to Select
Air Traffic Controllers
Linda G. Pierce
Dana Broach
Cristina L. Byrne
M. Kathryn Bleckley
Civil Aerospace Medical Institute
Federal Aviation Administration
Oklahoma City, OK 73125
October 2014
Final Report
DOT/FAA/AM-14/8
Ofce of Aerospace Medicine
Washington, DC 20591
Federal Aviation
Administration

NOTICE
This document is disseminated under the sponsorship
of the U.S. Department of Transportation in the interest
of information exchange. The United States Government
assumes no liability for the contents thereof.
___________
This publication and all Office of Aerospace Medicine
technical reports are available in full-text from the
Federal Aviation Administration website.

i
Technical Report Documentation Page
1. Report No.
2. Government Accession No.
3. Recipient's Catalog No.
DOT/FAA/AM-14/8
4. Title and Subtitle
5. Report Date
Using Biodata to Select Air Traffic Controllers
October 2014
6. Performing Organization Code
7. Author(s)
8. Performing Organization Report No.
Pierce LG, Broach D, Byrne C, Bleckley MK
9. Performing Organization Name and Address
10. Work Unit No. (TRAIS)
FAA Civil Aerospace Medical Institute
P.O. Box 25082
Oklahoma City, OK 73125
11. Contract or Grant No.
12. Sponsoring Agency name and Address
13. Type of Report and Period Covered
Office of Aerospace Medicine
Federal Aviation Administration
800 Independence Ave., S.W.
Washington, DC 20591
14. Sponsoring Agency Code
15. Supplemental Notes
Work was accomplished under approved task AM-B-11-HRR-523
16. Abstract
We examined biographical data (biodata) as predictors of training status (successful or unsuccessful) for
candidate air traffic control specialists (ATCSs): self-reported high school grade point average (GPA), high
school GPA in mathematics, highest educational degree achieved, completing an aviation program from a
school in the FAA’s collegiate training initiative program, and holding any pilot certificate. These factors
have been shown to predict controller training success in previous research or are being considered for use
as quality rating factors in controller selection.
Method. We computed separate logistic regression equations for en route and terminal trainees. Score on
the Air Traffic-Selection and Training (AT-SAT) test battery and age at entry on duty was entered first and
second into the equations. Finally, we entered the biodata items using a forward stepwise selection method.
Success in training, first at the FAA Academy and subsequently at the trainee’s first facility, was the
criterion measure.
Results. Results were only partially supported by previous research. As expected, AT-SAT score was a
significant predictor of training success in both regression models. Trainees with higher AT-SAT scores
were more likely to complete training successfully than trainees with lower AT-SAT scores. Also, and as
expected, age was inversely related to training success in both models. Younger trainees were more likely to
complete training successfully than older trainees were. En route trainees with a self-reported high school
math GPA of A and those with any type of pilot certificate were more likely to succeed in training than
trainees with a high school math GPA less than an A and/or without any type of pilot certificate. For
terminal trainees, no biodata items added to AT-SAT score and age in predicting training success.
Discussion. Based on an analysis of the relationship between selected biodata items and training success,
we conclude that the evidence for using these biodata items for controller selection is weak. We recommend
that if biodata are used to select ATCSs, additional research is needed to identify and validate items
predictive of success in training. We also recommend that a criterion measure representative of job
performance of air traffic controllers be developed and validated for use in future research on the selection
of air traffic controllers.
17. Key Words
18. Distribution Statement
ATCS Selection, Air Traffic Control, Biodata,
Biographical Data
Document is available to the public
through the Internet:
www.faa.gov/go/oamtechreports
19. Security Classif. (of this report)
20. Security Classif. (of this page)
21. No. of Pages
22. Price
Unclassified
Unclassified
13
Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

iii
ACKNOWLEDGMENTS
Research reported in this paper was conducted under the Air Trac Program Directive/Level of Eort Agreement
between the Human Factors Division (ANG-C1), FAA Headquarters, and the Aerospace Human Factors Research Division
(AAM-500) of the Civil Aerospace Medical Institute.
e opinions expressed are those of the authors alone, and do not necessarily reect those of the Federal Aviation
Administration, the Department of Transportation, or federal government of the United States.
Address correspondence concerning this report to Linda Pierce, Aerospace Human Factors Research Division (AAM-
500), P.O. Box 25082, Oklahoma City, OK 73125. Email: linda.pierce@faa.gov.

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Using biodata to select air traffic controllers" ?

In a recent study, Broach et al. this paper used the CBAS as a predictor of controller training performance.