Academic Research and Reform: A History of the Empirical Basis for NCAA Academic Policy

TLDR: The authors provide an historical overview of the National Collegiate Athletic Association's (NCAA) academic reform, with a particular focus on the empirical basis for the decisions made, and examine the types of information the NCAA has collected and used to make decisions about academic policy.

Abstract: The purpose of this article is to provide an historical overview of the National Collegiate Athletic Association’s (NCAA) academic reform, with a particular focus on the empirical basis for the decisions made. The authors outline four eras of academic reform, examine the types of information the NCAA has collected and used to make decisions about academic policy, and explore the limits of such academic data.

Figures

Figure 1 — First-Year Academic Performance of Low-Test vs. Low-GPA Qualifiers in Division I (2009-10 Division I Freshmen)

Figure 6 — Projected Impacts for Varying GPA (0 to 2.0) and 48 Credit-Hour Requirements in Second Year(Based on Data from 2000 NCAA Graduation Cohort)

Figure 2 — Five-Year Academic Performance of Low-Test vs. Low-GPA Qualifiers in Division I (2003-04 Division I Freshmen)

Figure 5 — Academic Year Credit Hours for Graduates vs. Non-Graduates Persisting Four or More Years

Table 1 Comparison of Federal Graduation Rates Between Student-Athletes and Student Body For Select Groups in 2004 Entering Class

Figure 3 — Best SAT score among PSAs on a Division I IRL in 2002 (Prop 16) vs. 2003- 2005 (Sliding Scale)

Full text

27
Journal of Intercollegiate Sport, 2012, 5, 27-40
© 2012 Human Kinetics, Inc.
Academic Research and Reform:
A History of the Empirical Basis
for NCAA Academic Policy
Todd A. Petr
NCAA
John J. McArdle
University of Southern California
The purpose of this article is to provide an historical overview of the National
Collegiate Athletic Association’s (NCAA) academic reform, with a particular
focus on the empirical basis for the decisions made. The authors outline four eras
of academic reform, examine the types of information the NCAA has collected
and used to make decisions about academic policy, and explore the limits of such
academic data.
We have been conducting research on issues related to the academic success of
student-athletes for the better part of two decades, and it is a pleasure for us to
be able to share some of what we have learned here. We want to be clear that the
data and ndings that will be presented in this paper represent the hard work of
lots of people other than just the authors, and we want to specically acknowledge
Ursula Walsh, Tom Paskus, Steve Boker, James Jackson, and the other members
of the National Collegiate Athletic Association (NCAA) Data Analysis Research
Network, and all our colleagues on the NCAA research staff for their signicant
contributions to this work.
We would also like to acknowledge all of the committed individuals who
generously give their time and talent to sit on NCAA committees and wrestle with
these difcult questions. Academic policy issues within the NCAA are complex and
controversial. Everyone associated with intercollegiate athletics has an opinion on
these issues, and the decisions made by these committees come under intense scru-
tiny. While none of us will agree with every decision that is made by these groups,
the motives of the people who serve are beyond question. Committee members
study reams of data and engage in sometimes-tortured discussions, but they work
incredibly hard to get to the best possible solution for all Division I schools and
students. It can be easy to second-guess their decisions from afar, but the people who
sit in those rooms understand that the decisions they make affect real people in real
Petr is the NCAA Managing Director of Research, Indianapolis, IN. McArdle is with the Psychology
Department, University of Southern California, Los Angeles, CA.

28 Petr and McArdle
ways. That can be a difcult burden, but we have been truly impressed by the way
that NCAA committee members have responded to the challenge over the years.
In this paper, we provide a brief history of academic research at the NCAA
and explain how data have been used to inform policy. We will look at the types of
information the NCAA has collected and used to make decisions about academic
policy, and we will look at the limits of such academic data. At the end of this paper,
it is our hope that you have an appreciation for the complexity of the issues related
to academic success of student-athletes and a better understanding of exactly how
the NCAA has tried to use data to address those issues.
For purposes of this paper, we have divided the history of NCAA academic
research into four “eras.” We will certainly focus our attention on the last two
eras—essentially the last 20 years. However, we want to provide a little bit of the
history to shed some light on how we got to where we are today. The eras we have
identied are not equal in scope of time, nor are they equal in the level of research
activity, as you will see.
The genesis of NCAA academic research is, to a great extent, wrapped around
the question of freshman (or initial) eligibility, as is the case with many elements
of NCAA policy. Dr. Michael Oriard (2012) provides an excellent summary of the
history of freshmen eligibility in his paper, so we will not rehash that here, but the
freshman question certainly has driven much of our academic research, especially
in the early years.
The rst era of academic research that we have identied runs from the begin-
ning of the NCAA in 1906 until about 1980. This might actually be best described
as the prehistory of NCAA academic research. During that time, there was little-
to-no research regarding student-athlete academic performance—at least from a
national perspective. There was some research done surrounding the 1.6 minimum
GPA (grade point average) rule in the 1960s, but most of that work was at the
institutional or conference level. It is fair to say that, to this point, the NCAA did
not use research in a systematic way.
The second era in NCAA academic research comprises the 1980s. In that
decade, there were signicant changes in the eligibility rules made by the Associa-
tion (e.g., Proposition 48), but, in the words of the NCAA’s Director of Research
at the time, Ursula Walsh, the policy was “remarkably free of empirical research.”
There were two primary catalyzing factors that led to the development and impo-
sition of Proposition 48. The rst was a series of high-prole academic scandals
within intercollegiate athletics, including the identication of a few high-level
student-athletes who had attended multiple years of college but were functionally
illiterate. The second was a national movement to improve higher education as a
whole that was exemplied by the inuential governmental report titled “A Nation
at Risk” (National Commission on Excellence in Education, 1983). That report
was released the same year Proposition 48 was passed, and that is probably not
coincidental. Proposition 48 was sponsored as NCAA legislation by the American
Council on Education—a group that was intimately involved with the larger reform
efforts occurring within the higher education community.
Thus, Proposition 48 was adopted within a climate of national educational
reform and with substantial support from the membership, but there was not any
specic research undergirding the legislation. Signicant questions arose on the oor
of the 1983 NCAA Convention, at which Proposition 48 was adopted, about how

NCAA Academic Research and Reform 29
the legislation would impact low-income and minority students. These questions
persisted even after the adoption of the legislation, and at that time there were no
data available to provide empirical answers. In response, the NCAA membership
created a special committee to study the issues, and that committee recommended
that the NCAA undertake a longitudinal study to better understand the impacts of the
new academic regulations. That became the NCAA Academic Performance Study
(APS) and was the NCAA’s rst real effort to collect data from its member schools
on these issues. Data from that study served as the backbone for the inquiries that
were undertaken in the third era of NCAA academic research.
In the 1980s, in the midst of the discussions related to Proposition 48, Ron
Smith and Jay Helman (1987) wrote an excellent paper on the history of the
freshman-eligibility question. In that paper, they identied three sources of ten-
sion that drove historical policy decisions related to the issue: academic integrity,
competitive equity, and nancial considerations. In simple terms, the argument was
that at times and in places where academic integrity was the primary consideration,
freshmen would be ineligible for athletics competition. Conversely, when nancial
issues were driving policy decisions, freshmen would be eligible to compete because
schools would be able to eld competitive teams less expensively by being able
to include freshmen on their rosters. Competitive-equity concerns have been used
to argue this issue from both sides, generally depending on the size of the school
making the argument.
These considerations continue to be germane to any discussion of NCAA
academic policy; however, a fourth very important consideration has been added
to the mix in current discussions of these issues: educational access for low-income
and minority populations. Because of large differences in secondary education in
this country, these rules do not affect all populations of prospective student-athletes
in the same way. One can argue athletics is the route to a college education for
many students from disadvantaged backgrounds and that restricting those students’
access to an athletics scholarship essentially closes the door to a college education
for them. Since the late 1980s, the issue of access has been at the forefront of any
discussion of NCAA initial eligibility regulations.
It was about 1990 when the NCAA had collected sufcient academic data
to begin to conduct its own analyses of impacts of academic policies on student-
athletes. This ushered in what we have identied as the third era of academic
research, and we will concentrate the rest of this paper on our work in this era.
Issues studied during this time included high school academic performance and
initial eligibility, college academic performance and continuing eligibility, and the
best ways to measure team-level academic success. During this era, we also came
to understand certain limits of how data can inform and shape policy. For instance,
we made the important separation between objective indices and subjective deci-
sions. Specically, we found that there was much data that could be informative on
specic issues (e.g., that high school grades and test scores were useful predictors
of college success), but there were limits to the data and points where decisions
would have to be made by group consensus (e.g., exactly where to set cut-scores
on eligibility rules).
Before discussing specics about ndings from our research over the past
two decades, it is important to provide some general background about the data
on which many of these ndings are based. The majority of results from the third

30 Petr and McArdle
era of academic research come from data collected in the APS. In that study, the
NCAA collected both high school and college academic data in a longitudinal
manner from over 10,000 Division I student-athletes across ve cohorts (from
1984 to 1988). Thus, data were collected from two cohorts of student-athletes who
entered college before Proposition 48 was implemented and from three cohorts
postimplementation. In 1994, the NCAA began certifying the eligibility of all
incoming-freshmen student-athletes. And while the driving force behind this effort
was related to compliance and equity concerns, this was a huge boon for our aca-
demic research. The Initial Eligibility Clearinghouse (IEC), as it came to be known,
collected complete high school academic records from over 100,000 prospective
student-athletes each year. (The IEC has since been transformed into the NCAA
Eligibility Center, or NEC, but the data collection remains a very important part of
our research efforts.) Between 1994 and 2002, the NCAA collected college-level
data on Division I student-athletes on a voluntary basis through the Academic Per-
formance Census (APC). In that collection, data were gathered on between 10,000
and 20,000 student-athletes per year. With the implementation of the Academic
Performance Program (APP) in 2003, Division I institutions were required to
submit academic data on all of their scholarship student-athletes. The Association
now receives college-level academic data from over 100,000 student-athletes per
year, and by matching data from the NEC to the APP, the NCAA is able to create
complete longitudinal academic records on Division I student-athletes from the
time they are in high school until they exit the Division I institutions. These data
are the best we are aware of in higher education regarding the complete academic
trajectories of a national sample of high school and college students (for more
information on NCAA data, see Petr & Paskus, 2009).
We will now describe a few specic results from the third era of academic
research at the NCAA. A rst set of ndings has to do with our ability to predict
college success. Over the course of the past two decades, we have run many models
attempting to predict college academic success from precollege variables. Through
most, if not all, of these models, a few ndings stand out:
• High school grades are better predictors of success than standardized test
scores. In most of our models, grades are 2–3 times more predictive than test
scores.
• Acombinationofgradesandtestsisabetterpredictorthaneitherofthetwo
variables used in isolation. The incremental benet of including test scores in
a prediction model is generally small, but meaningful.
• Usingacore-curriculumgradepointaverage(GPA)providesbetterprediction
than using the overall high school GPA. We have noted that our prediction
accuracy has improved as we have increased the number of courses included
in the core-curriculum requirement.
• Demographicvariables,suchasincomeandrace/ethnicity, are important
attributes to consider in the models but are generally accounted for once tests
and grades are included in the model.
• Differentdemographicgroupshavedifferentdistributionsofscoreswithinthe
variables of use (e.g., test scores and grades), so the imposition of almost any
rule will lead to differential impacts on various subgroups. Generally speaking,
African American and low-income students are more likely to be impacted

NCAA Academic Research and Reform 31
byanyrulesthatusegradesand/ortestsascriteria.However,thisisdueto
distributional differences in those variables as opposed to differences in the
actual predictive validity of the measurements.
These results have led NCAA committees to attempt to craft rules that use
tests and grades in conjunction with each other and to weight grades equally or
slightly higher than test scores. In addition, the impacts on different subgroups are
always studied and efforts are made to minimize adverse impacts without harming
the integrity of academic policy.
A second result that has been consistent over time is that use of a single cut-
score on standardized tests is not advisable. While this continues to be a very con-
troversial topic and passionate arguments continue to be made for the reimposition
of a standardized test cut-score in NCAA initial eligibility legislation, we would
argue that this is one issue on which the national data are quite clear. There is a
consistent set of factors that have led NCAA policy makers to avoid the use of such
a cut-score in NCAA academic regulations over the last decade. These include:
• Psychometricexperts atthe testingcompanies arein agreementthat using
test scores as a single factor in high-stakes decisions is an inappropriate use
of standardized tests. The testing agencies have been on public record against
using a single cut-score on the tests since Proposition 48 was adopted in 1983.
• AsusedinProposition48andProposition16,thetestcut-scoreledtotests
being overweighted by a 2-to-1 margin as compared with high school GPA.
As stated above, this weighting runs counter to our prediction models, which
indicate grades should be weighted equal to or higher than tests.
• Regulationsthatoverweightthetestscoretendtodecreaseoverallaccuracyof
our predictions and lead to increases in adverse impact within certain demo-
graphic subgroups.
• Useofasinglecut-scorecanleadtodifferentialinitialeligibilitydecisionsfor
student-athletes with the same predicted chance of success.
• Thereisevidencethatuseofacut-scoresimplychangesstudenttestingbehav-
ior, but does not necessarily make for better-prepared student-athletes.
We would like to turn now to a bit of the evidentiary data that underlie some
of the conclusions mentioned above. To begin with, we provide some informa-
tion on the actual academic outcomes of student-athletes with various incoming
academic proles.
In Figure 1, we have presented data on freshman-year academic outcomes
of various groups of student-athletes based on incoming high school grades and
test scores. The y axis represents core-curriculum grades in high school, and the x
axis represents standardized test scores (in SAT units). The diagonal line that runs
through this chart indicates the sliding scale that is currently in use as the mini-
mum for initial eligibility for Division I student-athletes in conjunction with a 2.0
high-school-core GPA (HSCGPA) minimum. Those student-athletes represented
by the Low-Test highlighted area in the upper left of this gure are student-athletes
who have fairly high GPAs, but relatively low test scores. The student-athletes in
this group were declared ineligible under both Propositions 48 and 16 because of
the 820 test score-cut, but have become eligible under the sliding scale that was
implemented in 2003. We compare those in this group to those in the Low-HSCGPA

Frequently asked questions

Q1. What are the contributions in "Academic research and reform: a history of the empirical basis for ncaa academic policy" ?

The purpose of this article is to provide an historical overview of the National Collegiate Athletic Association ’ s ( NCAA ) academic reform, with a particular focus on the empirical basis for the decisions made. The authors outline four eras of academic reform, examine the types of information the NCAA has collected and used to make decisions about academic policy, and explore the limits of such academic data. 

Q2. What is the final set of findings the authors will highlight from this era of academic research?

The final set of findings the authors will highlight from this era of academic research has to do with the overall academic performance of student-athletes and observed impacts of NCAA academic legislation. 

Q3. How many students graduated from college within 6 years of enrollment?

The SCORE data indicated that just over 60% of them graduated from their initial college within 6 years of enrollment, almost identical to federal graduation rates for the same cohorts.