Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools

TLDR: In this article, the authors measure the effect of Catholic high school attendance on educational attainment and test scores, and find that Catholic high schools substantially increase the probability of graduating from high school and, more tentatively, attending college.

Abstract: In this paper we measure the effect of Catholic high school attendance on educational attainment and test scores. Because we do not have a good instrumental variable for Catholic school attendance, we develop new estimation methods based on the idea that the amount of selection on the observed explanatory variables in a model provides a guide to the amount of selection on the unobservables. We also propose an informal way to assess selectivity bias based on measuring the ratio of selection on unobservables to selection on observables that would be required if one is to attribute the entire effect of Catholic school attendance to selection bias. We use our methods to estimate the effect of attending a Catholic high school on a variety of outcomes. Our main conclusion is that Catholic high schools substantially increase the probability of graduating from high school and, more tentatively, attending college. We find little evidence of an effect on test scores.

Figures

Table 10

Table 4 OLS, and Fixed Effect Estimates of Catholic High School Effects7>8

Table 5 OLS, Fixed Effect, and Probit Estimates of Catholic High School Effects by Race and Urban Residence. Full Set of Controls4>6

Table 9 The Amount of Selection on Unobservables Relative to Selection on Observables Required to Attribute the Entire Catholic School Effect to Selection Bias8 (Huber-White Standard Errors in Parentheses)

Table 8

Table 1 Comparison of Means of Key Variables by Sector

Full text

$(''



  !
!!"#!
$!%&'"!)*+,
--...#!!&-"!-.)*+,
/
,010/""2343
"5#! &6/07,+*
3&37000

 !"# !$%!&
'())%*+,,,-
.&%&& /"0'/#0'(
&%&%1#
#%'/1*)2-'0&
*)2!-03 '((
'. 
4+,,,!$)2& !)%/
(%!5(%4
%

2#!4 " #!4 "!"#&
88248"22
6  !6!!"#!
$!%&'"!)*+,
3&37000
,67

$ 45"5 "3"5382#!4"#"
5 ""&3 "53823#!4"#$."8#!4
4"!"#"!"!" 53#8""!&35#!88"2!"832 &34"!"#
" 3258!6!"#. 98#!4"#" !5
  &3 4"!"# "    9 " !5  325 . #  "5 " 
!" #.   2 8 3#!4"# " !5  . 4"!"#  5
2!235"28"2 5":# 3   8:  882 8   &34"!"#$"
!"8!5".":"24:#"#" 5"3!&!"82
3#!4"#2#!4"#".3 #!;3! 8"!#3!
8828 &34"!"#2#"$33!5 5"8828
" &""2&2"4"!:83253!5"223""2&
23#"":2!"!#"#:8&!" 3"&8!5&2" 65!"4:6
2&" "2$ 8 5324 28!"8822!

"!58252
!.!4!:
4"6<070*
" 
"=!.! 3
  !
"!58252
!.!4!:
4"6<070*
 !=!.! 3
!!"#!
"!58252
!.!4!:
4"6<070*
2"#!=!.! 3

4 Lq wurgxfwlrq
Glvwlqjxlvklqj ehwz hhq fruuhodwlrq dqg fdxvdolw| lv wkh prvw gl!fxow fkdoohqjh idfhg e|
hpslulfdo uhvhdufkhuv lq wkh vrfldo vflhqfhv1 Vrfldo vflhqwlvwv uduho| duh lq d srvlwlrq wr gr d
zhoo frqwuroohg h{shulphqw1 Frqvhtxhqwo|/ wkh| uho| rq d sulrul uhvwulfwlrqv rq wkh sdwwhuqv
ri lqwhudfwlrq dprqj wkh yduldeohv wkdw duh revhuyhg ru xqrevhuyhg1 Wkhvh uhvwulfwlrqv
duh w|slfdoo| lq wkh irup ri h{foxvlrq uhvwulfwlrqv/ uhvwulfwlrqv rq wkh ixqfwlrqdo irup ri
wkh prgho/ uhvwulfwlrqv rq wkh glvwulexwlrq ri wkh xqrevhuyhg y duldeohv/ ru uhvwulfwlrqv rq
g|qdplf lqwhudfwlrqv1 Rffdvlrqdoo|/ wkh d sulrul uhvwulfwlrqv duh ghulyhg iurp d zlgho|
dffhswhg wkhru| ru duh vxssruwhg e | rwkhu vwxglhv wkdw kdg dffhvv wr d ulfkhu vhw ri gdwd1
Exw lq prvw fdvhv/ grxew uhpdlqv derxw wkh y dolglw| ri wkh lghqwli|lqj dvvxpswlrqv dqg wkh
lqihuhqfhv wkdw duh edvhg rq wkhp1
Uhvhdufk rq zkhwkhu Fdwkrolf vfkrrov surylgh ehwwhu hgxfdwlrq wkdq sxeolf vfkrrov sur0
ylghv d jrrg looxvwudwlrq ri wkh gl!fxowlhv1 Wklv txhvwlrq lv dw wkh fhqwhu ri wkh ghedwh lq
wkh Xqlwhg Vwdwhv ryhu zkhwkhu yrxfkhuv/ fkduwhu vfkrrov/ dqg rwkhu uhirupv wkdw lqfuhdvh
fkrlfh lq hgxfdwlrq zloo lpsury h wkh txdolw| ri hgxfdwlrq1 Lw lv dovr kljko| uhohy dq w wr
wkh vhdufk iru z d |v wr lpsur y h whdfklqj dqg jr yhuqdqfh ri sxeolf vfkrrov1 Vlpsoh furvv
wdexodwlrqv ru p xowly duldwh uhjuhvvlrqv ri rxwfrphv vxfk dv whvw vfruhv dqg srvw vhfrqgdu|
hgxfdwlrqdo dwwdlqphq w w |slfdoo| vkrz d vxevwdqwldo srvlwlyh hhfw ri Fdwkrolf vfkrro dwwhq0
gdqfh1
4
Krzhyhu/ pdq| surplqhqw vrfldo vflhq wlvwv/ vxfk dv Jrogehujhu dqg Fdlq +4<;5,/
kdyh dujxhg wkdw wkh srvlwlyh hhfwv ri Fdwkrolf vfkrro dwwhqgdqfh pd| eh gxh wr vsxul0
rxv fruuhodwlrqv ehwz hhq Fdwkrolf vf krro dwwhqgdqfh dqg xqrevhuyhg vwxghqw dqg idplo|
fkdudfwhulvwlfv1 Wkh dujxphqw ehjlqv zlwk wkh revhuy dwlrq wkdw lw frvwv sduhqwv wlph dqg
prqh| wr vhqg wkhlu f kloguhq wr sulydwh vfkrro1 Lq wkh devhqfh ri h{shulphqwdo gdwd/ wkh
fkdoohqjh lq dgguhvvlqj wklv srwhqwldoo| odujh eldv lv qglqj h{rjhqrxv yduldwlrq wkdw dhfwv
vfkrro fkrlfh exw qrw rxwfrphv1 Prvw vwxghqw edfnjurxqg fkdudfwhulvwlfv wkdw lq xhqfh
wkh Fdwkrolf vfkrro ghflvlrq/ vxfk dv lqfrph/ dwwlwxghv/ dqg hgxfdwlrq ri wkh sduhqwv/ duh
olnho| wr lq xhqfh rxwfrphv lqghshqghqwo| ri wkh vfkrro vhfwru ehfdxvh wkh| duh olnho| wr eh
uhodwhg wr rwkhu sduhqwdo lqsxwv1 Fkdudfwhulvwlfv ri sulydwh dqg sxeolf vfkrro v wk dw lq x hq f h
fkrlfh/ vxfk dv wxlwlrq ohyhov/ vwxghqw erg| fkdudfwhulvwlfv/ ru vfkrro srolflhv/ duh olnho| wr
4
Wkh prvw lq xhqwldo h{dpsohv duh Frohpdq/ Krhu/ dqg Nlojruh +4<;5, dqg Frohpdq dqg Krhu +4<;:,1
Rwkhu hduo| h{dpsoh h{dpsohv ri vwxglhv ri Fdwkrolf vfkr rov dqg rw khu sulydwh vfkr rov duh Qrhoo +4<;5,/
Jrogehujhu dqg Fdlq +4<;5,/ dqg Doh{dqghu dqg Sdoodv +4<;8,1 Uhfhqw vwxglhv lqfoxgh Hydqv dqg Vfkzde
+4<<8,/ W|ohu +4<<7,/ Qhdo +4<<:,/ Ju rjjhu dqg Qhdo +4<<<,/ Iljolr dqg Vwrqh +5333,/ Vdqghu +5333, dqg
Mhsvhq +5333,1 Pxuqdqh +4<;7,/ Zlwwh +4<<5,/ Fkxee dqg Pr h +4<<3,/ Frrnvrq +4<<6, dqg Qhdo +4<<;,
surylgh ryhuylhzv ri wkh glvfxvvlrq dqg uhihuhqfhv wr wkh olwhu dwxuh1 Jurjjhu dqg Qhdo surylg h flwdwlrq v wr
d vpdoo h{s hulphqwdo olwhudwxuh/ zklfk iru wk h prvw s duw k dv irxqg s rvlwlyh hhfwv ri Fdwkrolf vfkrro1
4

eh uhodwhg wr wkh hhfwlyhqhvv ri wkh vfkrrov1
Vhyhudo uhfhqw vwxglhv/ lqfoxglqj Hydqv dqg Vfkzde +4<<8,/ Qhdo +4<<:,/ Jurjjhu dqg
Qhdo +4<<<,/ Iljolr dqg Vwrqh +4<<<, dqg Dowrqml/ Hoghu dqg Wdehu +4<<<, xvh ydulrxv h{0
foxvlrq uhvwulfwlrqv wr hvwlpdwh wkh Fdwkrolf vf krro hhfw rq d y dulhw| ri rxwfrphv1 Hy dqv
dqg Vfkzde +4<<8, xvh uholjlrxv d!oldwlrq dv dq h{rjhqrxv vrxufh ri y duldwlrq lq Fdwkrolf
vfkrro dwwhqgdqfh dqg frqup wkh odujh srvlwlyh hvwlpdwhv ri Fdwkrolf vfkrro hhfwv rq
kljk vfkrro judgxdwlrq dqg froohjh dwwhqgdqfh wkdw wkh| rewdlq zkhq Fdwkrolf vfkrro dw0
whqgdqfh lv wuhdwhg dv h{rjhqrxv1 Krzhyhu/ dv Hydqv dqg Vfkzde uhfrjql}h dqg Pxuqdqh
hw do +4<;8, dqg Qhdo +4<<:, qrwh/ ehlqj Fdwkrolf frxog z hoo eh fruuhodwhg zlwk fkdudfwhu0
lvwlfv ri wkh qhljkerukrrg dqg idplo| wkdw lq xhqfh wkh hhfwlyhqhvv ri vfkrrov1 Dqrwkhu
lq xhqwldo sdshu e| Q hdo +4<<:, xvhv sur{lhv iru jhrjudsklf sur{lplw| wr Fdwkrolf vfkrrov
dv dq h{rjhqrxv vrxufh ri y duldwlrq lq Fdwkrolf kljk vfkrro dwwhqgdqfh1 Wkh edvlf dvvxps0
wlrq lv wkdw wkh orfdwlrq ri Fdwkrolfv dqg2ru Fdwkrolf vf krrov zdv ghwhuplqhg e | klvwrulfdo
flufxpvwdqfhv dqg lv lqghshqghqw ri xqrevhuydeohv wkdw lq xhqfh shuirupdqfh lq vf krrov1
Kh qgv hylghqfh ri d srvlwlyh hhfw ri Fdwkrolf kljk vfkrro dwwhqgdqfh rq kljk vfkrro dqg
froohjh judgxdwlrq dprqj vwxghqwv lq xuedq duhdv/ sduwlfxoduo| lq wkh fdvh ri qrq zklwhv1
Lq Dowrqml/ Hoghu dqg Wdehu +4<<<,/ z h hpsor| d vlplodu phwkrgrorj| xvlqj gdwd rq }ls
frgh ri uhvlghqfh dqg wkh }ls frghv ri doo ri wkh Fdwkrolf kljk vfkrrov lq wkh frxqwu| wr
frpsxwh d phdvxuh ri glvwdqfh iurp wkh qhduhvw Fdwkrolf kljk vfkrro iru rxu vdpsohv1 Zh
frqfoxgh wkdw wkh xvh ri orfdwlrq ru orfdwlrq lqwhudfwhg zlwk uholjlrq lv qrw d jrrg zd| wr
hvwlpdwh Fdwkrolf vfkrro hhfwv1
5
Jurjjhu dqg Qhdo +5333, frph wr d vlplodu frqfoxvlrq1
6
Dowrqml/ Hoghu/ dqg Wdehu +4<<<, dovr qg wkdw Fdwkrolf uholjlrq kdv d vwurqj dvvrfldwlrq
zlwk judgxdwlrq udwhv iru vwxghqwv zkr dwwhqghg sxeolf hljk wk judghv hyhq wkrxjk vxf k
vwxghqwv uduho| dwwhqg Fdwkrolf kljk vfkrro1 Wklv hylghqfh dqg uhfhqw zrun e| Oxgzlj
+4<<:, udlvhv vhulrxv grxewv derxw wkh ydolglw| ri Fdwkrolf uholjlrq dv dq lqvwuxphqw1
5
Zh surylgh hylghqfh edvhg rq olqnv wr revhuyhg yduldeohv dqg wr hljkwk judgh whvw vfruhv wkdw vxjjhvwv
wkdw qhlwkhu glvwdqfh iurp Fdwkrolf kljk vfkrrov qru wkh lqwhudfwlrq ehwzhhq glvwdqfh dqg uholjlrq vkrxog
eh h{foxghg iurp wkh rxwfrph htxdwlrqv xqohvv ghwdlohg frqwurov iru orfdwlrq duh lqfoxghg1 +Wklv lqirupdo
xvh ri revhuydeohv dv d jxlgh wr fruuhodwlrq ehwzhhq wkh lqvwuxphqw dqg wkh xqrevhuydeohv ohdg wr wkh
fxuuhqw sdshu1, Idloxuh wr frqwuro iru wkhvh idfwruv ohdgv wr qhjdwlyh eldvhv lq hvwlpdwhv ri Fdwkrolf vfkrro
hhfwv1 Xqiruwxqdwho|/ lqfoxglqj ghwdlohg jhrjudsklf frqwurov +vxfk dv 6 gljlw }ls frgh, ohdgv wr yhu| odujh
vwdqgdug huuruv1 Zh dovr iroorz Qhdo +4<<:, dqg Hydqv dqg Vfkzde +4<<8, e| xvlqj elyduldwh surelw prghov
wr mrlqwo| hvwlpdwh wkh Fdwkrolf Vfkrro ghflvlrq zlwk wkh rxwfrphv1 Zh qg wkdw hpslulfdo lghqwlfdwlrq
frphv odujho| iurp wkh ixqfwlrqdo irup ri wkh prgho udwkhu wkdq h{foxvlrq ri wkh phdvxuh ri glvwdqfh iurp
Fdwkrolf vfkrrov1 Qrqolqhdulwlhv lq wkh hhfwv ri vwxghqw edfnjurxqg udwkhu wkdq sur{lhv iru glvwdqfh iurp
Fdwkrolf vfkrrov vhhp wr eh wkh pdlq vrxufh ri lghqwlfdwlrq1
6
Jurjjhu dqg Qhdo +5333, xvh QHOV=;;/ wkh gdwd vhw iru wkh suhvhqw vwxg|1 Dowrqml/ Hoghu dqg Wdehu
+4<<<, dqdo|}h QHOV=;; dqg wkh Qdwlrqdo Orqjlwxglqdo Vxuyh| ri wkh Kljk Vfkrro Fodvv ri 4<:51 Qhdo
+4<<:, xvhv wkh Qdwlrqdo Orqjlwxglqdo Vxuyh| ri \rxwk/ 4<:<1
5

Lq wklv sdshu zh pdnh w z r frqwulexwlrqv1 Wkh uvw lv wr ghyhors hvwlpdwlrq vwudwhjlhv
wkdw pd| eh xvhg zkhq vwurqj sulru lqirupdwlrq lv xqdydlodeoh uhjduglqj wkh h{rjhqhlw|
ri hlwkhu wkh yduldeoh ri lqwhuhvw ru lqvwuxphqwv iru wkdw yduldeoh1 Zh ylhz wklv wr eh wkh
vlwxdwlrq lq vwxglhv ri Fdwkrolf vfkrro hhfwv dqg lq wkh ydvw pdmrulw| ri dssolfdwlrqv lq
hfrqrplfv dqg wkh rwkhu vrfldo vflhqfhv1 Wkh vhfrqg lv wr xvh rxu vwudwhj| wr dvvhvv wkh
hhfwlyhqhvv ri Fdwkrolf vfkrrov1
Rxu dssurdfk wr hvwlpdwlrq lv edvhg rq wkh lghd ri xvlqj wkh ghjuhh ri vhohfwlrq rq
revhuydeohv dv d jxlgh wr krz pxfk vhohfwlrq wkhuh lv rq wkh xqrevhuy deohv1 Lw lv urxwlqh
iru uhvhdufkhuv wr lqirupdoo| dujxh iru wkh h{rjhqhlw| ri phpehuvkls lq d wuhdwphq w jurxs
ru ri dq lqvwuxphqwdo y duldeoh e | h{dplqlqj wkh uhodwlrqvkls ehwzhhq jurxs php ehuvkls ru
wkh lqvwuxphqwdo y duldeoh dqg d vhw ri revhuyhg fkdudfwhulvwlfv/ ru e | dvvhvvlqj zkhwkhu srlqw
hvwlpdwhv duh vhqvlwlyh wr wkh lqfoxvlrq ri dgglwlrqdo frqwuro yduldeohv1
7
Zh surylgh d irupdo
dqdo|vlv frquplqj wkh lq wxlwlrq wkdw vxfk hylghqfh fdq eh lqirupdwlyh lq vrph vhwwlqjv1
Pruh lpsruwdq wo|/ zh surylgh d z d | wr txdqwlwdwlyho| dvvhvv wkh ghjuhh ri hvwlpdwlrq eldv
wkdw lv olnho| zkhq d sduwlfxodu vrxufh ri yduldwlrq lv xvhg wr lghqwli| d p rgho1
8
Qrw vxusulvlqjo|/ wkh hpslulfdo uhohy dqfh ri xvlqj wkh olqn ehwz hhq wkh hqgrjhqrxv
y duldeoh dqg wkh revhuy deohv dv d jxlgh wr wkh olqn zlwk wkh xqrevhuy deohv klqjhv rq krz
wkh revhuy deohv dqg xqrevhuy deohv duh uhodwhg1 Zh pdnh wkh wkhruhwlfdo srlqw wkdw sulru
lqirupdwlrq derxw krz wkh revhuy deohv duh fkrvhq iurp wkh ixoo vhw ri idfwruv wkdw ghwhuplqh
wkh rxwfrph fdq eh xvhg wr lghqwli| dq hfrqrphwulf prgho1 Vshflfdoo|/ z h frqvlghu wkh
fdvh lq zklfk wkh revhuyhg y duldeohv duh d udqgrp vxevhw ri wkh idfwruv wkdw lq xhqfh
wkh hqgrjhqrxv y duldeoh dqg wkh rxwfrph1 Xvlqj rxu Fdwkrolf vfkrrov dssolfdwlrq/ ohw wkh
rxwfrph
t
eh ghwhuplqhg e |
t
'
kM
n
`

K
c
+414,
zkhuh
M
lv dq lqglfdwru iru zkhwkhu wkh vwxghqw dwwhqgv d Fdwkrolf kljk vfkrro/
`
lv d
y hfwr u frqwdlq l qj wkh ixoo vhw ri r wk hu ydu ld e oh v wkd w lq x hq fh
t
/dqg
k
dqg
K
duh frh!flhqwv1
Ohw
f
uhsuhvhq w wkh revhuyhg frpsrqhqwv ri
`
dqg

eh wkh dvvrfldwhg frpsrqhqwv ri
K
1
Wkhq rqh pd | uhzulwh wkh deryh htxdwlrq dv
7
Vhh iru h{dpsoh/ Fxuulh dqg Gxqfdq +4<<8,/ Hqjhq hw do +4<<9,/ Srwhued hw do +4<<7,/ Dqjulvw dqg
Hydqv +4<<;,/ Mdfrevhq hw do1 +4<<<,/ Eurqduv dqg Jurjjhu +4<<7,/ dqg Xgu| +4<<;,1
8
Wzr suhfxuvruv wr rxu vwxg| duh Dowrqml*v +4<;;, vwxg| ri wkh lpsruwdqfh ri revhuyhg dqg xqrevhuyhg
idplo| edfnjurxqg dqg vfkrro fkdudfwhulvwlfv lq wkh vfkrro vshflf yduldqfh ri hgxfdwlrqdo rxwfrphv dqg
hvshfldoo| Pxusk| dqg Wrsho*v +4<<3, vwxg| ri wkh lpsruwdqfh ri vhohfwlrq rq xqrevhuyhg delolw| dv dq
h{sodqdwlrq iru lqgxvwu| zdjh glhuhqwldov1
6

Frequently asked questions

Q1. ' kW czkhuh rgcK q c ?

E0W ' fVroylqj wkhvh htxdwlrqv iru }W |lhogv}WEf ' }Ef n REf Ek kW czkhuh REf lv wkh surshqvlw| vfruh dqg wkxv0W ' Ek kW E M REf n 0 

Q2. ' r k c s g g[ '?

Wkh wkhruhp vwdwhv wkdw dv g jhwv odujh wkh surmhfwlrq frh!flhqwv rq The authorg Sg ' r ` K dqg The authorg Sg ' E r ` K zloo dssurdfk hdfk rwkhu zlwk suredelolw| rqh1Wkhruhp 4 OhwMWg ' s g g[ ' ` q n gtg k Mg ' s g g[ ' ` K czkhuh ` dqg Eq cK duh lqghshqghqw qrqghjhqhudwh/ vwdwlrqdu|/ hujrglf surfhvvhv wkdw vdwlvi| wkh frqglwlrqv iru Zklwh*v +4<;7, Fhqwudo Olplw Wkhruhp 8148/ .E` ' f 4; iru' c cgc dqg g lv xqfruuhodwhg zlwk `Gh qh g dqg 2g vxfk wkdw frqglwlrqdo rq r c c rgcK c cKgc q c c qgcSurm # tg k Mg m sg g[ ' r ` K c s g g[ 'E r ` K $' g s g g[ ' r ` K n 2g s g g[ ' E r ` K czkhuh r lv dq _ elqdu| udqgrp yduldeoh1