Guidelines for the statistical analysis of a collaborative study of a laboratory method for testing disinfectant product performance.

TLDR: This paper presents statistical techniques suitable for analyzing a collaborative study (multilaboratory study or ring trial) of a laboratory disinfectant product performance test (DPPT) method with emphasis on the assessment of the repeatability, reproducibility, resemblance, and responsiveness of the DPPT method.

Abstract: This paper presents statistical techniques suitable for analyzing a collaborative study (multilaboratory study or ring trial) of a laboratory disinfectant product performance test (DPPT) method. Emphasis is on the assessment of the repeatability, reproducibility, resemblance, and responsiveness of the DPPT method. The suggested statistical techniques are easily modified for application to a single laboratory study. The presentation includes descriptions of the plots and tables that should be constructed during initial examination of the data, including a discussion of outliers and QA checks. The statistical recommendations deal with evaluations of prevailing types of DPPTs, including both quantitative and semiquantitative tests. The presentation emphasizes tests in which the disinfectant treatment is applied to surface-associated microbes and the outcome is a viable cell count; however, the statistical guidelines are appropriate for suspension tests and other test systems. The recommendations also are suitable for disinfectant tests using any microbe (vegetative bacteria, virus, spores, etc.) or any disinfectant treatment. The descriptions of the statistical techniques include either examples of calculations based on published data or citations to published calculations. Computer code is provided in an appendix.

Full text

Guidelines for the statistical analysis of a
collaborative study of a laboratory method
for testing disinfectant product performance
Authors: Martin A. Hamilton, Gordon C. Hamilton,
Darla M. Goeres, & Albert E. Parker
NOTICE: This is a postprint of an article that originally appeared in Journal of AOAC International
on September 2013. DOI: http://dx.doi.org/10.5740/jaoacint.12-217.
Hamilton MA, Hamilton GC, Goeres DM, Parker AE, "Guidelines for the statistical analysis of a
collaborative study of a laboratory method for testing disinfectant product performance," J AOAC
International. 2013 96(5):1138-1147
Made available through Montana State University’s ScholarWorks
scholarworks.montana.edu

*XLGHOLQHV௑IRU௑WKH௑6WDWLVWLFDO௑$QDO\VLV௑RI௑D௑&ROODERUDWLYH௑
6WXG\௑RI௑D௑/DERUDWRU\௑0HWKRG௑IRU௑7HVWLQJ௑'LVLQIHFWDQW௑
3URGXFW௑3HUIRUPDQFH
0DUWLQ$+DPLOWRQ௑DQG௑*RUGRQ&RUG+DPLOWRQ
%LJ6N\6WDWLVWLFDO$QDO\VWV//&6RXWK6L[WK$YH%R]HPDQ07
'DUOD0*RHUHV
௑
DQG௑$OEHUW(3DUNHU
0RQWDQD6WDWH8QLYHUVLW\&HQWHUIRU%LRILOP(QJLQHHULQJ%R]HPDQ07
7KLVSDSHUSUHVHQWVVWDWLVWLFDOWHFKQLTXHVVXLWDEOHIRUDQDO\]LQJDFROODERUDWLYHVWXG\PXOWLODERUDWRU\VWXG\RUULQJWULDORIDODERUDWRU\
GLVLQIHFWDQWSURGXFWSHUIRUPDQFHWHVW'337PHWKRG(PSKDVLVLVRQWKHDVVHVVPHQWRIWKHUHSHDWDELOLW\UHSURGXFLELOLW\UHVHPEODQFH
DQGUHVSRQVLYHQHVVRIWKH'337PHWKRG7KHVXJJHVWHGVWDWLVWLFDOWHFKQLTXHVDUHHDVLO\PRGLILHGIRUDSSOLFDWLRQWRDVLQJOHODERUDWRU\
VWXG\7KHSUHVHQWDWLRQLQFOXGHVGHVFULSWLRQVRIWKHSORWVDQGWDEOHVWKDWVKRXOGEHFRQVWUXFWHGGXULQJLQLWLDOH[DPLQDWLRQRIWKHGDWD
LQFOXGLQJDGLVFXVVLRQRIRXWOLHUVDQG4$FKHFNV7KHVWDWLVWLFDOUHFRPPHQGDWLRQVGHDOZLWKHYDOXDWLRQVRISUHYDLOLQJW\SHVRI'337V
LQFOXGLQJERWKTXDQWLWDWLYHDQGVHPLTXDQWLWDWLYHWHVWV7KHSUHVHQWDWLRQHPSKDVL]HVWHVWVLQZKLFKWKHGLVLQIHFWDQWWUHDWPHQWLVDSSOLHGWR
VXUIDFHDVVRFLDWHGPLFUREHVDQGWKHRXWFRPHLVDYLDEOHFHOOFRXQWKRZHYHUWKHVWDWLVWLFDOJXLGHOLQHVDUHDSSURSULDWHIRUVXVSHQVLRQWHVWV
DQGRWKHUWHVWV\VWHPV7KHUHFRPPHQGDWLRQVDOVRDUHVXLWDEOHIRUGLVLQIHFWDQWWHVWVXVLQJDQ\PLFUREHYHJHWDWLYHEDFWHULDYLUXVVSRUHV
HWFRUDQ\GLVLQIHFWDQWWUHDWPHQW7KHGHVFULSWLRQVRIWKHVWDWLVWLFDOWHFKQLTXHVLQFOXGHHLWKHUH[DPSOHVRIFDOFXODWLRQVEDVHGRQSXEOLVKHG
GDWDRUFLWDWLRQVWRSXEOLVKHGFDOFXODWLRQV&RPSXWHUFRGHLVSURYLGHGLQDQDSSHQGL[
,W LV LPSHUDWLYH WKDW DSSURSULDWH VWDWLVWLFDO SURFHGXUHV DUHXVHGWRUHYHDOVXPPDUL]HDQGFRPPXQLFDWHWKHUHVXOWV
RID PXOWLODERUDWRU\ FROODERUDWLYH VWXG\ ULQJ WULDO 9DULRXVRIILFLDOJXLGHOLQHVDUHDYDLODEOHWRVWHHUWKHVWDWLVWLFDO
DQDO\VLVRIFROODERUDWLYHVWXGLHVRIFKHPLFDODVVD\PHWKRGVk¼³*XLGHOLQHVIRUDQDO\]LQJVWXGLHVRIPHWKRGVIRU
DVFHUWDLQLQJPLFURELDOFRQWDPLQDWLRQRIIRRGVOLVWVWDWLVWLFDOWHFKQLTXHVWKDWDUHYHU\VLPLODUWRWKHRIILFLDOJXLGDQFH
IRUFKHPLFDODVVD\V/LPLWHGJXLGDQFHH[LVWVKRZHYHUIRUWKHVWXG\RIODERUDWRU\WHVWPHWKRGVGHVLJQHGWR
HYDOXDWHWKHDWWULEXWHVRIDGLVLQIHFWDQWSURGXFWSHUIRUPDQFHWHVW'337PHWKRG)RUVWXGLHVRIFKHPLFDODVVD\VDQG
PLFURELDO FRQWDPLQDWLRQ RI IRRG DVVD\V WKH VWDWLVWLFDO DQDO\VLV JXLGDQFH GRFXPHQWV XVH JHQHULF WHUPLQRORJ\ DQG
FRQIRUP WR HVWDEOLVKHG VFLHQWLILF SUDFWLFH FRQVHTXHQWO\ WKH\ DUH ZLGHO\ DSSOLHG IRU PHWKRG HYDOXDWLRQV LQ RWKHU
ILHOGV$OWKRXJKWKH\SURYLGHVRPHLQIRUPDWLRQUHOHYDQWWR'337PHWKRGVWXGLHVWKH\DUHLQDGHTXDWHIRUDQDO\]LQJ
GDWDIURPD'337PHWKRGFROODERUDWLYHVWXG\EHFDXVH'337PHWKRGVGLIIHULQPDQ\LPSRUWDQWZD\VIURPFKHPLFDO
DVVD\V DQG PLFURELDO FRQWDPLQDWLRQ RI IRRG DVVD\V ± $ '337 LV D PLFURELRORJLFDO DVVD\ WKDW DVVHVVHV WKH
DELOLW\ RI D GLVLQIHFWDQW SURGXFW WR NLOO RU RWKHUZLVH LQDFWLYDWH OLYLQJ PLFUREHV 7KH '337V HPSOR\ GLIIHUHQW
H[SHULPHQWDO GHVLJQV XVH GLIIHUHQW WHUPLQRORJ\ SURGXFH IXQGDPHQWDOO\ GLIIHUHQW UHVSRQVH PHDVXUHPHQWV KDYH
DGGLWLRQDO JRDOV DQG DSSO\WKH UHVXOWV WR GLIIHUHQW XVHV WKDQ FKHPLFDO DVVD\V RU PLFURELDO FRQWDPLQDWLRQ RI IRRG
DVVD\V6RPHWRSLFVLPSRUWDQWWR'337VDUHQRWFRYHUHGLQH[LVWLQJJXLGDQFHGRFXPHQWVEHFDXVHWKRVHWRSLFVDUH
QRW UHOHYDQW WR FKHPLFDO DVVD\V RU PLFURELDO FRQWDPLQDWLRQ RI IRRG DVVD\V 2Q WKH RWKHU KDQG WKHUH LV PXFK
LQIRUPDWLRQLQWKHJXLGHOLQHVIRUFKHPLFDODVVD\VDQGPLFURELDOFRQWDPLQDWLRQRIIRRGDVVD\VWKDWGRHVQRWSHUWDLQWR
'337V
5HJXODWRU\DXWKRULWLHVDQGFRQVXPHUVUHO\RQODERUDWRU\WHVWVWRLQGLFDWHWKHSHUIRUPDQFHRIGLVLQIHFWDQWSURGXFWV
ZKHQDSSOLHGLQSUDFWLFH%HFDXVHGLVLQIHFWLRQLVDFULWLFDOFRPSRQHQWRIGLVHDVHSUHYHQWLRQVWUDWHJLHVWKHODERUDWRU\
'337 PHWKRG PXVW EH ULJRURXVO\ HYDOXDWHG VR WKDW RQH FDQ EH FRQILGHQW LQ WKH WHVW UHVXOWV IRU DQ\ VSHFLILF
GLVLQIHFWDQWSURGXFW)RUDOOWKHVHUHDVRQVWKHUHLVDQHHGWRFRQVROLGDWHDQGSUHVHQWVWDWLVWLFDOWHFKQLTXHVWKDWDUH
VXLWHGVSHFLILFDOO\WRFROODERUDWLYHVWXG\HYDOXDWLRQVRI'337PHWKRGV
7KH SXUSRVH RI WKLV SDSHU LV WR GHVFULEH VWDWLVWLFDO WHFKQLTXHV IRU DQDO\]LQJ GDWD IURP D FROODERUDWLYH VWXG\ RI D
VWDQGDUGL]HG '337 PHWKRG 7KH SUHVHQWDWLRQ OLHV DW WKH LQWHUVHFWLRQ RI PLFURELRORJLFDO ODERUDWRU\ PHWKRGV DQG
VWDWLVWLFDO PHWKRGV ,W RUJDQL]HV WKH WHUPLQRORJ\ JRDOV DQG VWDWLVWLFDO WHFKQLTXHV LQWR D IUDPHZRUN WKDW VKRXOG
IDFLOLWDWH FRPPXQLFDWLRQ EHWZHHQ WKH PLFURELRORJLVW DQG VWDWLVWLFLDQ ,WV PDLQ JRDO LV WR KHOS WKH VWDWLVWLFLDQ E\
VXJJHVWLQJSURYHQSUDFWLFDOVWDWLVWLFDOWHFKQLTXHVIRUDQDO\]LQJ'337VWXG\GDWD
7KHWHUPLQRORJ\DQGPDWKHPDWLFDOQRWDWLRQXVHGLQWKLVSDSHUDUHVXPPDUL]HGLQ7DEOHVDQG7KHSUHVHQWDWLRQ
FRYHUVSUHYDLOLQJW\SHVRI'337VLQFOXGLQJERWKTXDQWLWDWLYHDQGVHPLTXDQWLWDWLYHWHVWVDVVXPPDUL]HGLQ7DEOH
 8QWLO D IHZ \HDUV DJR VRPH SRSXODU '337 PHWKRGV GLG QRW UHTXLUH GDWD IRU XQWUHDWHG FRQWURO FDUULHUV
KRZHYHUVXFKPHWKRGVDUHREVROHWHDQGDUHQRWGLVFXVVHGKHUH7KLVSDSHUGHVFULEHVDWKHEDVLFGDWDFROOHFWHGLQ
FROODERUDWLYHVWXGLHV

RI '337V LQFOXGLQJ WKH W\SLFDO VWXG\ IDFWRUV DQG UHVSRQVH
YDULDEOHV E WKH SORWV DQG WDEOHV WKDW KDYH SURYHQ KHOSIXO
GXULQJ WKH LQLWLDO H[DPLQDWLRQ RI FROODERUDWLYH VWXG\ GDWD
LQFOXGLQJ D GLVFXVVLRQ RI RXWOLHU GHWHFWLRQ DQG HYDOXDWLRQ
F UHFRPPHQGHG 4$ FKHFNV DQG G JXLGDQFH RQ VWDWLVWLFDO
WHFKQLTXHVIRUDVVHVVLQJWKHH[WHQWWRZKLFKWKH'337PHWKRG
DWWDLQVWKHLPSRUWDQWDWWULEXWHVRIUHSHDWDELOLW\UHSURGXFLELOLW\
UHVHPEODQFH DQG UHVSRQVLYHQHVV  ,W LV LPSRUWDQW WR
UHPHPEHU WKDW D FROODERUDWLYH VWXG\ LV FRQGXFWHG WR DVVHVV
WKHDWWULEXWHVRIWKH'337PHWKRGQRWWRHYDOXDWHWKHVSHFLIL
GLVLQIHFWDQW WUHDWPHQWV XVHG LQ WKH VWXG\ 1RWDEOH WRSLFV QRW
FRYHUHGKHUHLQFOXGHJXLGHOLQHVIRUWKHGHVLJQDQGPDQDJHPHQW
RIDFROODERUDWLYHVWXG\DQGVHUYLFHDEOHVWUDWHJLHVIRUFUHDWLQJ
DQGRSWLPL]LQJDQHZVWDQGDUGL]HG'337PHWKRG
5HFRPPHQGDWLRQV LQ WKLV DUWLFOH DGDSW DQG H[WHQG WKH
UHSHDWDELOLW\DQGUHSURGXFLELOLW\DVVHVVPHQWPHWKRGVGHVFULEHG
HDUOLHUPDNLQJWKHVHUHFRPPHQGDWLRQVFRQVLVWHQWZLWK
H[LVWLQJJXLGHOLQHVIRUFROODERUDWLYHVWXGLHVRIFKHPLFDODVVD\V
RU PLFURELDO FRQWDPLQDWLRQ RI IRRG DVVD\V )RU HDFK PDMRU
DQDO\VLVVWHSWKHVHJXLGHOLQHVFLWHDSXEOLVKHGGDWDDQDO\VLVDQG
LQDQDSSHQGL[VKRZKRZWRSHUIRUPWKHNH\FDOFXODWLRQVXVLQJ
DSRSXODUVWDWLVWLFDOVRIWZDUHSDFNDJH7KLVSDSHULQFRUSRUDWHV
WHUPLQRORJ\ DQG PHWKRGRORJLHV XVHG LQ UHFHQW UHSRUWV RQ
FROODERUDWLYHVWXGLHVRI'337PHWKRGV±%HFDXVHHDFK
'337 KDV LWV RZQ VSHFLDO FKDUDFWHULVWLFV DQG D FROODERUDWLYH
VWXG\ PD\ LQYROYH DOWHUQDWLYH JRDOV RQH VKRXOG DSSO\ ZLWK
FLUFXPVSHFWLRQWKHVWDWLVWLFDOWHFKQLTXHVSUHVHQWHGLQWKLVSDSHU
'LVLQIHFWDQW3URGXFW3HUIRUPDQFH7HVWV
$ '337 SURWRFRO GHVFULEHV LQ GHWDLO WKH PLFURELRORJLFDO
WHFKQLTXHV XVHG ZKHQ SHUIRUPLQJ WKH WHVW 7KH GLIILFXOW\ RI
WKH WHVW PHWKRG GHSHQGV RQ WKH H[LVWHQW HQYLURQPHQW EHLQJ
HPXODWHG WKH PHDQV E\ ZKLFK D FKHPLFDO GLVLQIHFWDQW LV
DSSOLHGOLTXLGVSUD\ZHWZLSHWRZHOHWWHHWFDQGRQWKHWHVW
PLFURRUJDQLVPVYHJHWDWLYHEDFWHULDVSRUHVYLUXVHVSURWR]RD
HWF 7KH PLFURELRORJLFDO PDQLSXODWLRQV XVXDOO\ UHTXLUH
H[SHULHQFHGPLFURELRORJ\ODERUDWRU\VSHFLDOLVWV
7KH GLVLQIHFWDQW PD\ EH WHVWHG DJDLQVW HLWKHU SODQNWRQLF
PLFUREHV VXVSHQGHG LQ D OLTXLG RU D FROOHFWLRQ RI
VXUIDFHDVVRFLDWHG PLFUREHV 9DULRXV VXVSHQVLRQ WHVW PHWKRGV
KDYH EHHQ HYDOXDWHG WKRURXJKO\ DQG DUH XVHG ZLGHO\ IRU
H[DPSOHWRVFUHHQFKHPLFDOVIRUDQWLPLFURELDODFWLYLW\DQGWR
DVVHVVWKHHIILFDF\ RIGULQNLQJZDWHUGLVLQIHFWDQWVRUUHFUHDWLRQDO
ZDWHUGLVLQIHFWDQWV0HWKRGVIRUWHVWLQJDGLVLQIHFWDQW
DJDLQVW VXUIDFHDVVRFLDWHG PLFUREHV W\SLFDOO\ XVH HDVLO\
PDQLSXODWHGFDUULHUVHJJODVVGLVNV$YDULHW\RIPHWKRGVDUH
XVHGWRSODFHDQGKROGPLFUREHVRQWKHFDUULHUV7KHH[DPSOHV
LQWKLVDUWLFOHIRFXVRQWHVWVXVLQJVXUIDFHDVVRFLDWHGPLFUREHV
EHFDXVH LW LV DQWLFLSDWHG WKDW PRVW QHZ VWDQGDUGL]HG '337V
ZLOO EH VXUIDFH WHVWV +RZHYHU WKH GLVFXVVLRQ FDQ EH DSSOLHG
WRVXVSHQVLRQWHVWVLIWKH VXVSHQVLRQ WHVWEHDNHUVRUWXEHVDUH
FRQVLGHUHGWREHFDUULHUV
,QDVXUIDFH'337VRPHPLFUREHEHDULQJFDUULHUVDUHWUHDWHG
ZLWK WKH GLVLQIHFWDQW DQG RWKHUV VHUYH DV XQWUHDWHG FRQWURO
FDUULHUV )RU WUHDWHG FDUULHUV WKH GLVLQIHFWDQW LV QHXWUDOL]HG
WR VWRS LWV DFWLYLW\ DW WKH HQG RI WKH GHVLJQDWHG FRQWDFW WLPH
8QWUHDWHGFDUULHUVUHFHLYHWKHVDPHPDQLSXODWLRQVDVWKHWUHDWHG
FDUULHUV LQFOXGLQJ QHXWUDOL]DWLRQ H[FHSW WKDW DQ LQDFWLYH
WUHDWPHQW VXFK DV GLOXWLRQ ZDWHU LV DSSOLHG LQVWHDG RI WKH
GLVLQIHFWDQW
$GLVLQIHFWDQWLV IRUPXODWHGWRNLOO PLFUREHVFRQVHTXHQWO\
IRUDQHIIHFWLYHGLVLQIHFWDQWWKHWUHDWHGFDUULHUVVKRXOGKROGIHZ
YLDEOHPLFUREHVUHODWLYHWRWKHXQWUHDWHGFDUULHUV'LVLQIHFWDQW
HIILFDF\ LV TXDQWLILHG E\ FRPSDULQJ WKH W\SLFDO QXPEHU RI
YLDEOH PLFUREHV RQ WKH WUHDWHG FDUULHUV WR WKH W\SLFDO QXPEHU
RQWKHXQWUHDWHGFDUULHUV'HSHQGLQJRQWKHPLFUREHVDQGWHVW
V\VWHPLWPD\EHSRVVLEOHWRSHUIRUPDTXDQWLWDWLYHDVVHVVPHQW
RIWKHQXPEHURIYLDEOHPLFUREHVRQDFDUULHUE\FRQYHQWLRQDO
HQXPHUDWLRQ PHWKRGV HJ KDUYHVWLQJ WKH PLFUREHV IURP WKH
FDUULHU VXUIDFH LQWR VXVSHQVLRQ GLVDJJUHJDWLQJ WKH PLFUREHV
SHUIRUPLQJDGLOXWLRQVHULHVSODWLQJLQFXEDWLQJDQG௑FRXQWLQJ
FRORQLHV
&ROODERUDWLYH6WXGLHV²$V9LHZHGE\WKH6WDWLVWLFDO
$QDO\VW
$FROODERUDWLYHVWXG\RID'337PHWKRGLVDPXOWLODERUDWRU\
DVVHVVPHQW RI WKH PHWKRG FRQGXFWHG E\ UHSOLFDWH WHVWLQJ RI
WKHVDPH³GLVLQIHFWDQWWUHDWPHQW´7DEOHLQHDFKRI VHYHUDO
ODERUDWRULHV7KH EDVLF JRDOV RI D FROODERUDWLYH VWXG\ LQFOXGH
GHWHUPLQLQJWKHH[WHQWWRZKLFKWKHHIILFDF\ RXWFRPHDFKLHYHV
WKH GHVLUDEOH DWWULEXWHV RI UHSHDWDELOLW\ ZLWKLQ D ODERUDWRU\
UHSURGXFLELOLW\ DPRQJ ODERUDWRULHV DQG UHVSRQVLYHQHVV
WR LQFUHDVLQJ GLVLQIHFWDQW HIILFDF  $Q DGGLWLRQDO JRDO LV
GHWHUPLQLQJWKHH[WHQWWRZKLFKWKHXQWUHDWHGFDUULHUVUHVHPEOH
HDFKRWKHUIURPWHVWWRWHVWDQGIURPODERUDWRU\WRODERUDWRU\
ZKHUHUHVHPEODQFHLVZLWKUHVSHFWWRWKHQXPEHURI PLFUREHV
RQHDFKXQWUHDWHGFDUULHU$GGLWLRQDOJRDOVDUHQRWXQXVXDOHJ
WRFRPSDUHDQHZ'337PHWKRGWRDVLPLODUHVWDEOLVKHG'337
PHWKRG
6WDWLVWLFDO$QDO\VW¶V3UHSDUDWLRQ
7KH VWDWLVWLFDO DQDO\VW VKRXOG VWXG\ WKH FROODERUDWLYH VWXG\
SURSRVDO JRDOV DQG SURWRFRO DV ZHOO DV UHOHYDQW EDFNJURXQG
LQIRUPDWLRQDERXWWKH'337PHWKRGHJUHVHDUFKFRQGXFWHG
GXULQJ GHYHORSPHQW RI WKH PHWKRG 7KH 6WXG\ 'LUHFWRU
VKRXOGQRWLI\WKHVWDWLVWLFDODQDO\VWDERXWDQ\ GHYLDWLRQVIURP
WKH SURWRFRO RU WKH VWXG\ GHVLJQ DQG DERXW DQ\ XQDQWLFLSDWHG
RFFXUUHQFHVWKDWSRWHQWLDOO\FRXOGDIIHFWUHVXOWVRUWKHFKRLFHRI
VWDWLVWLFDOWHFKQLTXHV,WLVDGYLVDEOHIRUWKHVWDWLVWLFDODQDO\VWWR
OLVWWKHVWXG\IDFWRUVUHVSRQVHYDULDEOHVDQGDQ\RWKHUYDULDEOHV
RI LPSRUWDQFH WKDW ZHUH UHFRUGHG LQ WKH FROODERUDWLYH VWXG\
1RWHWKDWIDFWRUVDUHDOVRFDOOHGLQGHSHQGHQWYDULDEOHVSUHGLFWRU
YDULDEOHVRUH[SODQDWRU\YDULDEOHVUHVSRQVHYDULDEOHVDUHDOVR
FDOOHGGHSHQGHQWYDULDEOHV*HQHUDOO\WKHRWKHUYDULDEOHV
RILPSRUWDQFHDUHFRYDULDWHVRUFRQFRPLWDQWYDULDEOHVVXFKDV
DPELHQWKXPLGLW\DQGWHPSHUDWXUH7KHOLVWRIIDFWRUVDQG
YDULDEOHVZLOOKHOSWKHDQDO\VWXQGHUVWDQGWKHVFRSHRIWKHVWXG\
DQG FKRRVH WKH VWDWLVWLFDO VXPPDULHV DQG DQDO\WLFDO PRGHOV
VXLWDEOHIRUDWWDLQLQJWKHVWXG\JRDOV
6WXG\)DFWRUV
,QDFRQYHQWLRQDOFROODERUDWLYHVWXG\WKHVWXG\IDFWRUVZLOO
LQFOXGH WHVW GDWH PLFUREH GLVLQIHFWDQW HIILFDF\ OHYHO RI WKH
GLVLQIHFWDQW WHFKQLFLDQ RU WHFKQLFLDQ WHDP DQG ODERUDWRU\
7KH WHVW GDWH LV HLWKHU WKH GD\ WKH WHVW EHJDQ RU WKH GD\ WKH
ILQDO UHVSRQVH ZDV REVHUYHG D FKRLFH DSSOLHG FRQVLVWHQWO\
WKURXJKRXW WKH VWXG\ 7KH UHFRUGHG UHVSRQVH IRU HDFK FDUULHU
ZLOOEHDFFRPSDQLHGE\DXQLTXHFRGHIRUHDFKWHVWDQGDFRGH

WKDWLQGLFDWHVZKHWKHUWKHUHVSRQVHLVIRUDQXQWUHDWHGFDUULHURU
DWUHDWHGFDUULHU,IWKHFROODERUDWLYHVWXG\KDVDQH[SDQGHGOLVW
RIJRDOVWKHGDWDVHWZLOOLQFOXGHDGGLWLRQDOIDFWRUVSHUWDLQLQJWR
WKRVHJRDOV)RUH[DPSOHLQDPHWKRGFRPSDULVRQFROODERUDWLYH
VWXG\WKHGDWDZLOOLQFOXGHDPHWKRGFRGH
5HVSRQVH9DULDEOHV
7KHUHVSRQVHYDULDEOHV DUHGHSHQGHQWRQWKHW\SHRI'337
EHLQJ FRQGXFWHG TXDQWLWDWLYH VHPLTXDQWLWDWLYH RU DOWHUQDWLYH
7DEOH  )RU TXDQWLWDWLYH DQG VHPLTXDQWLWDWLYH VWXGLHV WKH
SULPDU\ UHVSRQVH YDULDEOHV IRU HDFK WHVW DUH WKH XQWUHDWHG
FDUULHUORJ

WUDQVIRUPHGYLDEOHPLFUREHGHQVLWLHVDQGWKHORJ
UHGXFWLRQ /5 PHDVXUH RI HIILFDF\ IRUPXODV JLYHQ EHORZ LQ
VXEVHFWLRQV ³/5 IRU D VLQJOH TXDQWLWDWLYH WHVW´ DQG ³/5 IRU
D VLQJOH VHPLTXDQWLWDWLYH WHVW RI W\SH 64

´ RI WKH 5HVSRQVH
9DULDEOHVVHFWLRQ
0DWKHPDWLFDO IRUPXODV DUH XVHIXO ZKHQ GLVFXVVLQJ WKH
UHVSRQVHYDULDEOHVQRWDWLRQLQ7DEOH:KHQYLDEOHPLFUREHV
FDQEHHQXPHUDWHGE\FRXQWLQJFRORQ\IRUPLQJXQLWV&)8OHW
'GHQRWHWKHYLDEOHPLFUREHGHQVLW\DVVRFLDWHGZLWKDFDUULHU
W\SLFDOO\ H[SUHVVHG LQ XQLWV RI &)8FDUULHU RU &)8FP

 RI
FDUULHUVXUIDFHDUHD7KHSURWRFROIRUWKH'337PHWKRGVKRXOG
VSHFLI\WKHGHQVLW\XQLWVDQGWKHVDPHXQLWVPXVWEHXVHGIRU
ERWKWUHDWHGDQGXQWUHDWHGFDUULHUV
7KH ORJ

WUDQVIRUPHG GHQVLW\ ' LV FDOOHG WKH ORJ GHQVLW\
GHQRWHG E\ /' ([SHULHQFH KDV VKRZQ WKDW QRW RQO\ GR WKH
/' YDOXHV IRU UHSOLFDWH FDUULHUV XVXDOO\ IROORZ D V\PPHWULF
GLVWULEXWLRQ  EXW WKH\ DOVR FRQIRUP WR WKH DVVXPSWLRQV
UHTXLUHGIRUFRQYHQWLRQDOQRUPDOWKHRU\VWDWLVWLFDOWHFKQLTXHV
7KHUHIRUH ZKHQ DQDO\]LQJ '337 GDWD LW LV VWDQGDUG SUDFWLFH
WR SHUIRUP VWDWLVWLFDO FDOFXODWLRQV RQ /' YDOXHV QRW RQ WKH
XQWUDQVIRUPHG GHQVLWLHV   7KH PHDQ RI /' YDOXHV
DYHUDJHG DFURVV D VHW RI FDUULHUV IRUPXOD DQG QRWDWLRQ LQ
WKH QH[W IHZ SDUDJUDSKV LV DQ LPSRUWDQW VXPPDU\ VWDWLVWLF
:KHQ WKH ILQDO UHVXOWV DUH SUHVHQWHG RQ WKH GHQVLW\ VFDOH WKH
JHRPHWULFPHDQGHQVLW\ZKLFKLVWKHDQWLORJRIWKHPHDQ/'
LV WKH SUHIHUUHG LQGLFDWRU RI WKH W\SLFDO GHQVLW\ IRU WKH VHW RI
FDUULHUV
)RUDVLQJOHWHVWOHW-GHQRWHWKHQXPEHURIXQWUHDWHGFDUULHUV
DQG.GHQRWHWKHQXPEHURIWUHDWHGFDUULHUV/HW8
M
GHQRWHWKH
/' IRU WKH M
WK
 XQWUHDWHG FDUULHU M    « - 7KH PHDQ RI
XQWUHDWHGFDUULHU/'VLQWKHWHVWLV
8
-
7HVW/'
-
M
M
¦


7KH JHRPHWULF PHDQ GHQVLW\ IRU WKH WHVW LV 
7HVW/'
 7KH
7HVW/' LV D VWDQGDUG JDXJH RI WKH PLFURELDO FKDOOHQJH ,Q
DGGLWLRQWREHLQJDFULWLFDOFRPSRQHQWRIWKH/57HVW/'LVWKH
NH\UHVSRQVHYDULDEOHIRUDUHVHPEODQFHDQDO\VLVGHWDLOVODWHU
)RU TXDQWLWDWLYH DQG VHPLTXDQWLWDWLYH WHVWV DQ LPSRUWDQW
UHVSRQVHYDULDEOHLV/5WKHFRQYHQWLRQDOPHDVXUHRIHIILFDF\
IRU D GLVLQIHFWDQW WUHDWPHQW 7KH /5 IRU D WHVW LV FDOFXODWHG
GLIIHUHQWO\IRUTXDQWLWDWLYHDQGVHPLTXDQWLWDWLYHPHWKRGV
7KHUHFRPPHQGHGFDOFXODWLRQIRUPXODVZLOOQRZEHSUHVHQWHG
/5 IRU D VLQJOH TXDQWLWDWLYH WHVW²)RU D TXDQWLWDWLYH WHVW
/5 LV IRXQG E\ VXEWUDFWLQJ WKH PHDQ RI ORJ GHQVLWLHV IRU WKH
WUHDWHGFDUULHUVIURPWKHPHDQRIORJGHQVLWLHVIRUWKHXQWUHDWHG
FDUULHUV /HW 7
N
 GHQRWH WKH /' IRU WKHN
WK
 WUHDWHG FDUULHU N 
  «
. WKH PHDQ RI WKH WUHDWHG FDUULHU ORJ GHQVLWLHV LV
¦
.
N
N
7
.
7



(TXDWLRQSUHVHQWVWKHIRUPXODIRUWKHTXDQWLWDWLYH
WHVW/5
77HVW/'

/5

/HW 86 DQG 76 GHQRWH WKH ZLWKLQWHVW 6' RI ORJ GHQVLWLHV
IRU XQWUHDWHG FDUULHUV DQG WUHDWHG FDUULHUV LQ D VLQJOH WHVW
UHVSHFWLYHO\DVGHILQHGLQ(TXDWLRQ




¦

-
7HVW/'8
86
-
M
M

DQG




¦

.
 77 
76
.
N
N

(TXDWLRQ  VKRZV WKH ZLWKLQWHVW 6' RI /5  ZKLFK LV
GHQRWHGE\6
.
76
-
86
6




/5 IRU D VLQJOH VHPLTXDQWLWDWLYH WHVW RI W\SH 64

²)RU D
VHPLTXDQWLWDWLYHWHVWVRPHRUDOORIWKHYLDEOHPLFUREHGHQVLWLHV
DUHEDVHGRQSRVLWLYHRUQHJDWLYH31REVHUYDWLRQV7DEOH
)RUD7\SH64

WHVWZKLFKLVWKHPRVWFRPPRQVHPLTXDQWLWDWLYH
WHVWYLDEOHPLFUREHVDUHHQXPHUDWHGRQXQWUHDWHGFDUULHUVEXWD
31RXWFRPHLVUHFRUGHGIRUHDFKWUHDWHGFDUULHU/HW13
7
GHQRWH
WKHQXPEHURISRVLWLYHWUHDWHGFDUULHUVDPRQJWKHVHWRI.LQWKH
WHVW/HW
13
7
GHQRWHWKHPHDQ/'IRUWUHDWHGFDUULHUVDYDOXH
WKDWFDQEHFDOFXODWHGIURPWKH13
7
RXWFRPHXVLQJWKHIRUPXOD
VKRZQLQ(TXDWLRQZKHUHOQ[LVWKHQDWXUDOORJDULWKPRIWKH
SRVLWLYHQXPEHU[
13
7
@


>O R J

¸
¹
·
¨
©
§



.
1 3 .
OQ
7


7KH
13
7
YDOXHRI(TXDWLRQLVWKHORJ

WUDQVIRUPDWLRQRI
WKHVLQJOHGLOXWLRQPRVWSUREDEOHQXPEHU031DQRIWHQXVHG
PHWKRGIRUFDOFXODWLQJ WKHW\SLFDOGHQVLW\ ZKHUH WKH
FRQYHQWLRQDO031IRUPXODKDVEHHQDGMXVWHGVOLJKWO\E\DGGLQJ
WRWKHQXPHUDWRUDQGWRWKHGHQRPLQDWRUZKHQFDOFXODWLQJ
WKH IUDFWLRQ RI FDUULHUV WKDW ZHUH QHJDWLYH 7KH DGMXVWPHQW
DVVXUHV WKDW
13
7
 FDQ EH FDOFXODWHG HYHQ ZKHQ WKH REVHUYHG
13
7
LVRQHRIWKHH[WUHPHVRU.*DUWKULJKWGHWHUPLQHG
WKDWDVDQHVWLPDWHRIWKHPHDQ/'WKHORJWUDQVIRUPHG031
LVQHJOLJLEO\ELDVHG
(TXDWLRQ  SURYLGHV WKH IRUPXOD IRU /5 IRU D W\SH 64

WHVW
13
77HVW/'

/5

)RUVHPLTXDQWLWDWLYHWHVWVRIW\SHV64

DQG64

WKH/5DOVR
FDQEHFDOFXODWHGIURPORJWUDQVIRUPHG031YDOXHVIRUERWK
WKHWUHDWHGDQGXQWUHDWHGFDUULHUV
4$6WHSV
%HIRUHFRPSXWLQJDQDO\WLFDOUHVXOWVLWLVLPSRUWDQWWRFKHFN
WKH GDWD IRU FRPSOHWHQHVV ZLWK UHVSHFW WR WKH OLVW RI IDFWRUV
UHVSRQVHYDULDEOHVDQGRWKHUYDULDEOHVUHTXLUHGWRDFKLHYHWKH
VWXG\JRDOV7KHDQDO\VWVKRXOGFRQGXFWFKHFNVRQWKHPDVWHU
GDWD VHW WR GLVFRYHU FRUUHFWDEOH HUURUV LQ FRGLQJ GDWD HQWU\
EDVLF FDOFXODWLRQV HWF ,W LV DOVR LPSRUWDQW WR FKHFN WKH GDWD
XVLQJWDEOHVDQGSORWVZKLFKDUHXVHIXOWRROVIRU4&FKHFNVIRU

7DEOH  7HUPLQRORJ\
7HUP 0HDQLQJ
$OWHUQDWLYHWHVW $'337LQZKLFKPHDVXUHVRIYLDEOHFHOOVDUHEDVHGRQWHFKQLTXHVGLIIHUHQWIURPFRQYHQWLRQDOFXOWXUH
PHWKRGVWKDWXWLOL]HHQXPHUDWLRQRUSRVLWLYHQHJDWLYHRXWFRPHV
$QDO\VLVRIFRYDULDQFH $QH[WHQVLRQRI$129$WRLQFOXGHWKHPDWKHPDWLFDOUHODWLRQVKLSEHWZHHQWKHUHVSRQVHYDULDEOHDQGD
FRYDULDWHHJWHPSHUDWXUHLQWKHVWDWLVWLFDOPRGHO
$QDO\VLVRIYDULDQFH$129$ ,QWKLVFRQWH[WDVWDWLVWLFDOPHWKRGRORJ\IRUSDUWLWLRQLQJWKHREVHUYHGYDULDQFHRID
UHVSRQVHYDULDEOHLQWRFRPSRQHQWYDULDQFHVDVVRFLDWHGZLWKWKHUDQGRPHIIHFWVIDFWRUV
%DODQFHGGHVLJQ $OOODERUDWRULHVFRQGXFWWKHVDPHVHWRIWHVWV
%LRILO $VHOIRUJDQL]HGFRRSHUDWLYHVHVVLOHFRPPXQLW\RIPLFURRUJDQLVPVW\SLFDOO\HPEHGGHGLQDPDWUL[RI
H[WUDFHOOXODUSRO\PHULFVXEVWDQFHV
%LRILOPWHV $'337LQZKLFKWKHELRFKDOOHQJHLVDPLFURELDOELRILOPRQWKHVXUIDFHRIDFDUULHUHJ
&DUULHU $VPDOOHDVLO\PDQLSXODWHGREMHFWZLWKZKLFKWKHPLFURELDOELRFKDOOHQJHLVDVVRFLDWHGLQELRILOPWHVWVRUGULHG 
VXUIDFHWHVWVDOVRFDOOHGDFRXSRQ
&ROODERUDWLYHVWXG\ (YDOXDWLRQRIDVWDQGDUGL]HG'337PHWKRGEDVHGRQWHVWVRILGHQWLFDOGLVLQIHFWDQWWUHDWPHQWVFRQGXFWHGE\D
QXPEHURIODERUDWRULHVHDFKIROORZLQJWKHZULWWHQSURWRFROIRUWKHPHWKRG
'HQVLW\ 1XPEHURIYLDEOHPLFUREHVXVXDOO\H[SUHVVHGDV&)8XQLWHJ³&)8FDUULHU´RU³&)8FP

RI
FDUULHUVXUIDFHDUHD´
'LVLQIHFWDQWSURGXFW
SHUIRUPDQFHWHVW'337
$PLFURELRORJLFDODVVD\FRQGXFWHGWRDVVHVVWKHDELOLW\RIDGLVLQIHFWDQWSURGXFWWRNLOORURWKHUZLVHLQDFWLYDWH
OLYLQJPLFUREHV
'LVLQIHFWDQWWUHDWPHQW 8VXDOO\DFKHPLFDORUDIRUPXODWLRQRIPXOWLSOHFKHPLFDOVDWDVSHFLILHGVWUHQJWKRUGLOXWLRQIRUDVSHFLILHGFRQWDF 
WLPHDQGWKHPHDQVE\ZKLFKWKHPLFUREHVDUHH[SRVHGWRWKHFKHPLFDOHJLPPHUVHGRUVSUD\HG6RPHWLPHV
DSK\VLFDOWUHDWPHQWVXFKDVVRQLFDWLRQRUDSK\VLFRFKHPLFDOWUHDWPHQWVXFKDVDGLVLQIHFWLQJWRZHOHWWH
'ULHGVXUIDFHWHVW $'337LQZKLFKWKHELRFKDOOHQJHLVDQDVVHPEODJHRIPLFUREHVLQDGULHGRUJDQLFILOPRQDFDUULHUVXUIDFH
/RJGHQVLW\ ORJ

WUDQVIRUPHGGHQVLW\
/RJUHGXFWLRQ/5 7KHPHDQRIORJ

WUDQVIRUPHGGHQVLWLHVIRUWKHWUHDWHGFDUULHUVVXEWUDFWHGIURPWKHPHDQRIORJ

WUDQVIRUPHG
GHQVLWLHVIRUWKHXQWUHDWHGFDUULHUV,IWKHGLVLQIHFWDQWLQDFWLYDWHVQRQHRIWKHPLFUREHVWKHH[SHFWHG/5LV]HURDQG
LIWKHGLVLQIHFWDQWLVHIIHFWLYHWKHH[SHFWHG/5LVSRVLWLYH
0RVWSUREDEOHQXPEHU031 %DVHGRQDVHULDOGLOXWLRQDVVD\ZKHUHD31UHVSRQVHLVREVHUYHGIRUHDFKDVVHPEODJHZHOOWXEHFDUULHUHWF
DWHDFKGLOXWLRQWKH031LVWKHGHQVLW\DWZKLFKWKHREVHUYHGGDWDDUHPRVWOLNHO\
2XWOLHU 2EVHUYDWLRQWKDWDSSHDUVWRGHYLDWHPDUNHGO\IURPRWKHUPHPEHUVRIWKHGDWDVHW
3RVLWLYHQHJDWLYH31
RXWFRPH
)RUDQDVVHPEODJHRIPLFUREHVRQDFDUULHULQDGLOXWLRQWXEHHWFWKHRXWFRPHLVSRVLWLYHZKHQWKHUHLVDWOHDVW
RQHYLDEOHPLFUREHLQWKHDVVHPEODJHDQGLVQHJDWLYHZKHQDOOPLFUREHVLQWKHDVVHPEODJHDUHQRQYLDEOH
4XDQWDOWHVW 31RXWFRPHLVREVHUYHGIRUHDFKWUHDWHGFDUULHUDQGQRGDWDDUHFROOHFWHGIRUXQWUHDWHGFDUULHUV
4XDQWLWDWLYHWHVW 9LDEOHPLFUREHVRQHDFKFDUULHUDUHHQXPHUDWHGWKHW\SLFDOUHVXOWRIWKHHQXPHUDWLRQLVWKHGHQVLW\RIYLDEOH
PLFUREHV
5HSHDWDEOH $UHSHDWDEOH'337SURGXFHVQHDUO\WKHVDPH/5YDOXHVZKHQWKHVDPHGLVLQIHFWDQWWUHDWPHQWLVWHVWHGRQGLIIHUHQW
GD\VZLWKLQDODERUDWRU\
5HSURGXFLEOH $UHSURGXFLEOH'337PHWKRGZLOOSURGXFHQHDUO\WKHVDPH/5YDOXHZKHQWKHVDPHGLVLQIHFWDQWWUHDWPHQWLV
UHWHVWHGLQDGLIIHUHQWODERUDWRU\
5HVHPEODQFH ,QRFXODWHGFDUULHUVDUHWKHH[SHULPHQWDOXQLWVLQDVXUIDFH'337,WLVGHVLUDEOHIRUWKHFDUULHUVWRUHVHPEOHHDFK
RWKHU)RUTXDQWLWDWLYHDQGVHPLTXDQWLWDWLYH'337PHWKRGVUHVHPEODQFHDVVHVVPHQWLVEDVHGRQYLDEOHPLFUREH
GHQVLWLHVRQFRQWUROFDUULHUV
5HVSRQVLYH $UHVSRQVLYH'337PHWKRGLVVHQVLWLYHHQRXJKWKDWLWFDQGHWHFWDQLPSRUWDQWHIILFDF\±UHVSRQVHH IHFW
6HPLTXDQWLWDWLYHWHVW 'DWDDUHFROOHFWHGIRUERWKWUHDWHGDQGXQWUHDWHGFDUULHUVVRPHRUDOORIWKHYLDEOHPLFUREHGHQVLWLHVDUHEDVHG
RQ31RXWFRPHV
6XVSHQVLRQWHVW $'337LQZKLFKWKHELRFKDOOHQJHLVDVXVSHQVLRQRISODQNWRQLFPLFUREHV
7\SH64

VHPLTXDQWLWDWLYHWHVW )RUHDFKXQWUHDWHGFDUULHUWKHYLDEOHPLFUREHGHQVLW\LVHQXPHUDWHGE\DFRQYHQWLRQDOFRXQWLQJWHFKQLTXHDQGIRU
HDFKWUHDWHGFDUULHUD31ZKROHFDUULHURXWFRPHLVUHFRUGHG
7\SH64

VHPLTXDQWLWDWLYHWHVW )RUHDFKFDUULHUWUHDWHGRUXQWUHDWHGWKHPLFUREHVDUHKDUYHVWHGIURPWKHFDUULHULQWRVXVSHQVLRQDQGDGLOXWLRQ
VHULHVLVIRUPHGIRUHDFKVXVSHQVLRQ,QVWHDGRIXVLQJFRXQWLQJWHFKQLTXHVIRUHQXPHUDWLQJYLDEOHPLFUREHVPXOWLSOH
WXEHVRUZHOOVDUHFUHDWHGIURPHDFKGLOXWLRQD31RXWFRPHLVREVHUYHGIRUHDFKWXEHDQGWKHGHQVLW\RIYLDEOH
PLFUREHVIRUHDFKFDUULHULVIRXQGE\WKH031PHWKRG
7\SH64

VHPLTXDQWLWDWLYHWHVW $ZKROHFDUULHU31RXWFRPHLVREVHUYHGIRUHDFKWUHDWHGFDUULHU)RUHDFKFRQWUROFDUULHUWKHPLFUREHVDUH
KDUYHVWHGWKHVXVSHQVLRQLVVHULDOO\GLOXWHGPXOWLSOHWXEHVRUZHOOVDUHFUHDWHGIURPHDFKGLOXWLRQ31RXWFRPHV
DUHREVHUYHGIRUHDFKWXEHDQGFDUULHUGHQVLW\LVFDOFXODWHGE\WKH031PHWKRG
8QEDODQFHGGHVLJQ 6RPHRIWKHODERUDWRULHVFRQGXFWDGLIIHUHQWVHWRIWHVWVWKDQRWKHUODERUDWRULHV