Psychological
Review
1987, Vol.
94,
No.
2,
211-228
Copyright
1987
by the
American
Psychological
Association,
Inc.
0033-295X/87/$00.75
Confirmation,
Discontinuation,
and
Information
in
Hypothesis
Testing
Joshua
Klayman
and
\bung-Won
Ha
Center
for
Decision
Research, Graduate School
of
Business, University
of
Chicago
Strategies
for
hypothesis
testing
in
scientific investigation
and
everyday reasoning
have
interested
both psychologists
and
philosophers.
A
number
of
these scholars
stress
the
importance
of
disconnr-
mation
in
reasoning
and
suggest that people
are
instead
prone
to
a
general
deleterious
"confirmation
bias."
In
particular,
it is
suggested
that
people tend
to
test
those
cases
that
have
the
best chance
of
verifying
current
beliefs
rather than those
that
have
the
best
chance
of
falsifying
them.
We
show,
howevei;
that many phenomena labeled "confirmation
bias"
are
better
understood
in
terms
of a
general
positive
test
strategy.
With
this
strategy, there
is a
tendency
to
test cases that
are
expected
(or
known)
to
have
the
property
of
interest rather than those expected
(or
known)
to
lack
that property.
This strategy
is not
equivalent
to
confirmation bias
in the first
sense;
we
show that
the
positive
test
strategy
can be a
very good heuristic
for
determining
the
truth
or
falsity
of a
hypothesis
under
realistic
conditions.
It
can,
however,
lead
to
systematic
errors
or
inefficiencies.
The
appropriateness
of
human
hypothesis-testing
strategies
and
prescriptions
about optimal strategies must
be
under-
stood
in
terms
of the
interaction between
the
strategy
and the
task
at
hand.
A
substantial proportion
of the
psychological
literature
on
hypothesis testing
has
dealt
with
issues
of
confirmation
and
dis-
confirmation.
Interest
in
this topic
was
spurred
by the
research
findings
of
Wason
(e.g.,
1960,1968)
and by
writings
in the
phi-
losophy
of
science
(e.g.,
Lakatos,
1970;
Platt,
1964; Popper,
1959,
1972),
which related hypothesis testing
to the
pursuit
of
scientific
inquiry. Much
of
the
work
in
this
area,
both empirical
and
theoretical, stresses
the
importance
of
disconfirmation
in
learning
and
reasoning.
In
contrast, human reasoning
is
often
said
to be
prone
to a
"confirmation
bias"
that hinders
effective
learning.
However,
confirmation bias
has
meant
different
things
to
different
investigators,
as
Fischhoff
and
Beyth-Marom
point
out in a
recent
review
(1983).
For
example,
researchers
studying
the
perception
of
correlations
have
proposed that people
are
overly
influenced
by the
co-occurrence
of two
events
and in-
sufficiently
influenced
by
instances
in
which
one
event occurs
without
the
other
(e.g.,
Arkes&Harkness,
1983;
Crocker,
1981;
Jenkins
&
Ward,
196S;
Nisbett
&
Ross,
1980;
Schustack
&
Stemberg,
1981;
Shaklee
&
Mims,
1982;
Smedslund,
1963;
Ward
&
Jenkins,
1965).
Other researchers
have
suggested that
people
tend
to
discredit
or
reinterpret information counter
to a
hypothesis
they
hold
(e,g.,
Lord, Ross,
&
Lepper,
1979; Nisbett
&Ross,
1980;
Ross
&
Lepper,
1980)
or
they
may
conduct biased
tests that pose little risk
of
producing
discontinuing
results
This
work
was
supported
by
Grant
SES-8309586
from the
Decision
and
Management Sciences program
of the
National Science Founda-
tion.
We
thank
Hillel
Einhorn,
Ward
Edwards. Jackie
Cnepp,
William
Goldstein,
Steven
Hoch,
Robin Hogarth, George
Loewenstein,
Nancy
Penningtou,
Jay
Russo,
Paul
Schoemakez;
William Swann,
Tom
Tra-
basso,
Ryan
Tweney,
and
three
anonymous
reviewers
for
invaluable
comments
on
earlier
drafts.
Correspondence concerning this article should
be
addressed
to
Joshua Klayman, Graduate School
of
Business,
University
of
Chicago,
1101
East 58th
Street,
Chicago,
Illinois
60637.
(e.g.,
Snyder,
1981;
Snyder
&
Campbell,
1980;
Snyder
&
Swann,
1978).
The
investigation
of
hypothesis testing
has
been concerned
with
both descriptive
and
prescriptive issues.
On the one
hand,
researchers
have
been interested
in
understanding
the
processes
by
which people
form,
test,
and
revise hypotheses
in
social
judg-
ment,
logical
reasoning,
scientific
investigation,
and
other
do-
mains.
On the
other hand, there
has
also
been
a
strong implica-
tion that people
are
doing
things
the
wrong
way
and
that
efforts
should
be
made
to
correct
or
compensate
for the
failings
of hu-
man
hypothesis
testing.
This
concern
has
been expressed with
regard
to
everyday
reasoning (e.g.,
see
Bruner,
1951; Nisbett
&
Ross, 1980)
as
well
as
professional
scientific
endeavor
(e.g.,
Mahoney,
1979;
Platt,
1964).
In
this article,
we focus on
hypotheses about
the
factors
that
predict,
explain,
or
describe
the
occurrence
of
some
event
or
property
of
interest.
We
mean this broadly,
to
include hypothe-
ses
about causation
("Cloud
seeding increases
rainfall"),
cate-
gorization ("John
is an
extrovert"), prediction ("The
major
risk
factors
for
schizophrenia
are . .
."),
and
diagnosis ("The most
diagnostic
signs
of
malignancy
are.
.
.").
We
consider both
de-
scriptive
and
prescriptive issues concerning information gather-
ing
in
hypothesis-testing
tasks.
We
include
under
this rubric
tasks
that require
the
acquisition
of
evidence
to
determine
whether
or not a
hypothesis
is
correct
The
task
may
require
the
subject
to
determine
the
truth value
of a
given
hypothesis
(e.g.,
Jenkins
&
Ward,
1965; Snyder
&
Campbell, 1980; Wason,
1966),
or to find the one
true hypothesis
among
a
set or
universe
of
possibilities (e.g.,
Bruner,
Goodnow,
&
Austin,
1956;
Mynatt,
Doherty,&
Tweney,
1977,1978;
Wason,
1960,1968).
The
task
known
as
rule discovery
(Wason,
1960)
serves
as
the
basis
for
the
development
of our
analyses,
which
we
later extend
to
other kinds
of
hypothesis testing.
We first
examine what
"confirmation"
means
in
hypothesis
testing.
Different
senses
of
confirmation
have
been poorly distinguished
in the
literature,
contributing
to
misinterpretations
of
both empirical
findings
211
212
JOSHUA
KLAYMAN
AND
YOUNG-WON
HA
and
theoretical
prescriptions.
We
propose that many phenom-
ena of
human
hypothesis testing
can be
understood
in
terms
of
a
general
positive
test
strategy.
According
to
this strategy,
you
test
a
hypothesis
by
examining instances
in
which
the
property
or
event
is
expected
to
occur
(to
see
if it
does occur),
or by
exam-
ining
instances
in
which
it is
known
to
have
occurred
(to see if
the
hypothesized conditions prevail). This basic strategy
sub-
sumes
a
number
of
strategies
or
tendencies
that
have
been sug-
gested
for
particular
tasks,
such
as
confirmation strategy, veri-
fication
strategy,
matching
bias,
and
illicit conversion.
As
some
of
these names
imply,
this approach
is not
theoretically proper.
We
show,
however,
that
the
positive test strategy
is
actually
a
good all-purpose heuristic across
a
range
of
hypothesis-testing
situations, including situations
in
which
rules
and
feedback
are
probabilistic. Under commonly occurring conditions, this strat-
egy
can be
well
suited
to the
basic goal
of
determining whether
or not a
hypothesis
is
correct.
Next,
we
show
how the
positive test strategy provides
an
inte-
grative
frame
for
understanding
behavior
in a
variety
of
seem-
ingly
disparate domains, including concept identification, logi-
cal
reasoning, intuitive personality testing, learning
from
out-
come
feedback,
and
judgment
of
contingency
or
correlation.
Our
thesis
is
that
when
concrete, task-specific information
is
lacking,
or
cognitive demands
are
high, people
rely
on the
posi-
tive
test strategy
as a
general
default
heuristic.
Like
any
all-pur-
pose strategy, this
may
lead
to a
variety
of
problems
when
ap-
plied
to
particular situations,
and
many
of the
biases
and
errors
described
in the
literature
can be
understood
in
this light.
On
the
other hand,
this
general heuristic
is
often
quite adequate,
and
people
do
seem
to be
capable
of
more sophisticated strate-
gies
when
task conditions
are
favorable.
Finally,
we
discuss some
ways
in
which
our
task analysis
can
be
extended
to a
wider
range
of
situations
and how it can
con-
tribute
to
further
investigation
of
hypothesis-testing
processes.
Confirmation
and
Disconfirmation
in
Rule
Discovery
The
Rule
Discovery
Task
Briefly,
the
rule discovery task
can be
described
as
follows:
There
is a
class
of
objects with
which
you are
concerned; some
of
the
objects
have
a
particular property
of
interest
and
others
do
not.
The
task
of
rule discovery
is to
determine
the set of
characteristics
that
differentiate
those with
this
target property
from
those without
it. The
concept identification paradigm
in
learning studies
is a
familiar
example
of a
laboratory
rule-dis-
covery
task
(e.g.
Bruner,
Goodnow,
&
Austin,
1956;
Levine,
1966;
Trabasso
&
Bower,
1968). Here,
the
objects
may be, for
example,
visual stimuli
in
different
shapes,
colors,
and
loca-
tions.
Some
choices
of
stimuli
are
reinforced, others
are
not.
The
learner's goal
is to
discover
the
rule
or
"concept"
(e.g.,
red
circles) that determines reinforcement.
Wason
(1960)
was the first to use
this type
of
task
to
study
people's understanding
of the
logic
of
confirmation
and
discon-
firmation. He saw the
rule-discovery
task
as
representative
of
an
important aspect
of
scientific reasoning (see also
Mahoney,
1976,
1979;
Mynatt
et
al.,
1977, 1978; Simon,
1973).
To
illus-
trate
the
parallel between rule discovery
and
scientific investiga-
tion,
consider
the
following
hypothetical case.
You
are an
astro-
physicist,
and you
have
a
hypothesis about
what
kinds
of
stars
develop
planetary systems. This hypothesis
might
be
derived
from
a
larger theory
of
astrophysics
or may
have
been induced
from
past observation.
The
hypothesis
can be
expressed
as a
rule,
such
that those stars
that
have
the
features
specified
in the
rule
are
hypothesized
to
have
planets
and
those
not fitting the
rule
are
hypothesized
to
have
no
planets.
We
will
use the
symbol
R
H
for the
hypothesized
rule,
H for the set of
instances
that
fit
that
hypothesis,
and H for the set
that
do not fit it.
There
is a
domain
or
"universe"
to
which
the
rule
is
meant
to
apply
(e.g.,
all
stars
in our
galaxy),
and in
that domain there
is a
target
set
(those
stars
that really
do
have
planets).
You
would
like
to find
the
rule that exactly specifies which members
of the
domain
are
in
the
target
set
(the rule
that
describes
exactly
what
type
of
stars
have
planets).
We
will
use T for the
target
set,
and
R
T
for
the
"correct"
rule, which
specifies
the
target
set
exactly.
Let us
assume
for now
that such
a
perfect rule exists. (Alternate ver-
sions
of the
rule might exist,
but for our
purposes, rules
can be
considered identical
if
they
specify
exactly
the
same
set T.) The
correct
rule
may be
extremely complex, including conjunc-
tions, disjunctions,
and
trade-offs
among features.
Your
goal
as
a
scientist,
though,
is to
bring
the
hypothesized
rule
R
H
in
line
with
the
correct
rule
R
T
and
thus
to
have
the
hypothesized
set
H
match
the
target
set T.
\ou
could then predict exactly which
stars
do and do not
have
planets. Similarly,
a
psychologist might
wish
to
differentiate
those
who are at risk for
schizophrenia
from
those
who are
not,
or an
epidemiologist
might
wish
to
understand
who
does
and
does
not
contract AIDS.
The
same
structure
can
also
be
applied
in a
diagnostic context.
For
exam-
ple,
a
diagnostician might seek
to
know
the
combination
of
signs
that
differentiates
benign
from
malignant tumors.
In
each case,
an
important component
of the
investigative
process
is the
testing
of
hypotheses. That
is, the
investigator
wants
to
know
if the
hypothesized rule
R
H
is
identical
to the
correct
rule
R
T
and if
not,
how
they
differ.
This
is
accomplished
through
the
collection
of
evidence, that
is, the
examination
of
instances.
For
example,
you
might
choose
a
star hypothesized
to
have
planets
and
train your telescope
on it to see if it
does
indeed
have
planets,
or you
might examine tumors expected
to
be
benign,
to see if any are in
fact
malignant.
Wason
(1960, 1968) developed
a
laboratory version
of
rule
discovery
to
study
people's hypothesis-testing strategies
(in
par-
ticular,
their
use of
confirmation
and
disconfirmation),
in a
task
that "simulates
a
miniature
scientific
problem" (1960,
p.
139).
In
Wason's
task,
the
universe
was
made
up of all
possible sets
of
three numbers
("triples").
Some
of
these triples
fit the
rule,
in
other
words,
conformed
to a
rule
the
experimenter
had in
mind.
In our
terms,
fitting the
experimenter's
rule
is the
target
property that subjects must learn
to
predict.
The
triples that
fit
the
rule, then, constitute
the
target set,
T.
Subjects were
pro-
vided
with
one
target triple
(2, 4, 6), and
could
ask the
experi-
menter about
any
others
they
cared
to. For
each triple
the
sub-
ject
proposed,
the
experimenter responded
yes
(fits
the
rule)
or
no
(does
not fit).
Although subjects might
start
with
only
a
vague
guess,
they
quickly
formed
an
initial hypothesis about
the
rule
(R
H
).
For
example, they might guess
that
the
rule
was
"three
consecutive even numbers." They could then
perform
one
of
two
types
of
hypothesis
tests
(Htests):
they
could propose
a
triple
they
expected
to be a
target (e.g.,
6, 8,
10),
or a
triple
CONFIRMATION,
DISCONFIRMATION,
AND
INFORMATION
Result
213
u
Action
+Htest
-Htest
•Yes"
(in T)
'No'
On
T)
HnT:
Ambiguous
verification
HnT:
Conclusive
falsification
HnT:
Impossible
HnT:
Ambiguous
verification
Figure
1.
Representation
of
a
situation
in
which
the
hypothesized
rule
is
embedded
within
the
correct
rule,
as
in
Wason's
(I960)
"2, 4, 6"
task.
(U = the
universe
of
possible
instances
[e.g.,
all
triples
of
numbers];
T = the set of
instances
that
have
the
target
property
[e.g.,
they
fit the
experimenter's
rule:
increasing];
H
=
the set of
instances
that
fit the
hypothesized
rule
[e.g.,
increasing
by
2].)
they
expected
not to be
{e.g.,
2,4,7).
In
this
paper,
we
will
refer
to
these
as a
positive hypothesis test
(+Htest)
and a
negative
hypothesis test
(-Htest),
respectively.
Wason
found
that
people
made much more
use of
+Htests
than
-Htests.
The
subject whose hypothesis
was
"consecu-
tive
evens,"
for
example, would
try
many examples
of
consec-
utive-even
triples
and
relatively
few
others.
Subjects
often
be-
came quite
confident
of
their hypotheses
after
a
series
of+Ht-
ests
only.
In
Wason's
(1960)
task
this confidence
was
usually
unfounded,
for
reasons
we
discuss
later. Wason
described
the
hypothesis testers
as
"seeking
confirmation" because they
looked predominantly
at
cases that
fit
their hypothesized rule
for
targets (e.g.,
different
sets
of
consecutive even numbers).
We
think
it
more appropriate
to
view
this "confirmation
bias"
as a
manifestation
of the
general
hypothesis-testing
strategy
we
call
the
positive test
(+
test) strategy.
In
rule
dis-
covery,
the
+test
strategy leads
to the
predominant
use of
+Htests,
in
other words,
a
tendency
to
test
cases
you
think
will
have
the
target property.
The
general
tendency toward
-(-testing
has
been widely repli-
cated.
In a
variety
of
different
rule-discovery
tasks
(KJayman
&
Ha,
1985;Mahoney,
1976, 1979;
Mynattetal.,
1977, 1978;
Taplin,
1975;
Tweney
et
al.,
1980;
Wason
&
Johnson-Laird,
1972)
people look predominantly
at
cases they expect
will
have
the
target property, rather than
cases
they expect
will
not As
with
nearly
all
strategies, people
do not
seem
to
adhere strictly
to
-(-testing,
however.
For
instance, given
an
adequate number
of
test opportunities
and a
lack
of
pressure
for a
quick evaluation,
people
seem
willing
to
test more
widely
(Gorman
&
Gorman,
1984;
Klayman
& Ha,
1985).
Of
particular interest
is one ma-
nipulation that greatly improved success
at
Wason's
2,4,6
task.
Tweney
et al.
(1980)
used
a
task structurally identical
to Wa-
son's
but
modified
the
presentation
of
feedback. Triples were
classified
as
either
DAX or
MED, rather than
yes
(fits
the
rule)
or no
(does
not fit). The
rule
for DAX was
Wason's original
ascending-order
rule,
and all
other triples
were
MED. Subjects
in the
DAX/MED
version used even
fewer
—Htests
than
usual.
However,
they treated
the DAX
rule
and the MED
rule
as two
separate hypotheses,
and
tested each with
-(-Htests,
thereby
fa-
cilitating
a
solution.
The
thrust
of
this work
has
been more than just descriptive,
however.
There
has
been
a
strong emphasis
on the
notion that
a
-Hest
strategy
(or
something like
it)
will
lead
to
serious errors
or
inefficiencies
in the
testing
of
hypotheses.
We
begin
by
taking
a
closer look
at
this assumption.
We
examine
what
philosophers
of
science such
as
Popper
and
Platt
have
been
arguing,
and how
that translates
to
prescriptions
for
information gathering
in
different
hypothesis-testing
situations.
We
then examine
the
task characteristics that control
the
extent
to
which
a
-Hest
strategy
deviates
from
those prescriptions.
We
begin with rule
discovery
as
described above,
and
then consider
what
happens
if
additional information
is
available
(examples
of
known tar-
gets
and
nontargets),
and if an
element
of
probabilistic error
is
introduced.
The
basic question
is, if
you
are
trying
to
determine
Action
+Htest
-Htest
Result
'Yes'
(in T)
'No'
(in
T)
HnT:
Ambiguous
verification
HnT:
Conclusive
falsification
HnT:
Conclusive
falsification
HnT:
Ambiguous
verification
Figure
2.
Representation
of a
situation
in
which
the
hypothesized
rule
overlaps
the
correct
rule.
214
JOSHUA
KLAYMAN
AND
YOUNG-WON
HA
u
Action
+Htest
-Htest
Result
'Yes'
On T)
'No"
On T)
HnT:
Ambiguous
verification
HnT:
Impossible
HnT:
Conclusive
falsification
HnT:
Ambiguous
verification
Figure
3.
Representation
of a
situation
in
which
the
hypothesized
rule surrounds
the
correct rule.
the
truth
or
falsity
of a
hypothesis,
when
is a
4-
test
strategy
un-
wise
and
when
is it
not?
The
Logic
of
Ambiguous
Versus
Conclusive
Events
As
a
class, laboratory
rule-discovery
tasks share three simpli-
fying
assumptions.
First,
feedback
is
deterministically
accurate.
The
experimenter provides
the
hypothesis tester with
error-free
feedback
in
accordance with
an
underlying
rule.
Second,
the
goal
is to
determine
the one
correct
rule
(R
T
).
All
other rules
are
classified
as
incorrect,
without regard
to
how
wrong
R
H
may
be,
although
the
tester
may be
concerned with
where
it is
wrong
in
order
to
form
a new
hypothesis. Third,
correctness
requires
both
sufficiency
and
necessity:
A
rule
is
incorrect
if it
predicts
an
instance will
be in the
target
set
when
it is not
(false
positive),
or
predicts
it
will
not be in the
target
set
when
it
is
(false
nega-
tive).
We
discuss
later
the
extent
to
which each
of
these
assump-
tions
restricts
generalization
to
other tasks.
Consider again
Wason's
original task. Given
the
triple
(2,
4,
6),
the
hypotheses that occur
to
most people
are
"consecutive
even
numbers,"
"increasing
by 2," and the
like.
The
correct
rule,
however,
is
much
broader,
"increasing
numbers."
Con-
sider subjects whose hypothesized rule
is
"increasing
by 2."
Those
who use
only
+Htests
(triples
that increase
by 2,
such
as
6,
8,10)
can
never discover that their rule
is
incorrect,
because
all
examples
of
"increasing
by 2"
also
fit the
rule
of
"increas-
ing." Thus,
it is
crucial
to try
-Htests
(triples that
do not
in-
crease
by 2,
such
as
2,4,7),
This situation
is
depicted
in
Figure
1.
Here,
U
represents
the
universe
of
instances,
all
possible tri-
ples
of
numbers.
T
represents
the
target set, triples
that
fit the
experimenter's rule
("increasing").
H
represents
the
hypothe-
sized
set,
triples that
fit the
tester's
hypothesized
rule
(say, "in-
creasing
by
2"). There
are in
principle
four
classes
of
instances,
although
they
do not all
exist
in
this
particular
example:
1.
HOT:
instances correctly hypothesized
to be in the
target
set
(positive
hits).
2.
HOT:
instances incorrectly hypothesized
to be in the
target
set
(false
positives).
3.
HOT:
instances correctly hypothesized
to
be
outside
the
target
set
(negative
hits).
4.
HOT:
instances incorrectly hypothesized
to be
outside
the
tar-
get set
(false
negatives).
Instances
of the
types
HnT
and H
n
T
falsify
the
hypothesis.
That
is, the
occurrence
of
either shows
conclusively
that
H
¥=
T,
thus
RH
i
1
RT?
the
hypothesized rule
is not the
correct one.
Instances
of the
types
H n T and H
H
f
verify
the
hypothesis,
in
the
sense
of
providing favorable evidence. However, these
in-
stances
are
ambiguous:
The
hypothesis
may be
correct,
but
these instances
can
occur even
if the
hypothesis
is not
correct.
Note that
there
are
only conclusive
falsifications,
no
conclusive
verifications.
This
logical condition
is the
backbone
of
philoso-
phies
of
science that urge investigators
to
seek
falsification
rather
than verification
of
their hypotheses
(e.g.,
Popper,
1959).
Put
somewhat
simplistically,
a
lifetime
of
verifications
can be
countered
by a
single conclusive falsification,
so it
makes sense
for
scientists
to
make
the
discovery
of
falsifications
their pri-
mary
goal.
Suppose,
then,
that
you are the
tester
in
Wason's
task, with
the
hypothesis
of
"increasing
by 2." If you try a
-f
Htest
(e.g.,
6,
8,10)
you
will
get
either
a yes
response, which
is an
ambiguous
verification
of the
type
H
Pi
T, or a no,
which
is a
conclusive
falsification
of the
type
H
Pi
f.
The
falsification
H n T
would
show
that meeting
the
conditions
of
your rule
is not
sufficient
to
guarantee membership
in T.
Thus,
+Htests
can be
said
to
be
tests
of the
rule's
sufficiency.
However,
unknown
to the
subjects
in the 2, 4, 6,
task (Figure
1)
there
are no
instances
of H n T,
because
the
hypothesized rule
is
sufficient:
Any
instance
follow-
ing
R
H
("increasing
by 2")
will
in
fact
be in the
target
set T
("increasing").
Thus,
+Htests
will
never produce falsification.
If
you
instead
try a
-Htest
(e.g.,
2,4,7)
you
will
get_either
a no
answer
which
is an
ambiguous verification
(H
fl
T) or a yes
answer
which
is a
conclusive
falsification
(HOT).
The
falsifica-
tion H
fl
T
shows that your conditions
are not
necessary
for
membership
in T.
Thus,
—Htests
test
a
rule's necessity.
In the
2, 4, 6
task,
—Htests
can
result
in
conclusive
falsification
be-
cause
R
H
is
sufficient
but not
necessary (i.e., there
are
some
target
triples
that
do not
increase
by 2).
In the
above situation,
the
Popperian
exhortation
to
seek
fal-
sification
can be
fulfilled
only
by
-Htesting,
and
those
who
rely
on
+Htests
are
likely
to be
misled
by the
abundant verification
they receive.
Indeed,
Wason
deliberately designed
his
task
so
that
this would
be the
case,
in
order
to
show
the
pitfalls
of
"con-
firmation
bias"
(Wason, 1962),
The
hypothesis-tester's
situa-
tion
is
not
always
like
this,
however.
Consider
the
situation
in
which
the
hypothesized
set
merely overlaps
the
target set,
as
shown
in
Figure
2,
rather than being
embedded
within
it,
as
CONFIRMATION,
DISCONF1RMATION,
AND
INFORMATION
Result
215
u
Action
+Htest
-Htest
'Yes'
(in T)
'No'
Cin T)
HnT:
Impossible
HnT:
Conclusive
falsification
HnT:
Conclusive
falsification
HnT:
Ambiguous
verification
Figure
4.
Representation
of a
situation
in
which
the
hypothesized rule
and the
correct rule
are
disjoint.
shown
in
Figure
1.
This
would
be the
case
if, for
example,
the
correct
rule
were
"three
even numbers." There
would
be
some
members
of H n T,
instances that were "increasing
by 2" but
not
"three
evens" (e.g.,
1,
3, 5), and
some members
of H
fl
T,
"three
evens"
but not
"increasing
by 2"
(e.g.,
4, 6, 2).
Thus,
conclusive
falsification
could occur with either
+Htests
or -H-
tests.
Indeed,
it is
possible
to be in a
situation just
the
opposite
of
Wason's,
shown
in
Figure
3.
Here,
the
hypothesis
is too
broad
and
"surrounds"
the
target
set.
This would
be the
case
if the
correct
rule were, say, "consecutive even numbers."
Now a
tes-
ter who did
only
-Htests
could
be
sorely misled, because there
are no
falsifications
of
the
type
HOT;
any
instance that violates
"increasing
by 2"
also violates "consecutive evens."
Only
+H-
tests
can
reveal conclusive
falsifications
(HOT
instances such
as
1,3,5).
Aside
from
these three
situations,
there
are two
other possible
relationships between
H
and T.
When
H
and
T
are
disjoint (Fig-
ure 4), any
+Htest
will
produce conclusive
falsification,
be-
cause nothing
in H is in T;
-Htests
could produce either
verifi-
cation
or
falsification.
This
is not
likely
in the
2,4,
6
task,
be-
cause
you are
given
one
known target instance
to
begin with.
In
the
last case (Figure
5), you
have
finally
found
the
correct
rule,
and H
coincides with
T.
Here, every
test
produces ambiguous
information;
a final
proof
is
possible only
if
there
is a finite
uni-
verse
of
instances
and
every case
is
searched.
In
naturally occurring situations,
as in
Wason's
(1960)
task,
one
could
find
oneself
in any
of
the
conditions
depicted,
usually
with
no way of
knowing
which.
Suppose,
for
example, that
you
are a
manufacturer
trying
to
determine
the
best
way
to
advertise
your
line
of
products,
and
your current hypothesis
is
that televi-
sion
commercials
are the
method
of
choice.
For
you,
the
uni-
verse,
U, is the set of
possible advertising methods;
the
target
set,
T, is the set of
methods that
are
effective,
and the
hypothe-
sized
set,
H, is
television commercials. Suppose that
in
fact
the
set of
effective
advertising methods
for
these products
is
much
broader:
any
visual medium (magazine
ads, etc.)
will
work.
This
is the
situation depicted
in
Figure
1.
If you try
+Htests
(i.e.,
try
instances
in
your
hypothesized
set,
television commer-
cials)
you
will
never discover that
your
rule
is
wrong,
because
television commercials will
be
effective.
Only
by
trying things
you
think
will
not
work
(-Htests)
can you
obtain
falsification.
\bu
might then discover
an
instance
of
the
type
HOT
nontele-
vision
advertising that
is
effective.
Suppose instead that
the
correct rule
for
effectively
advertis-
ing
these products
is to use
humor.
This
is the
situation
in
Fig-
ure
2. You
could
find a
(serious) television commercial that
you
thought
would
work,
but
does
not (H
D
T), or a
(humorous)
npntelevision
ad
that
you
thought
would
not
work,
but
does
(H
Pi
T).
Thus, conclusive
falsification
could occur
with
either
a
+Htest
or
a
-Htest.
If
instead
the
correct
rule
for
these prod-
ucts
is
more restricted, say, "prime-time television only,"
you
would
have
an
overly
broad
hypothesis,
as
shown
in
Figure
3.
In
that case,
you
will
never
obtain
falsification
if you use -H-
tests
(i.e.,
if you
experiment with methods
you
think
will
not
work),
because anything that
is not on
television
is
also
not on
prime time. Only
+
Htests
can
reveal conclusive falsifications,
by
finding
instances
of H n T
(instances
of
television commer-
cials
that
are not
effective).
What
is
critical, then,
is not the
testing
of
cases that
do not
fit
your
hypothesis,
but the
testing
of
cases that
are
most
likely
U
Action
-r-Htest
-Htest
Result
'Yes'
Cin
TD
'No'
Cin T)
HnT:
Ambiguous
verification
HnT:
Impossible
HnT:
Impossible
HnT:
Ambiguous
verification
Figure
5.
Representation
of the
situation
in
which
the
hypothesized rule coincides
with
the
correct
rule.