Optical
implementation
of
the
Hopfield
model
Nabil
H.
Farhat,
Demetri
Psaltis,
Aluizio
Prata,
and
Eung
Paek
Optical
implementation
of
content
addressable
associative
memory
based
on the
Hopfield
model
for
neural
networks
and on
the addition
of nonlinear
iterative
feedback
to a vector-matrix
multiplier
is described.
Nu-
merical
and experimental
results
presented
show
that the
approach
is capable
of introducing
accuracy
and
robustness
to optical
processing
while
maintaining
the traditional
advantages
of optics,
namely,
parallelism
and
massive
interconnection
capability.
Moreover
a
potentially
useful
link
between
neural
processing
and
optics that
can be of
interest in
pattern recognition
and
machine
vision is established.
1.
Introduction
It is
well known
that
neural networks
in
the eye-brain
system
process
information
in parallel
with
the aid
of
large numbers
of simple
interconnected
processing
el-
ements,
the neurons.
It is
also known
that
the
system
is
very adept
at recognition
and recall
from
partial
in-
formation
and
has
remarkable
error
correction
capa-
bilities.
Recently
Hopfield
described
a simple
model'
for
the
operation
of
neural
networks.
The
action
of individual
neurons
is
modeled
as a thresholding
operation
and
information
is stored
in the
interconnections
among
the
neurons.
Computation
is performed
by setting the
state
(on
or off)
of some
of the
neurons
according
to
an
external
stimulus
and,
with the
interconnections
set
according
to the
recipe
that Hopfield
prescribed,
the
state
of all
neurons
that
are interconnected
to those
that
are externally
stimulated
spontaneously
converges
to
the stored
pattern
that
is most similar
to
the external
input.
The basic operation
performed
is a nearest-
neighbor
search, a fundamental
operation
for pattern
recognition,
associative
memory, and
error correction.
A
remarkable
property
of the model
is that powerful
global computation
is performed
with
very simple,
identical
logic elements
(the neurons).
The intercon-
nections
provide the
computation power
to these simple
logic
elements and
also enhance
dramatically the
stor-
Nabil Farhat is with
University of Pennsylvania,
Moore
School of
Electrical Engineering,
Philadelphia,
Pennsylvania
19104; the other
authors
are with California
Institute
of Technology,
Electrical En-
gineering
Department,
Pasadena,
California
91125.
Received
24 December 1984.
0003-6935/85/101469-07$02.00/0.
© 1985 Optical
Society of America.
age
capacity; approximately
N/4 InN bits/neuron
can
be stored in a
network in which
each neuron
is con-
nected to N
others.
2
Another
important
feature is that
synchronization
among
the parallel
computing
elements
is
not required,
making
concurrent,
distributed
pro-
cessing feasible
in a massively
parallel structure.
Fi-
nally, the
model is insensitive
to local imperfections
such as
variations in the
threshold level
of individual
neurons or
the weights
of the interconnections.
Given these
characteristics
we were motivated
to
investigate
the feasibility
of implementing
optical
in-
formation
processing
and storage
systems that are
based
on this and
other similar models
of associative
memo-
ry.
3 4
Optical
techniques offer
an effective
means for
the implementation
of
programmable
global intercon-
nections
of very
large numbers
of identical
parallel
logic
elements. In
addition, emerging
optical technologies
such as
2-D spatial light
modulators, optical
bistability,
and
thin-film optical
amplifiers
appear to be very
well
suited for performing
the thresholding
operation
that
is necessary
for the implementation
of
the model.
The principle
of the Hopfield
model
and its impli-
cations
in optical information
processing
have been
discussed
earlier.
5 6
Here
we review
briefly the main
features of the
model, give as
an example the
results of
a numerical
simulation, describe
schemes
for its optical
implementation,
then
present
experimental
results
obtained
with one
of the schemes
and discuss their
implications
as
a content addressable
associative
memory
(CAM).
11. Hopfield
Model
Given a
set of M bipolar,
binary (1,-1)
vectors vim),
i
= 1,2,3. ..N, m
= 1,2,3 ... M,
these are stored
in a
synaptic
matrix in accordance
with
the recipe
;=
L v
,
i,]
= 1,2,3..
.N,
Ti =
0,
m1
(1)
vim) are
referred
to as
the
nominal
state
vectors
of the
15 May
1985 / Vol. 24,
No. 10 / APPLIED
OPTICS
1469
memory.
If the
memory
is addressed
by
multiplying
the
matrix
Tij
with
one
of the
state
vectors,
say
VImo)
it
yields
the
estimate
v(mo)
=
E Tijv(mO)
(2)
ii
E
E
V m)
m)
Vmo)
iji m
(3)
=
(N-
)v,(°)
+
E_
ammov,(')
m'.mO
where
N
am~mo
=
V(mo)V(m).
J
(mo)
consists
of the
sum
of
two
terms:
the
first
is
the
input
vector
amplified
by
(N
- 1);
the
second
is a
linear
combination
of
the
remaining
stored
vectors
and
it
represents
an
unwanted
cross-talk
term.
The
value
of
the
coefficients
ammo
is
equal
to
VN
?Ton
the average
(the
standard
deviation
of
the
sum
of
N -
1 random
bits),
and
since
(M
-
1) such
coefficients
are randomly
added,
the
value
of
the
second
term
will
on
the
average
be
equal
to
\/(M
- 1)(N
-
1).
If
N
is sufficiently
larger
than
M, with
high
probability
the elements
of the
vector
vmO)
will
be
positive
if the
corresponding
ele-
ments
of
v("m)
are
equal
to
+1
and
negative
otherwise.
Thresholding
of
vmo)
will
therefore
yield
v
mo):
vl)
sgn[v0°)1
= 1+1
if
0mo)
>
o
-1
otherwise.
When
the
memory
is
addressed
with
a binary
valued
vector
that
is
not
one
of the
stored
words,
the
vector-
matrix
multiplication
and
thresholding
operation
yield
an
output
binary
valued
vector
which,
in
general,
is an
approximation
of
the
stored
word
that
is at
the
shortest
Hamming
distance
from
the
input
vector.
If this
out-
put
vector
is
fed
back
and
used
as the
input
to
the
memory,
the
new
output
is
generally
a
more
accurate
version
of the
stored
word
and
continued
iteration
converges
to
the
correct
vector.
The
insertion
and
readout
of
memories
described
above
are
depicted
schematically
in Fig.
1. Note
that
in
Fig.
1(b)
the
estimate
O(mo) can
be
viewed
as
the
weighted
projection
of Tij.
Recognition
of
an input
vector
that
corresponds
to
one of
the
state
vectors
of
the
memory
or
is
close
to
it (in
the
Hamming
sense)
is
manifested
by
a stable
state
of
the
system.
In
practice
unipolar
binary
(0,1)
vectors
or words
b m)
of
bit length
N
may
be
of
interest.
The
above
equations
are
then
applicable
with
[2br)
-
1]
replacing
v.m)
in
Eq.
(1) and
b~mo)
replacing
v(mo)
in Eq.
(2).
For
such
vectors
the
SNR
of
the
estimate
v(mo)
can
be
shown
to be
lower
by
a factor
of V-2.1
An
example
of
the
Tij
matrix
formed
from
four
binary
unipolar
vectors,
each
being
N
= 20
bits
long,
is
given
in Fig.
2
along
with
the
result
of a
numerical
simulation
of
the
process
of initializing
the
memory
matrix
with
a
partial
version
of bW
4
) in
which
the
first
eight
digits
of
bW
4
are
retained
and
the
remainder
set
to
zero.
The
Hamming
distance
between
the
initializing
vector
and
bI
4
)
is 6 bits
and it is
9 or more
bits for
the other
three
V(m)
T;
(M)
T
V(m
Vlm)V(m)
V/mo)
Tr(m)=
O
~~~
~~A(MO)
(O
j
Tm),
TO
Vo 0
T
V|
(a)
READ-IN
(b)
READ-OUT
Fig.
1.
(a) Insertion
and
(b) readout
of
memories.
stored
vectors.
It
is
seen
that
the
partial
input
is
rec-
ognized
as bi
4
) in
the
third
iteration
and
the
output
re-
mains
stable
as
h(4)
thereafter.
This
convergence
to
a
stable
state
generally
persists
even
when
the
Tij
matrix
is binarized
or
clipped
by
replacing
negative
elements
by
minus
ones
and
positive
elements
by
plus
ones
evi-
dencing
the
robustness
of
the
CAM.
A
binary
synaptic
matrix
has
the
practical
advantage
of
being
more
readily
implementable
with
fast
programmable
spatial
light
modulators
(SLM)
with
storage
capability
such
as the
Litton
Lightmod.
7
Such
a binary
matrix,
implemented
photographically,
is
utilized
in
the
optical
implemen-
tation
described
in
Sec.
III
and
evaluated
in
Sec.
IV
of
this
paper.
Several
schemes
for
optical
implementation
of
a
CAM
based
on the
Hopfield
model
have
been
described
ear-
lier.
5
In
one
of
the
implementations
an
array
of light
emitting
diodes
(LEDs)
is
used
to
represent
the
logic
elements
or neurons
of the
network.
Their
state
(on
or
off)
can
represent
unipolar
binary
vectors
such
as
the
state
vectors
b m)
that
are
stored
in
the
memory
matrix
Ti.
Global
interconnection
of
the
elements
is
realized
as shown
in
Fig.
3(a)
through
the
addition
of
nonlinear
feedback
(thresholding,
gain,
and
feedback)
to
a con-
ventional
optical
vector-matrix
multiplier
8
in which
the
array
of
LEDs
represents
the
input
vector
and
an
array
of
photodiodes
(PDs)
is
used
to
detect
the
output
vector.
The
output
is thresholded
and
fed
back
in parallel
to
drive
the
corresponding
elements
of
the
LED
array.
Multiplication
of
the
input
vector
by
the
Tij
matrix
is
achieved
by
horizontal
imaging
and
vertical
smearing
of
the
input
vector
that
is
displayed
by the
LEDs
on
the
plane
of the
Tij
mask
[by
means
of an
anamorphic
lens
system
omitted
from
Fig.
3(a)
for
simplicity].
A
second
anamorphic
lens
system
(also
not
shown)
is
used
to
collect
the
light
emerging
from
each
row
of
the
Tij
mask
on
individual
photosites
of
the
PD
array.
A bipolar
Tj
matrix
is
realized
in
incoherent
light
by
dividing
each
row
of
the Tij
matrix
into
two
subrows,
one
for
positive
and
one
for
negative
values
and
bringing
the
light
emerging
from
each
subrow
to
focus
on
two
adjacent
photosites
of
the
PD
array
that
are
electrically
con-
nected
in
opposition
as
depicted
in
Fig.
3(b).
In the
system
shown
in
Fig.
3(a),
feedback
is
achieved
by
electronic
wiring.
It is
possible
and
preferable
to
dis-
pose
of
electronic
wiring
altogether
and
replace
it
by
optical
feedback.
This
can
be
achieved
by
combining
the
PD
and
LED
arrays
in
a single
compact
hybrid
or
1470
APPLIED
OPTICS
/
Vol. 24,
No.
10
/ 15
May
1985
(1)
(2) (3)
(4)
bi
bi bi b
I I 0 1
a 1 9 9
1 9 1
I aut
9
I
1
9
(a)
Tij
-2 9-2 -2
-2 9-2 9 9
0-2 02-2
-2 92 9
-2 9-2 9 0-
2
0 0-4 1
9-2 4 2-2
2 9-2 9 9
92 9-2-2
4-2 9-2-2
9-2 42-2
4-2 0-2-2
0 2-4-2 2-i
0-2 92 2-i
-2 4-2 0 9O
2 9-2-4 9
2 9 2 9-4
4
92 0-2-2
2
-4 2 9 2 2-2
0-2 92 2-2
1 -;
!ii
I -I
I
I
1-2
9
4
12041
9 2-2-i
I-2 0
9
-2 -2
9 -2 -2 -i
922
21
1-2 0 9 4
' -2 2-2
I-2
9 9
9
'299
-2 0 9 0
2949
2 0
9-4
2-4 99
92 -2 -2
2 2-2
0222
-2 4 9 9
-2 0-4
2-4 9 9
4
-2 2 -
9 -4
-2 -2
-2 2
2-2 -
9 -4
2 2
0 9-
4 9
0-4
-C
2
2 -
9
-4
9
-C
-2
-2
-4
9
4
3-2 2 2
4 0 9
-2 -2
2
9-4 0-
9 9 -4-
994
-2 -2 2
222
-2 2 2
-2 -2 2
-2-2 2
2 2-2
-2 -2 -2 -I
999;
000;
0909
2221
2-2 -2 i
-2 -2 -2 -9
0-4 9
2 -2
109
!2 2
222
2-2 -2
199
2-2 2
9-4
1-4 9
I90
I-4 0
90
90 4
2-2
-2 -2
-2
-2
9 -4
9 9
90
(b)
91 1 1 1 1 IIIIIIII
0 9
-2-49
4-l 4 9-4 2-2
19-2-19-2 9-9 8
2 2-2
9 11
911 9 19 11 I
s 11
9911
911 1 91999191199
91119119119
I191119
911191191919
99191119
(C)
(4)
PARTIAL INPUT (bi )
1 st ESTIMATE
1 t
1st THRESHOLDING
ITERATION
2nd IERATION
3 rd ITERATION
STABLE
4 th
ITERATION
1
Fig. 2. Numerical example of re-
covery from partial input; N = 20,
M = 4. (a) Stored vectors, (b)
memory or (synaptic) matrix, (c)
results of initializing with a partial
version of b(4.
MASK
FEEDBACK.GAIN,AND
THRESHOLDING
( )
-I,
ONE ROW
OF {
Tat MASK
(I2- I,)
Tij MASK
(b
Fig. 3. Concept for optical implementation
of a content addressable
memory based on the Hopfield model.
(a) Matrix-vector multiplier
incorporating nonlinear
electronic feedback. (b) Scheme for realizing
a binary bipolar
memory mask transmittance in incoherent light.
monolithic structure that
can also be made to contain
all ICs for thresholding,
amplification, and driving of
LEDs.
Optical feedback becomes even more attractive
when we consider that arrays of nonlinear
optical light
amplifiers with internal
feedback
9
or optical bistability
devices (OBDs)'
0
can be used to replace the PD/LED
arrays.
This can lead to simple compact CAM struc-
tures
that may be interconnected
to perform higher-
order
computations than the nearest-neighbor search
performed
by a single CAM.
We have assembled a
simple optical system that is a
variation
of the scheme presented in Fig. 3(a) to simu-
late a
network of N = 32 neurons. The system, details
of which are given in Figs. 5-8, was
constructed with an
array of thirty-two
LEDs and twd multichannel silicon
PD arrays, each consisting of thirty-two
elements.
Twice as many PD elements
as LEDs are needed in
order to implement
a bipolar memory mask transmit-
tance in incoherent light in accordance with
the scheme
of Fig. 3(b). A bipolar binary Tij
mask was prepared
for M = 3 binary state
vectors. The three vectors or
words
chosen, their Hamming distances from each
other,
and the resulting Tij memory matrix are shown
in Fig. 4. The mean Hamming
distance between the
three vectors
is 16. A binary photographic transpar-
ency
of 32 X 64 square pixels was computer generated
from
the Tij matrix by assigning the positive values
in
any given row of Tij to transparent pixels
in one subrow
of the mask and the negative values to
transparent
pixels in the adjacent subrow.
To insure that the image
of the
input LED array is uniformly smeared over
the
memory mask it was found convenient
to split the mask
in two halves, as shown
in Fig. 5, and to use the resulting
submasks
in two identical optical arms as shown in
Fig.
6. The size of the subrows of the memory
submasks was
made exactly equal to the element
size of the PD arrays
in the vertical
direction which were placed in register
15 May
1985 / Vol. 24, No. 10 / APPLIED OPTICS
1471
I
II
I
II
41
i
i
0
I
I
i
Ii'
II
IJ
J
I
I
Stored
words:
Word
l : 1
1 0 0
0 0
1 0 10
1
1 1 0
1 1
0 1 1
1 10
1 1
0 0
0 0 0
1 0
Word
2 :
0 1 1
0 00
0 0
0 01
0 0
1 0
1 0
1 0 0
1 11
1 0
1 0
1 1
0 1 0
Word 3
: 1
0 1 1
0 0 1
1 1
1 1 1
1 1 1
0 0
0 1 0
1 1
0 0 0
0 1 1
0 0
0 0
Hamming
distance
from
word
to word:
WORD
1
2
3
1
0
15
14
2
15
0 19
3
14
19 0
Clipped
memory
matrix:
-1
1 1
1 1 -1
1
1 1 1
-1 1
-1 1
1 1
1 -1
-1 1
-1 1 -1
-1 -1
-1 -1
-1
- 1 -1
-1
-1 1
-1 1
-1 1
1 1 -1
1 1
1 1
1
1 '
-1 1
-1 1
-1
-1 -1
1 -1
1
1 1
1 1 -1
1 -1
-1 1
-1 1
1 -1
1 -1
1 -I
1 -1 -1
1 -1
1
1 1
1 1
1 1
1 -1
1 -1
1 -1
1 -1 -1
-1 -1
-1 -'
-1 1
1 -1
1 -1
1
1
1 -1
1 -1
-1 1
- 1
1 -1
1 1
-1 1 -1
-1
1 -1
1 1
1 -1 1
1 -1
1
0 1
-1 1-1
-1
-1l-1
1-1
1
1
I-1 1-1-1
1
-1 1
-11-1
1
1 - 1
1 0
1 1
1 1
1 1
-1 1
-1 -1
-1 1
-1 -1
-1 -1
-1 -1 -1
1
1 -1
1 -1 1
,-1
1 0 1
1 -1
1 1
1 1
-1 1
-1 1
1 1
1 -1 -1
1 -1
1 -1
-1 -1
-1 -1
1 1
1 0 1
1 1
1 -1
1 -1
-1 -1
1 -1
-1 -1
-1 -1 -1
-1
1 1 -1
1 -1
1
,-1
1 1
1 0
-1 1
1 1
1 -1
1 -1
1 1
1 1 -1
-1 1
-1 1
-1 -1
-1 -1
-1
.-1
1 -1 1
-1 0-1
-11
1-1
-1
1-1
-1 11
1-1-1
1
11-1-1-1
-
,-1 1
1 1
1 -1
0 1
1 1 -1
1 -1
1 1
1 1
-1 -1
1 -1 1
-1 -1
-1 -1
-1
.-1
1 1
1 1 -1
1 0
1 1
-1 1
-1 1
1 1 1
-1 -1
1 -1
1 -1 -1
-1 -1
-1
.-
1 -1
1 -1
1 1
1 1 0
-1 1
-1 -1
1 -1
1 1
-1 1 -1
-1 -1
1 -1
-1 1
-1
1 1
1 1
1 1
1 1 -1
0 --
1 -1
1 -1
-1 -1
-1 -1
-1 -1
1l l-1
1 -1
1
,-1 -1
-1 -1
-1 -1
1 -1
.1 -1
1 -1
1 1
-1 -1
,-1 -1
-1
1 -1
-1 -1
-1
.1 1
1 -1
1 1
1 1
.1 -1
1 -1
1 1
-1 -1
1 1-1
1
1 1
1 1
-1
1 1-1
1 1 1
1 1
-1
.-1 -1
-1 1
-1 -1
-1 -1
.-1
-1 -1
-1 -1
-1 1
-1
.1 -1
1 -1
1 1 -1
-1
.-1 -1
-1 1
-1 -1
-1 -1
1 1
1-1 1
1 -1 -1 1
1
1 -1
-1 -1
-1
-1
-1 -1
-1 1
-1 -1
-1 1
-1
-1
-1 -1
-1 1
0 1
1 -1
1 1
1 1 1
1 1
-1 -1
1-1
1-1
1 0
-1 1
1 -1 -1
-1 1
1 -1 -1
-1 -1
1 1
1
1 -1
0 -1
-1 -1
-1 1
1 -1 1
-1 1
1 1
1 1
-1
1 -1 0
1 1 1
-1 -1 1
-1 1 -1
-1 -1 -1
-1
1 1 -1
1 0 -1
-1 -1
1 1 -1
-1 -1 -1
1 1
1
1 -1 -1
1 -1
0 1 -1
1 -1 -1
-1 1 -1
-1 1
-1
1 -1
-1 1 -1
1 0 -1
1 -1
-1 -1 1
-1 -1
1 -1
1-1 1
-1-1
-1 -1 0
1-1
1 -1 1
1 1
1 1
1
1 1 -1
1 1
1 1 0
1 1 -1
-1 1-1
1-1
1
1 -1
1 1
-1 -1
-1 1
0-1-
-1 -1
1 1
1 1
1-1
1 -1
-1 -1
-1
1 1-1
0-1
1 1
1 1
1
-1
-1 -1
1 -1
-1 -1
-1 -1
-1 -1
0 1
-1 1 -1
1
,-1
-1 1
-1 -1
1 1
1 -1 -1
1
1 0
1 -1 -1
-1
1 -1
1 -1
-1 -1
-1 1
1 -1
1 -1
1 0
1 1
1
,-1
1
1 -1
1 -1 -1
1 -1
1 1
1 -1
1 0 -1
1
1
1 1 -1
1
1 1 1
1 1
1 -1
-1 1
-1 0 -1
,-1 1
1 -1
1 -1 -1
1 -1
1 1
1 -1 1
1 -1
0
Fig.
4. Stored
words,
their
Hamming
distances,
and their
clipped
Tij
memory
matrix.
1-1
-1 1
1 1
1 1 1
1
-1 -1
1 1 -1
-1 1 -1
1 -1
-1
1 -1 -1
1 1 -1
-1 -1 -1
-1 -1
-1 1 1
1 1 -1
1 -1
-1
1 1 -1
-1 -1 -1
-1 -1
-1
-1
-1 -1
1 1 1
1 -1
1 -1
Fig.
5. Two
halves of
Tij memory
mask.
against
the masks.
Light
emerging
from
each
subrow
of a
memory
submask
was
collected
(spatially
inte-
grated)
by
one of
the vertically
oriented
elements
of the
multichannel
PD array.
In this
fashion
the anamorphic
optics
required
in the
output
part
of Fig.
3(a)
are dis-
posed
of,
resulting
in
a more
simple
and
compact
sys-
tem.
Pictorial
views
of
the input
LED
array
and
the
two
submask/PD
array
assemblies
are shown
in Figs.
7(a)
and
(b),
respectively.
In Fig.
7(b)
the
left
memory
submask/PD
array
assembly
is shown
with
the submask
removed
to
reveal
the
silicon
PD
array
situated
behind
it.
All
electronic
circuits
(amplifiers,
thresholding
comparators,
LED
drivers,
etc.)
in
the
thirty-two
par-
allel
feedback
channels
are
contained
in the
electronic
amplification
and
thresholding
box
shown
in
Fig.
6(a)
and
in
the boxes
on which
the LED
array
and
the two
submask/PD
array
assemblies
are mounted
(see
Fig.
7).
A pictorial
view
of a composing
and
display
box is
shown
in
Fig.
8. This
contains
an
arrangement
of thirty-two
switches
and
a thirty-two
element
LED
display
panel
whose
elements
are connected
in parallel
to
the input
LED
array.
The
function
of
this box
is to
compose
and
1472
APPLIED
OPTICS
/
Vol. 24,
No.
10 / 15
May 1985
0 -1
1 1
-1
-1
0 1
-1 -1
1 1
0 -1 -1
1 -1
-1 0
1
-1 -1
-1
1 0
-1
-1 -1
1 1
1 -1
-1 1
1
1 -1
1 1 -1
1 -1
-1 1
1
1 -1 1
1 -1
-1 -1
1 1
-1
1 -1
1 1 -1
1 -1
1 1
-1
1
1 .1
-1 -1
1
-1 -1
1 1
-1
1 1 -1
-1
1 1
-1 -1
1
-1
1 -1
-1 1
1 -1
1 1
-1
1
1 -1 -1
1
1 1 1
-1 -1
1 1
1 -1 -1
-1
1 -1
-1 1
-1 1
1 -1 -1
1 1 -1
-1 1
-1
1 -1
-1
1
-1
-1
1 -1
1 -1
-1
1
1 -1
-1 -1
-1
-1
1 1
-1 -1
1
-1
1 -1
ELECTRONIC
AMPLIFICATION
a
THRESHOLDI
NG
Fig.
8.
Word
composer
and
display
box.
LED
ARRAY
Fig.
6.
Arrangement
for
optical
implementation
of
the
Hopfield
model:
(a)
optoelectronic
circuit
diagram,
(b)
pictorial
view.
a
b
Fig.
7.
Views
of (a)
input
LED
array
and
(b)
memory
submask/PD
array
assemblies.
display
the
binary
input
word
or
vector
that
appears
on
the
input
LED
array
of
the
system
shown
in
Fig.
7(a).
Once
an
input
vector
is
selected
it appears
displayed
on
the
composing
box
and
on
the
input
LED
box
simulta-
neously.
A
single
switch
is
then
thrown
to
release
the
system
into
operation
with
the
composed
vector
as
the
initializing
vector.
The
final
state
of
the
system,
the
output,
appears
after
a
few
iterations
displayed
on
the
input
LED
array
and
the
display
box
simultaneously.
The
above
procedure
provides
for
convenient
exercising
of
the
system
in
order
to
study
its
response
vs
stimulus
behavior.
An
input
vector
is
composed
and
its
Hamming
distance
from
each
of
the
nominal
state
vectors
stored
in
the
memory
is noted.
The
vector
is
then
used
to
initialize
the
CAM
as
described
above
and
the
output
vector
representing
the
final
state
of
the
CAM
appearing,
almost
immediately,
on
the
display
box
is
noted.
The
response
time
of
the
electronic
feedback
channels
as determined
by
the
3-dB
roll-off
of
the
am-
plifiers
was
-60
msec.
Speed
of
operation
was
not
an
issue
in
this
study,
and
thus
low
response
time
was
chosen
to
facilitate
the
experiment.
IV.
Results
The
results
of
exercising
and
evaluating
the
perfor-
mance
of
the
system
we
described
in
the
preceding.
section
are
tabulated
in
Table
I.
The
first
run
of
ini-
tializing
vectors
used
in exercising
the
system
were
error
laden
versions
of
the
first
word
b(').
These
were
ob-
tained
from
b
1
) by
successively
altering
(switching)
the
states
of
1,2,3
...
up
to
N
of its
digits
starting
from
the
Nth
digit.
In
doing
so
the
Hamming
distance
between
the
initializing
vector
and
bW')
is
increased
linearly
in
unit
steps
as
shown
in
the
first
column
of
Table
I
whereas,
on
the
average,
the
Hamming
distance
be-
tween
all
these
initializing
vectors
and
the
other
two
state
vectors
remained
approximately
the
same,
about
N/2
=
16.
The
final
states
of
the
memory,
i.e.,
the
steady-state
vectors
displayed
at
the
output
of
the
system
(the
composing
and
display
box)
when
the
memory
is
prompted
by
the
initializing
vectors,
are
listed
in
column
2
of
Table
I.
When
the
Hamming
distance
of the
initializing
vector
from
bW')
is
<11,
the
input
is always
recognized
correctly
as b(l).
The
CAM
is
able
therefore
to
recognize
the
input
vector
as
by)
even
when
up
to
11
of
its
digits
(37.5%)
are
wrong.
This
performance
is identical
to
the
results
obtained
with
a
digital
simulation
shown
in
parenthesis
in column
2 for
comparison.
When
the
Hamming
distance
is
increased
further
to
values
lying
between
12
and
22,
the
CAM
is
confused
and
identifies
erroneously
other
state
vectors,
mostly
b(3),
as
the
input.
In
this
range,
the
Hamming
distance
of
the
initializing
vectors
from
any
of
the
stored
vectors
is approximately
equal
making
it
more
difficult
for
the
CAM
to
decide.
Note
that
the
performance
of
15
May
1985
/
Vol.
24,
No.
10
/
APPLIED
OPTICS
1473