University of Pennsylvania University of Pennsylvania
ScholarlyCommons ScholarlyCommons
Departmental Papers (BE) Department of Bioengineering
February 2003
A biomorphic digital image sensor A biomorphic digital image sensor
Eugenio Culurciello
Johns Hopkins University
Ralph Etienne-Cummings
Johns Hopkins University
Kwabena A. Boahen
University of Pennsylvania
, boahen@seas.upenn.edu
Follow this and additional works at: https://repository.upenn.edu/be_papers
Recommended Citation Recommended Citation
Culurciello, E., Etienne-Cummings, R., & Boahen, K. A. (2003). A biomorphic digital image sensor. Retrieved
from https://repository.upenn.edu/be_papers/20
Copyright 2003 IEEE. Reprinted from
IEEE Journal of Solid-State Circuits
, Volume 38, Issue 2, February 2003, pages
281-294.
Publisher URL: http://ieeexplore.ieee.org/xpl/tocresult.jsp?isNumber=26391&puNumber=4
This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply
IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or personal use of this
material is permitted. However, permission to reprint/republish this material for advertising or promotional
purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing
to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws
protecting it.
This paper is posted at ScholarlyCommons. https://repository.upenn.edu/be_papers/20
For more information, please contact repository@pobox.upenn.edu.
A biomorphic digital image sensor A biomorphic digital image sensor
Abstract Abstract
An arbitrated address-event imager has been designed and fabricated in a 0.6-μm CMOS process. The
imager is composed of 80 x 60 pixels of 32 x 30 μm. The value of the light intensity collected by each
photosensitive element is inversely proportional to the pixel’s interspike time interval. The readout of each
spike is initiated by the individual pixel; therefore, the available output bandwidth is allocated according to
pixel output demand. This encoding of light intensities favors brighter pixels, equalizes the number of
integrated photons across light intensity, and minimizes power consumption. Tests conducted on the
imager showed a large output dynamic range of 180 dB (under bright local illumination) for an individual
pixel. The array, on the other hand, produced a dynamic range of 120 dB (under uniform bright illumination
and when no lower bound was placed on the update rate per pixel). The dynamic range is 48.9 dB value at
30-pixel updates/s. Power consumption is 3.4 mW in uniform indoor light and a mean event rate of 200
kHz, which updates each pixel 41.6 times per second. The imager is capable of updating each pixel 8.3K
times per second (under bright local illumination).
Keywords Keywords
arbitrated, address event, digital image sensor, high dynamic range, low-power imager
Comments Comments
Copyright 2003 IEEE. Reprinted from
IEEE Journal of Solid-State Circuits
, Volume 38, Issue 2, February
2003, pages 281-294.
Publisher URL: http://ieeexplore.ieee.org/xpl/tocresult.jsp?isNumber=26391&puNumber=4
This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way
imply IEEE endorsement of any of the University of Pennsylvania's products or services. Internal or
personal use of this material is permitted. However, permission to reprint/republish this material for
advertising or promotional purposes or for creating new collective works for resale or redistribution must
be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document,
you agree to all provisions of the copyright laws protecting it.
This journal article is available at ScholarlyCommons: https://repository.upenn.edu/be_papers/20
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 38, NO. 2, FEBRUARY 2003 281
A Biomorphic Digital Image Sensor
Eugenio Culurciello, Ralph Etienne-Cummings, and Kwabena A. Boahen
Abstract—An arbitrated address-event imager has been
designed and fabricated in a 0.6-
m CMOS process. The imager
is composed of 80
60 pixels of 32 30 m. The value of the
light intensity collected by each photosensitive element is inversely
proportional to the pixel’s interspike time interval. The readout
of each spike is initiated by the individual pixel; therefore, the
available output bandwidth is allocated according to pixel output
demand. This encoding of light intensities favors brighter pixels,
equalizes the number of integrated photons across light intensity,
and minimizes power consumption. Tests conducted on the imager
showed a large output dynamic range of 180 dB (under bright
local illumination) for an individual pixel. The array, on the other
hand, produced a dynamic range of 120 dB (under uniform bright
illumination and when no lower bound was placed on the update
rate per pixel). The dynamic range is 48.9 dB value at 30-pixel
updates/s. Power consumption is 3.4 mW in uniform indoor light
and a mean event rate of 200 kHz, which updates each pixel
41.6 times per second. The imager is capable of updating each
pixel 8.3K times per second (under bright local illumination).
Index Terms—Arbitrated, address event, digital image sensor,
high dynamic range, low-power imager.
I. INTRODUCTION
C
ONVENTIONAL cameras produce images by scanning
the photosensitive pixels in a sequential (raster) format,
functionally dividing the output bandwidth equally among
all pixels. The sequential scan requires that signal processing
performed on the video stream be completed within one pixel
readout time. This requirement can be difficult to fulfill for
large (
256 256) or fast ( 100 frames per second) imaging
arrays. To circumvent this sequential bottleneck, in the late
1980s researchers demonstrated a new imaging paradigm that
mimicked the human retina with silicon integrated circuits [1].
The main advantage of the silicon retina was its highly parallel
computational nature, which allowed high-speed pixel-parallel
image processing at the focal plane. Mahowald and Mead’s
silicon retina provided the first glimpse of the great potential
of CMOS integrated circuits technology for imaging [1]. This
potential, however, has still not been fully realized today. It
should be noted that CMOS imagers designed as substitutes for
charge-coupled device (CCD) imagers have made significant
Manuscript received January 22, 2002; revised August 15, 2002. The
work of E. Culurciello was supported by the Defense Advanced Research
Projects Agency under DARPA/ONR MURI N0014-95-1-0409. The work
of R. Etienne-Cummings was supported by the National Science Foundation
under CAREER Award 9896362. The work of K. A. Boahen was supported by
a Whitaker Foundation Research Initiation Award.
E. Culurciello and R. Etienne-Cummings are with the Department of Elec-
trical and Computer Engineering, The Johns Hopkins University, Baltimore,
MD 21218 USA (e-mail: euge@jhu.edu).
K. A. Boahen is with the Department of Bioengineering, University of Penn-
sylvania, Philadelphia, PA 19104 USA.
Digital Object Identifier 10.1109/JSSC.2002.807412
inroads into the commercial marketplace, yet the focal plane
image-processing capabilities of the technology has not been
fully exploited [2]. The early silicon retinas were doomed as an
alternative imaging approach because the CMOS technology
in the early 1990s was not mature enough to compete with
the quality of CCD imagers. This is especially true when
considering that the noise introduced by the photo detector,
amplification circuits, and image processing (edge and motion
detection) circuits are significantly higher than CCD imagers,
although the latter do not provide any processing on the image
plane. Furthermore, the silicon retina pixels were too large to
realize high-resolution arrays at a reasonable yield per cost.
Consequently, the idea of a silicon retina as a commercially
viable imager was abandoned. Recently, the silicon retina
concept has been resurrected because three-dimensional (3-D)
integration techniques promise small footprints with pixel-par-
allel spatiotemporal image processing [3], [4]. However, we are
still far from a commercial product in these technologies. The
research on biologically inspired imagers and image processing
chips in standard CMOS processes have continued over the
past ten years [5]–[7]. The imager presented here continues the
trend of “reverse engineering biology,” where the outcome is
a silicon retina with focal-plane image processing/encoding,
small pixel sizes, extremely high dynamic range, relatively low
power consumption, and “photon-to-bits” phototransduction.
Conventional imagers integrate the photocurrent for a fixed
time, usually dictated by the scanning period. Subsequently, the
integrated voltage is output according to a raster scan. Here,
we invert the process by integrating the photocurrent to a fixed
voltage (threshold). When the threshold is crossed, a 1-b pulse
(spike)isgeneratedby the pixel.The magnitude of the photocur-
rent is represented as the interspike interval between two suc-
cessive spikes. This interspike interval is inversely proportional
to the intensity. Our system is also different from conventional
methods because the readout of each spike is initiated by the
pixel itself. That is, each pixel requests access to the output bus
when the integration threshold has been crossed [8].
This biologically inspired readout method simultaneously
favors brighter pixels, minimizes power consumption by
remaining dormant until data is available, and offers pixel-par-
allel readout. In contrast, a serially scanned array allocates an
equal portion of the bandwidth to all pixels independent of
activity and continuously dissipates power because the scanner
is always active. Here, brighter pixels are favored because
their integration threshold is reached faster than darker pixels,
i.e., the request–acknowledge–reset–integrate cycle operates
at a higher frequency. Consequently, brighter pixels request
the output bus more often than darker ones. Also, virtually no
power is used by the pixel until an event is generated; there-
fore, low-intensity pixels consume little power. Furthermore,
0018-9200/03$17.00 © 2003 IEEE
282 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 38, NO. 2, FEBRUARY 2003
representing intensity in the temporal domain allows each pixel
to represent a large dynamic range of outputs [11], [12]. The
integration time is, in fact, not dictated by a regular scanning
clock and, therefore, a pixel can use the whole bus bandwidth
by itself or can abstain from the image forming process. This
provides a simple and efficient way of obtaining dynamic range
control, without the use of additional circuitry that varies the
integration time of each pixel based on the light intensity [13].
Pixel-parallel automatic gain control is an inherent property of
our time-domain imaging and readout scheme, which is called
address-event representation (AER) [8]–[10], [14].
We will describe the AER architecture in Section II, the event
or spike generation circuits in Section III, the spike communica-
tion circuits in Section IV, the imager operation and its analysis
in Section V, and present results and discussion in Section VI
and the conclusion in Section VII.
II. AER
The imager uses AER output format. The address-event (AE)
communication channel is a model of the transmission of neural
information in biological systems [14]. Information is presented
at the output in the form of a sequence of pulses or spikes, where
the interspike interval or the spike frequency encodes the analog
value of the data being communicated. Encoding the data as a
stream of digital pulses provides noise immunity by quantiza-
tion and redundancy. The frequency-modulated signal can be
reconstructed by integration or simply by counting the number
of received events over a predetermined window of time. The
imager presented here mimics the octopus’ retina by converting
the light intensity directly into a spike train [15]; most other bio-
logical retinas represent light intensity as an analog signal [16],
[17].
The AER model trades the complexity in wiring of the bio-
logical systems for the processing speed of integrated circuits.
Neurons in the human brain make up to 10
connections with
their neighbors [16], [17], a prohibitive number for integrated
circuits. Nevertheless, the latter are capable of handling com-
munication cycles that are six orders of magnitude smaller than
the interevent interval for a single neuron. Thus, it is possible to
share this speed advantage among many cells and create a single
communication channel to convey all the information between
two neural populations. AER uses an asynchronous protocol for
communication between different processing units [8]–[10].
As shown in Fig. 1, the information, divided into “events,”
is sent from a unique sender to a unique element in a receiving
population. Events are generally in the form of a spike; there-
fore, only their address is the important data to reconstruction
and the time of occurrence. The information packet is, therefore,
the address of the spiking cell or transmitter. In the case of our
imager, events are individual pixels reaching a threshold voltage
and requesting the bus for communication with a receiver. As
a result, the system represents light intensity on a pixel as a
frequency-modulated sequence of addresses, where the time in-
terval between identical addresses (pixels) is inversely propor-
tional to the intensity. An AE system is generally composed of
a multitude of basic cells or elements either transmitting, re-
ceiving, or transceiving data. Reconstruction of data necessi-
Fig. 1. AE system: A general-purpose protocol for the transmission of data
from an array of senders to an array of receivers.
tates storage, since events must be counted or accumulated to
reassume the form of intensity signals.
A few frequency-modulated and/or AEimaging systems have
been previously reported, however, the one presented here is the
first to combine a conventional active pixel sensor (APS) with a
fully arbitrated AE system, to provide a high-resolution image
with one of the best quality reported [2], [11], [12], [19], [20].
III. E
VENT GENERATION
The key element in an address event imager is the spike
generator circuit. This element, generally incorporated in the
pixel cell, is responsible for requesting access to the output bus
when a pixel has reached the integration threshold. Generally,
a prototypical CMOS imager employs a photodiode as a
photosensitive element. The relatively small photocurrent is
integrated on a capacitor and subsequently read out. An AE
imager will convert light into events by integrating photocurrent
up to a fixed threshold. The integrated voltage changes very
slowly if the light intensity is low. The event generator must
convert this slow-changing voltage into a fast-changing signal
in order to minimize the delay between the time when the
threshold is passed and when the output bus access is requested.
Furthermore, the fast transition also limits power consumption.
Hence, the event generator is an important component of the
AER imager and will be described in detail. After the pixel’s
request has been acknowledged, the pixel is reset and all accu-
mulated charges on the integration capacitor are drained. The
integration process is then immediately restarted. Notice that a
natural ordering of the pixels’ readout occurs that minimizes
pixel request collisions. Collisions translate into temporal jitter,
which degrade the image quality. Jitter due to arbitration will
also be discussed in Section V-C.
A. Simple Inverter as Event Generator
The simplest event generator is a solitary inverter. The high
inversion gain of a CMOS inverter is an immediate solution for
implementing a threshold circuit with a binary output. Its gain
is capable of amplifying the tiny slew rate of the input signal.
On the other hand, its power consumption is proportional to the
switching time, which, in turn, is proportional to the input signal
slew rate.
CULURCIELLO et al.: BIOMORPHIC DIGITAL IMAGE SENSOR 283
Fig. 2. Capacitive feedback in integrate and fire neurons.
In ambient lighting, the photosensor input slew rate is six
orders of magnitude slower than typical digital signals (or
1 V/ms). This means that the input voltage remains in the
high power consumption region of the inverter for a long time,
creating a direct current path between the supplies. A simple
inverter used as an event generator, in a 0.5-
m process and
3.3-V supply, consumes about 3.9 nJ (15
W 0.26 ms).
A typical digital inverter using minimum size transistors, in
a 0.5-
m process and 3.3-V supply, consumes only about
0.06 pJ (40
W 3ns 0.5) per off-transition (rising input,
falling output) and about 0.18 pJ (120
W 3ns 0.5) per
on-transition (falling input, rising output). Therefore, the power
consumption of the inverter as the event generator is about four
to five orders of magnitude greater than that of a minimum-size
inverter in a digital circuit. Clearly, a simple inverter is not a
good candidate as an event generator for low-power imaging
applications. To limit power consumption, a starved inverter
can be used, where the output current is limited by a current
source to a few nanoamperes. However, there is a severe impact
on switching speed when this approach is taken, as will be
evident in Section III-D.
B. Capacitive-Feedback Inverters as Event Generator
In order to decrease the power consumption of the event gen-
erator, it is necessary to increase its gain, at least in the vicinity
of the threshold. A voltage feedback circuit employing capaci-
tive feedback can speed up the transition and, therefore, limit
the time spent in the high power consumption region (Fig. 2).
The capacitive feedback multiplies the inverter ac gain by the
feedback ratio
[23].
A further improvement is obtained by operating the capaci-
tive feedback inverters with the MOSFETs in weak inversion.
This improves power consumption significantly in ambient
light conditions of 1 W/m
. The second inverter uses about
7
A for only 7 ns to generate an output spike, but the first
inverter remains for 4
s in the high power consumption region
because of the slow rising input. The pixel readout rate is,
however, severely reduced when the event generator operates
in subthreshold. While we receive some power consumption
benefits from the capacitive-feedback circuit, those benefits are
shadowed by the increased size (a large feedback capacitor is
required) and lower readout rate of the pixel.
C. Current-Feedback Event Generator
The event generator used in the imager solves both the
transition speed and power consumption problems with an
Fig. 3. Current-feedback event generator pixel.
elegant current positive feedback circuit. Power consumption
and transition speed are closely related because CMOS digital
circuits only consume power during switching. Hence, reducing
the transition time will also reduce the power consumption.
Our event generator has simultaneously a large gain, large
bandwidth, and minute power consumption. This circuit can be
used for various other applications where high speed and low
power consumption are required. Fig. 3 shows the schematic
of the pixel and the event generator. Photons collected by an
n-type photodiode are integrated on a 0.1-pF capacitor to give
a slew rate of 0.1 V/ms in typical indoor light (0.1 mW/cm
).
In dimmer conditions, the input slew rate can be much lower.
Event generation occurs as follows. Initially, the inverter
input voltage
is high (after the reset pulse). Transistor
is off and so is the feedback switch . In addition, the inverter
output voltage
is low. As the capacitor is discharged
by the photocurrent,
decreases and transistor begins
conducting. Slightly before
reaches the threshold of ,
a subthreshold current flows through the inverter and is fed
back to the input, through transistors
– . Notice that
starts to rise before the feedback circuit is activated, which
subsequently switches
on and starts the current feedback.
The mirror pair
– is sized for current gain. The feedback
current mirror operates in subthreshold initially, but increases
exponentially as
decreases further. We approximate the
start of the switching process as the value of
where the
fed-back current equals and surpasses the photocurrent. At
this point, the
accelerates toward ground, accelerates
toward
, and the switch transistor turns off, which
disconnects the integration capacitor from
and causes
to accelerate further. Furthermore, as plunges below the
threshold voltage of
, it shuts off the feedback mirror, which
cuts off the current in the
– branch and causes to
accelerate further toward
. As can be seen, the transition
takes place just before the threshold voltage of
is reached.
The capacitance at the
node is suddenly decreased, and
and cut off for a low-current yet high-speed circuit.
This circuit is unique in this respect. Fig. 4 shows a SPICE
simulation of the circuit operation. The upper traces plot the
input and output voltage versus time. Note first the slow rise
in the voltage, due to the photocurrent, then the sudden switch
