COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 25,
331-347 (1984)
HIPS? A Unix*-Based image Processing System
MICHAEL S. LANDY, YOAV COHEN, AND GEORGE SPERLING
Human Information Processing Laboratoty, Psychology Department, New York University
Received January 13,1983; revised February 25,1983
A software system for image processing, HIPS, was developed for use in a UNIX environ-
ment. It includes a small set of subroutines which primarily deals with a standardized
descriptive image sequence header, and an ever-growing library of image transformation tools
in the form of UNIX “filters.” Programs have been developed for simple image transforma-
tions, filtering, convolution, Fourier and other transform processing, edge detection and line
drawing manipulation, simulation of digital compression and transmission, noise generation,
and image statistics computation. The system has the useful feature that images are self-docu-
menting to the extent that each image as stored in the system includes a history of the
transformations that have been applied to that image. Although it has been used primarily with
a Grinnell image processor, the bulk of the system is machine-independent. The system has
proven itself a highly flexible system, both as an interactive research tool, and for more
production-oriented tasks. It is both easy to use, and quickly adapted and extended to new
uses.
1. INTRODUCTION
There is currently rapid growth in the number of research labs involved in image
processing. In setting up such a facility, there is a seemingly inevitable period of time
during which hardware and software tools are developed in order to provide a
convenient and flexible research environment. In setting up our lab, we decided to
give some thought to the software environment before we embarked on our various
research projects. Our lab includes a variety of image-related tasks at any given time,
many of which are carried out by neophytes in computerized image processing. Our
problem was thus to provide an environment in which later development of software
image processing tools would be easy, and yet provide general flexible tools which
were easy to use. A wide variety of image processing related projects are now
ongoing, and using these tools we feel that the system has proven its flexibility.
The original project which prompted the construction of this image processing
facility was research into the bandwidth requirements for comprehensible yet
maximally compressed transmission of imagery of American Sign Language (ASL)
[l-3]. (ASL is a gesture language used for communication with and among the deaf.)
This research involves both an investigation of the relative importance of the various
visual features of ASL imagery, and a measurement of the ability of speakers of ASL
to comprehend imagery which has been modified along various parameters such as
pixel density, gray scale, spatial frequency content, applied image transformations,
and so on. Thus, a computing environment was needed in which sequences of images
*UNIX is a trademark of Bell Laboratories.
+HIPS stands for the Human Information Processing Laboratory’s Image Processing System.
331
0734-189X/84 $3.00
Copyr@t 0 1984 by Academic Press, Inc.
All rights of reproduction in any form reserved.
332
LANDY, COHEN, AND SPERLING
could be input into the computer, transformed in various ways in the spatial and
frequency domains, and eventually be presented to experimental subjects in order to
test for comprehensibility.
Once the development of this system was made known to other researchers and
computer users, it became apparent that any software tools might also be used by
several other local projects in the visual sciences. This made the need for a highly
flexible and yet easy-to-use system all the more pressing. We therefore decided to
expend some extra initial effort in order to provide such a working environment, and
we feel that the effort has been repaid handsomely.
2. ENVIRONMENT
The computing environment consists of a VAX 11/750 computer running the
Berkeley 4.1 version of the UNIX operating system. This is an operating system
designed for generality and portability, with very little hardware-specific code, and
thus provided impetus for the design of a software system which is equally portable
and non-device specific. The main image processing device is a Grirmell GMR 27-30
image processor (connected to the VAX via a DRll-B DMA interface), which
possesses the capability of storing and digitizing video frames into 256 gray levels
with a resolution of 480
x
512. It can also convert digital frames back to video, and
has a joystick, cursor, alphanumerics, etc. Other peripherals include dot matrix
printers (used for half-tone representation of images), film and video cassette
equipment, etc. Lastly, a slow-speed parallel interface (a DRll-C) is used to control
film and video equipment, and to allow the system to synchronize its image output
with the Grirmell’s vertical retrace.
The system we have developed is closely tied with the UNIX operating environ-
ment [4], and so a brief introduction to UNIX concepts is in order. The two most
important ideas here are those of
afilter
and a
pipe.
Standard utility programs in the
UNIX system are generally written as fairly simple transformations. The command
language is such that every program that is executed has an associated
standard input
and
standard output.
Filters are programs that provide an easily described transform
of their standard input as their standard output. For example, a sorting routine
might take a list of unsorted items as its input, and yield a sorted set as its output.
The sorting parameters (such as type of ordering, keys, etc.) can be provided along
with the routine name when the command is entered. Thus, like an ordinary
electronic filter, a UNIX filter is merely a transformation of input to output. The
UNIX conventions for typing a command with parameters are
transform parameter, . . . parameter, < input > output.
Electronic filters can be cascaded in order to form more complex filtering
operations from simple primitive filters. This is also the case with UNIX filters. The
operation that provides this cascading is called the
pipe
facility, and in the command
language is symbolized as 1. Using a pipe-line allows one to automatically plug the
standard output of one filter into the input of the next. For example, the line
transform, ( pipe-input 1 transform, ] . . . ] transform, > pipe-output
applies
transform,
to
pipe-input,
sends its output to
transfotq,
and so on, until
UNIX-BASED IMAGE PROCESSING
333
finally transform,‘s output is saved in the file pipe-output. The structure of filters and
pipes is used extensively in our image processing system.
3. SYSTEM STRUCTURE
General features. Taking our cue from the UNIX pipe and filter structure, we chose
to program our image processing routines as filters. Thus, each program would
perform only one type of transformation, and complex transformations would be
built up, whenever possible, as pipelines of simpler transformations. This sort of
implementation is natural given the UNIX operating system. The use of pipelines in
image processing systems was demonstrated by Stevens and Hunt [5] as a means of
greater efficiency given that the use of temporary files for the storage of intermediate
results is avoided.
When applying a filter to a sequence of frames, one has to specify a number of
parameters. Generally, the parameters can be divided into two groups: those that
describe the image sequence itself, and those that apply to the operation of the filter.
For example, any filter needs to know the number of frames in its input sequence,
but only the reduce filter needs to know how much to reduce an image. Thus, in
order to ease our programming efforts, the image parameters were gathered together
in a standardized image header which was then made an integral part of each image
sequence as stored in the computer.
The image header. The use of standardized image headers is by no means new to
image processing. It was initially our intent to use an already defined header in our
system in order for our images to be portable and easily transferred to other labs,
but we found that other such headers were inadequate for our purposes. One such
header is the NATO header [6], which is used for picture transmission by ARPA
facilities, among others. Although it gives full description of pictures in pixel matrix
format, it is rigid in that graphic data in other formats, such as vector representa-
tions, cannot be represented. The NATO header also allows only fixed space for
image sequence documentation, and is primarily intended as a format for magnetic
tape storage of imagery. Other headers which are more geared for parameter passing
include an extension of the NATO header called CVL [7], and the header used in the
EIDES system [8].
Our design departs from this approach in allowing variable length textual descrip-
tion of images, and allows more freedom in the documentation of picture sequences.
Being of variable size, the header can be expanded with no limit. The rationale
behind this approach is that in a research environment (as opposed to a production
environment in which large volumes of pictures are processed), considerations of
execution efficiency and data integrity are secondary to considerations of modular-
ity, simple and effective documentation of image files, and easy software generation
and maintenance.
Our image header scheme provides two facilities: the automatic passing of image
parameters through a transformation pipeline, and the ability to automatically
document an image sequence integral to how the sequence is stored and manipu-
lated. The image parameters in the header include:
number of frames,
number of rows,
number of columns,
334
LANDY, COHEN, AND SPERLING
bits per pixel,
bit packing (if filler is used to fill out bytes, or not),
pixel format (byte, integer, floating point, complex, vector plot, histogram,
quad-tree and hierarchical encoding,. . .).
The descriptive information includes:
originator’s name,
sequence name,
sequence date,
sequence description (a text of arbitrary length),
sequence history.
The originator’s name, sequence name, and sequence date are all text fields of
arbitrary length which can be used to describe the imagery from which this data was
originally derived. The sequence description is also in free text format, and com-
ments may be added to this description at any point. The sequence history is an
interesting and useful feature. It contains an executable command string of all
transformations that have been applied to this sequence, including the date and time
they were run. For example, the following sequence history describes an image which
was read in from the Grinnell, reduced by a factor of 4, cropped to 96
X
64 pixels,
converted to floating point format, then linearly scaled back to byte format such that
the lowest floating pixel maps to 0, and the highest to 255 (which is a cheap way of
contrast enhancement):
rframe “-D Thu Mar 18 16:17:46 1982” 1
reduce 4 “-D Tue May 4 11:19:27 1982” 1
extract 96 64 11 19 “-D Fri Sep 17 16:lO:lO 1982” 1
btof “-D Fri Sep 17 17:01:59 1982” 1
scale “-D Fri Sep 17 17:02:03 1982”
(this string is actually an executable UNIX command string). The quoted parameters
which begin with
-D
are dummy parameters which describe when the transformation
was applied to this image, and are ignored by all programs. The \ at the end of each
line is the UNIX method of saying that a command continues on the next input line.
Thus, if the original image was still in the Grinnell’s image memory, the image
described by this history could be regenerated by executing the history string as a
UNIX command.
The image header can be modified in two ways. After an image filter has read an
image header, it can simply assign new values to header parameters, or append text
to the descriptive information. Also, an image filter exists, called adddesc, which
allows one to update any or all of the documentary fields in the image header. For
UNIX-BASED IMAGE PROCESSING
335
example, the command
adddesc - a “new descriptive information” -C input-sequence > output-sequence
will add the string “new descriptive information” to the sequence description.
A small library has been written which includes routines that manipulate image
headers. Routines exist which read an image header from the standard input, update
the description, update the history, and write the new header to the standard output.
/=
* logimg - takes log of Input Image.
* Input image Is In byte format. output fmage is floating point,
* usage: logimg cseq >oseq
* to load: cc -0 logimg logimgc -Ihipl -Im
‘/
#Include cstdlo.hb
#Include chipI-format.hb
#Include cmath.hl
float logtabf2561:
malntargcargv)
char
‘argvtl:
int factor.fr.f,r.c.b.i.j.k:
char
=ptc.*p:
float val:
struct header hd:
read-header f8hd):
iffhd.pixel-format != PFBYTE) f
fprlntffstderr.‘loglmg: pixel format must be byte\n”):
exhfl):
1
r = hd.rows:
c = hdcols:
hd.pixel-format = PFFLOAT:
update-headert&hO.argc.argv):
write-headerfbhd):
if ffpic = (char *) caIIocfr*c.sizeoffcharH~ == 0) I
fprlntffstderr.‘loglmg: can’t allocate core\n”);
exit(l):
)
lor d=O:i~256:i++)
logtabfll = logffdouble) ti+l)):
for ff=O:fchd.num-frame:f++) f
If fpreadtO,plc.r*c*slzeoftchar)) != r*c*sizeoffchar)) f
fprlntffstderr.“logimg: error during read\n’):
exit(l);
p = pit:
for fi=O:icr:i++) f
for f)=O:jcc:j++) f
val = logtabf*p++ 8 03771;
writerl.8val.4~:
return(O):
FIG.
1. Logimg program