Showing papers in "IEEE Transactions on Information Theory in 1962"
TL;DR: It is shown that recognition of geometrical patterns and alphabetical characters independently of position, size and orientation can be accomplished and it is indicated that generalization is possible to include invariance with parallel projection.
Abstract: In this paper a theory of two-dimensional moment invariants for planar geometric figures is presented. A fundamental theorem is established to relate such moment invariants to the well-known algebraic invariants. Complete systems of moment invariants under translation, similitude and orthogonal transformations are derived. Some moment invariants under general two-dimensional linear transformations are also included. Both theoretical formulation and practical models of visual pattern recognition based upon these moment invariants are discussed. A simple simulation program together with its performance are also presented. It is shown that recognition of geometrical patterns and alphabetical characters independently of position, size and orientation can be accomplished. It is also indicated that generalization is possible to include invariance with parallel projection.
7,963 citations
TL;DR: The condition for discrimination was found to be based primarily on clusters or lines formed by proximate points of uniform brightness, and a similar rule of connectivity with hue replacing brightness was obtained by using varicolored dots of equal subjective brightness.
Abstract: Visual discrimination experiments were conducted using unfamiliar displays generated by a digital computer. The displays contained two side-by-side fields with different statistical, topological or heuristic properties. Discrimination was defined as that spontaneous visual process which gives the immediate impression of two distinct fields. The condition for such discrimination was found to be based primarily on clusters or lines formed by proximate points of uniform brightness. A similar rule of connectivity with hue replacing brightness was obtained by using varicolored dots of equal subjective brightness. The limitations in discriminating complex line structures were also investigated.
931 citations
TL;DR: By refining Hamming's geometric sphere-packing model a new upper bound for nonsystematic binary error-correcting codes is found and improves on Wax's bounds in all but four of the cases he lists.
Abstract: By refining Hamming's geometric sphere-packing model a new upper bound for nonsystematic binary error-correcting codes is found. Only combinatorial arguments are used. Whereas Hamming's upper bound estimate for e -error-correcting codes involved a count of all points \leq e Hamming distance from the set of code points, the model is extended here to include consideration of points which are >e distance away from the code set. The percentage improvement from Hamming's bounds is sometimes quite sizable for cases of two or more errors to be corrected. The new bound improves on Wax's bounds in all but four of the cases he lists.
474 citations
431 citations
TL;DR: A general theorem is provided for the moments of a complex Gaussian video process that states that an n th order central product moment is zero if n is odd and is equal to a sum of products of covariances when n is even.
Abstract: A general theorem is provided for the moments of a complex Gaussian video process. This theorem is analogous to the well-known property of the multivariate normal distribution for real variables, which states that an n th order central product moment is zero if n is odd and is equal to a sum of products of covariances when n is even.
411 citations
TL;DR: A class of decoding algorithms using encoding-and-comparison is considered for error-correcting code spaces and it is suggested on operational grounds that it may prove most useful in those cases where m is relatively large with respect to the code length n.
Abstract: A class of decoding algorithms using encoding-and-comparison is considered for error-correcting code spaces. Code words, each of which agrees on some information set for the code with the word r to be decoded, are constructed and compared with r . An operationally simple algorithm of this type is studied for cyclic code spaces A . Let A have length n , dimension k over some finite field, and minimal Hamming distance m . The construction of fewer than n^2/2 code words is required in decoding a word r . The procedure seems to be most efficient for small minimal distance m , but somewhat paradoxically it is suggested on operational grounds that it may prove most useful in those cases where m is relatively large with respect to the code length n .
396 citations
TL;DR: By simply adding some noise to the signal before it is quantized and subtracting the same noise at the receiver, the quantization steps can be broken up and the source rate reduced to three bits per sample.
Abstract: In order to transmit television pictures over a digital channel, it is necessary to send a binary code which represents the intensity level at each point in the picture. For good picture quality using standard PCM transmission, at least six bits are required at each sample point, since the eye is very sensitive to the small intensity steps introduced by quantization. However, by simply adding some noise to the signal before it is quantized and subtracting the same noise at the receiver, the quantization steps can be broken up and the source rate reduced to three bits per sample. Pseudo-random number generators can be synchronized at the transmitter and receiver to provide the identical "noise" which makes the process possible. Thus, with the addition of only a small amount of equipment, the efficiency of a PCM channel can be doubled.
384 citations
TL;DR: A speech recognition model is proposed in which the transformation from an input speech signal into a sequence of phonemes is carried out largely through an active or feedback process.
Abstract: A speech recognition model is proposed in which the transformation from an input speech signal into a sequence of phonemes is carried out largely through an active or feedback process. In this process, patterns are generated internally in the analyzer according to an adaptable sequence of instructions until a best match with the input signal is obtained. Details of the process are given, and the areas where further research is needed are indicated.
278 citations
270 citations
TL;DR: The synthesis of radar ambiguity functions is approached by minimizing the integrated square error between an arbitrary desired function and a realizable ambiguity function via an orthonormal signal basis which generates an "induced basis" over the time-frequency plane consisting of all pair-wise cross-ambiguity functions of the signal basis.
Abstract: The synthesis of radar ambiguity functions is approached by minimizing the integrated square error between an arbitrary desired function and a realizable ambiguity function. The approximation is carried out via an orthonormal signal basis which generates an "induced basis" over the time-frequency plane consisting of all pair-wise cross-ambiguity functions of the signal basis. The minimum mean-square error and the corresponding signal are determined through an eigenvalue problem for 1) approximation by complex autoambiguity function and 2) approximation by complex cross-ambiguity function. A new form of the realizability theorem shows that the conditions for cross-ambiguity functions differ from those for autoambiguity functions only in the absence of the symmetry condition: F(\tau, \omega) = F^{\ast}(-\tau, -\omega) . Moreover, the approximations by cross- and autoambiguity functions coincide whenever the desired function has the above symmetry. When an ambiguity function realizable by a known signal is to be approximated on a finite basis, least-square approximation in signal space or in ambiguity-function space leads to equivalent results. The relation between the mean-square errors in the two spaces is obtained. For the phase incoherent radar case an iteration procedure is presented for successively modifying the arbitrary phase assigned to the desired function. Convergence is proved in the sense that the error is nonincreasing at each stage of the iteration, but arrival at the best approximation to the desired magnitude is not guaranteed. As an aid in numerical applications, a formulation based on a discrete sample grid in the time-frequency plane is derived. With the appropriate grid dimensions, the analytic procedures carry over directly into sampled-data representation.
197 citations
TL;DR: For a wide class of signals it is shown that jitter does not affect the nature of the optimal operations; linear operations, for instance, remain linear, although with different weights.
Abstract: There are many communication as well as control systems--in fact, an increasing number of them--in which at some stage a continuous data source is sampled "periodically," at the nominal rate of 2 W samples a second, W being the highest frequency component of the data. There are, generally speaking, two kinds of errors introduced by the sampling mechanism: errors in amplitude and errors in timing, or "time jitter." This paper is concerned with the latter. We assume a random model for the jitter. We begin with a study of the properties of the jittered samples for both deterministic and stochastic signals. Depending on the stochastic properties of the jitter, the presence of a discrete component in the signal may give rise to new discrete components as a result of jitter. Generally speaking, however, the effect of jitter is to produce a (frequency) selective attenuation as well as a uniform spectral density component. The more correlation in the jitter, the less the spectral distribution is affected. Various measures of the "error" due to jitter are estimated. Thus the error may be the mean square in the fitted samples or some linear or nonlinear operation thereof. Also, weighted mean-square errors are considered, and general methods of estimating these errors are developed. The problem of optimal use of the jittered samples is next considered. Interpreting the optimality to be in the mean-square sense, an explicit solution for optimal linear operation is obtained. Also for a wide class of signals it is shown that jitter does not affect the nature of the optimal operations; linear operations, for instance, remain linear, although with different weights. To illustrate the methods an example drawn from telemetry is given, where the timing is derived from the zero crossings of a sine wave and the time jitter is taken as due to noise. The jitter is highly correlated and the results involve some lengthy calculations.
TL;DR: This paper examines the notion of a single number statistic for each characteristic which would have certain desirable properties related to the "goodness" of the characteristic, and shows that, in general, no such statistic exists.
Abstract: Design of pattern recognition systems usually involves a number of uncertainties which can be resolved only by experiment. In statistical recognition systems, pattern information is stored in the machine in the form of pattern characteristics with statistics relating the characteristics to the patterns. If the number of characteristics a system can process is limited, and if the system designer, while able to conceive of a large number of relevant characteristics, does not know which are the most important, then an experimental selection of these characteristics is required. As tools, the designer may have at his disposal a large computer, a large sample of the patterns to be recognized, and a set of programs to measure the characteristics. Thus, he can compare certain statistics relating the patterns to the characteristics. The problem is: what statistics should he calculate in order to select the best characteristics? Assuming the characteristics to be independent in their effect on the decision, this paper examines the notion of a single number statistic for each characteristic which would have certain desirable properties related to the "goodness" of the characteristic. It is shown that, in general, no such statistic exists. However, a statistic is proposed which, while not having these properties in general, at least has them in a wide range of situations. An experimental study of the validity of this choice is reported together with the design of a letter recognition system. Using a sample of 15 complete 62 symbol alphabets, 13 characteristics were selected. The resulting system recognized correctly 81.9% of the letters presented to it.
TL;DR: It appears that the product of the maximum time- and frequency-spreadings produced by the time-variant filter sets a limit on the ability to determine the instantaneous values unambiguously even in the absence of additive noise.
Abstract: We discuss the problems of making detailed measurements of instantaneous values and the statistical parameters of time-variant filters when observations are permitted at the filter terminals only. It appears that the product of the maximum time- and frequency-spreadings produced by the time-variant filter sets a limit on our ability to determine the instantaneous values unambiguously even in the absence of additive noise. This limit can be relaxed when it comes to determining average or statistical parameters of the filter. For the determination of second-order filter statistics a fourth-moment method is presented that exhibits some novel aspects.
TL;DR: It is demonstrated that all impulse-equivalent pulse trains can be generated by exciting a linear digital filter with two input pulses that correspond in time to the initiation and the termination of the desired output sequence.
Abstract: The necessary and sufficient conditions for a function to be the autocorrelation function of a waveform consisting of a finite length train of pulses of various amplitudes is derived. The number of pulse trains having a given realizable autocorrelation function is determined. An "impulse-equivalent" pulse train is defined as one that yields, as closely as is theoretically possible, the same autocorrelation function that a single pulse gives. It is shown that impulse-equivalent pulse sequences exist for all lengths; the number of classes of these sequences is exponentially related to their length. It is demonstrated that all impulse-equivalent pulse trains can be generated by exciting a linear digital filter with two input pulses that correspond in time to the initiation and to the termination of the desired output sequence.
TL;DR: It has been shown experimentally that speech intelligibility is a function of grammatical content, and this fact implies that automatic speech recognizers may well need to include information about linguistic structure.
Abstract: It has been shown experimentally that speech intelligibility is a function of grammatical content. This fact implies that automatic speech recognizers may well need to include information about linguistic structure.
IBM1
TL;DR: A block code is presented that will correct an error consisting of the gain or loss of a bit (binary digit) within the block, which can be generalized to correct the loss or gain of a burst of bits.
Abstract: A block code is presented that will correct an error consisting of the gain or loss of a bit (binary digit) within the block. The code can be generalized to correct the loss or gain of a burst of bits. A further feature is the possibility of correcting additive errors appearing in the vicinity of the bit loss or gain. An additive error is a bit changed from 0 to 1 or from 1 to 0. The code is constructed by inserting a known character into a burst-error-correcting code at periodic intervals. The known character locates the approximate position of the bit loss or gain. At the location a bit is inserted or removed from the block, depending on whether a loss or a gain has occurred. The error-correcting code then corrects the erroneous bits between where the error occurred and where the correction took place.
TL;DR: Handwriting can be characterized as a sequence of basic strokes connected according to rule, and so generated approximates that of humans very closely.
Abstract: Handwriting can be characterized as a sequence of basic strokes connected according to rule. Handwriting so generated approximates that of humans very closely. Such a matching process can be used as the fundamental principle in a handwriting recognizer.
TL;DR: It is shown that the use of prior observations for pattern recognition may be described as a process of learning the statistical characteristics of the patterns involved, and optimal systems are shown to consist of banks of generalized correlators.
Abstract: This paper deals with the optimal use of a sequence of prior observations in order to recognize patterns. The class of pattern recognition problems which we treat is completely general in that the results may be applied to patterns of a visual, aural or electromagnetic origin. We show that the use of prior observations for pattern recognition may be described as a process of learning the statistical characteristics of the patterns involved. The mathematical description of the learning process is analyzed in order to obtain optimal (i.e., minimum probability of error) solutions to various types of pattern recognition problems. These solutions to the mathematical problem are then interpreted in terms of the systems implementation they imply. The optimum systems are shown to consist of banks of generalized correlators. Under assumption of independent measurements, the optimum systems reduce to a class of systems which function by reinforcing desired responses.
TL;DR: A nonstationary, nonlinear operation has been found which selectively removes moderate amounts of additive Gaussian noise from a received picture signal, and could in principle be incorporated in home receivers to improve reception in fringe areas.
Abstract: It is quite often possible to separate "picture" from "noise" in a television image. We do this when we view television under unfavorable conditions. This ability of the human observer suggests that completely automatic operation might at least partially separate noise and picture. A nonstationary, nonlinear operation has been found which selectively removes moderate amounts of additive Gaussian noise from a received picture signal. Running tests are performed upon the signal to detect the presence of perceptually significant picture detail in a number of different categories. Depending upon the test results, a selection is made from a number of available smoothing filter modes to maximize the suppression of noise without picture blurring. No preparatory operation is required at the transmitter, so that the technique is compatible with existing picture transmission systems. The process is applicable to conventional broadcast television, and could in principle be incorporated in home receivers to improve reception in fringe areas.
TL;DR: It is shown that the polarity-coincidence correlator (PCC), a two-input detection device, is nonparametric in its false-alarm rate with respect to a wide class of signal and noise distributions.
Abstract: It is shown that the polarity-coincidence correlator (PCC), a two-input detection device, is nonparametric in its false-alarm rate with respect to a wide class of signal and noise distributions. Operating in fixed distributions, the probability of detection is a nondecreasing function of input SNR. Although the PCC performs less well than an ordinary correlator or a Neyman-Pearson detector for Gaussian inputs, it can be markedly superior to both.
TL;DR: A pattern recognition technique is described in which a parametric representation of input signals or stimuli is employed and was successfully applied to the automatic recognition of speaker identity regardless of the spoken test.
Abstract: A pattern recognition technique is described in which a parametric representation of input signals or stimuli is employed. An input is considered as a vector, while the stimulus class is a multivariate process in the vector space. An adaptive sample set construction technique is described through which the conditional joint probability density of a class is approximated by the sum of Gaussian densities. The mean of each such density is an adaptively chosen "typical" sample of the class, and the set of samples so chosen are contained in the region of the space in which samples of the class are most populous. The decision process using the typical samples partitions the space into regions that envelop the chosen samples of a class. Arbitrary shaped and multiply connected regions can be constructed in this way, and multimodal probability densities can be approximated with a computationally simple procedure. Decision making on an incomplete set of parameters and on multiple observations of the input stimulus are discussed. This technique was successfully applied to the automatic recognition of speaker identity regardless of the spoken test. Experimental results are given.
TL;DR: The techniques of sequential analysis are applied to a multiple-resolution-element search radar for different types of detection and results suggest that truncation is not necessary for the tests which were considered.
Abstract: In this paper the techniques of sequential analysis are applied to a multiple-resolution-element search radar for different types of detection. By using sequential methods such a radar shows considerable saving compared to fixed-scan-rate detection radars. Many of the problems occurring in sequential radar detection are examined; the difficulties of obtaining analytical results for multiple-resolution-element sequential radars are discussed. Most of the results of this paper were obtained by simulation. Included are interesting results which suggest that truncation is not necessary for the tests which were considered. Curves are given describing the improvement obtained by sequential detection procedures. The curves are discussed in detail to make this paper useful to those whose primary interest is in the results which are obtained.
TL;DR: Variable-length codes recursively defined by certain sequential machines are investigated and it is seen that the recursive definition may be used to control error propagation as well as to provide a conceptually simple decoding procedure.
Abstract: Variable-length codes recursively defined by certain sequential machines are investigated. It is seen that the recursive definition may be used to control error propagation as well as to provide a conceptually simple decoding procedure. Furthermore, the variety of these codes is such that the theoretical minimum average code-word length can be approached quite closely for many distributions. Methods for obtaining efficient codes are discussed, and examples are given.
TL;DR: A number of statistical properties of the contours of such surfaces are derived and an application of the results to the problem of obtaining bounds on the information content of quantized random surfaces is outlined.
Abstract: A random surface is a sample function of a random process \{f(x, y)\} depending on two real parameters. Examples of random surfaces would be: photographs or television pictures; topographic maps; atmospheric pressure charts; and the like. A number of statistical properties of the contours of such surfaces are derived. An application of the results to the problem of obtaining bounds on the information content of quantized random surfaces is outlined.
TL;DR: It is shown that an adaptive error-correction procedure can bring the threshold to the point of equal false alarm and missed detection probabilities.
Abstract: This paper considers a threshold detector operating on noisy binary pulses. It is shown that an adaptive error-correction procedure can bring the threshold to the point of equal false alarm and missed detection probabilities.
TL;DR: A detailed study has been made of a certain class of systematic binary error-correcting codes that will correct the error bursts typical of some digital channels.
Abstract: A detailed study has been made of a certain class of systematic binary error-correcting codes that will correct the error bursts typical of some digital channels. These codes--generalizations of codes discovered by Abramson and Melas--are cyclic codes designed to correct any single burst of errors per n -digit word provided that the width of the burst (regarded cyclically) does not exceed a certain number of digits, b . Moreover, these codes are optimum in the sense that they employ the minimum number of redundant digits theoretically possible for a cyclic code with given values of n and b . A cyclic code is completely characterized by its generator polynomial g(x) , hence, the properties of the code can be determined by analysis of the corresponding g(x) . Necessary and sufficient conditions on g(x) have been formulated for the corresponding cyclic code to be an optimum burst- b correcting code. These conditions have been formulated into a series of tests that can be carried out (in principle) on any g(x) . All optimum burst- b cyclic codes with n and b have been found in this way and their generators are tabulated in the paper. In all, 98 codes are listed (not counting reciprocals) for b = 3 and b = 4 ; it was shown that no optimum codes exist for b = 5 within the limits stated. Practical codes for b \geq 6 will probably be nonoptimum codes because of the extreme word lengths required for optimum ones.
TL;DR: A novel by-product of the study is an error-probability expression for non-adaptive multichannel reception of quite general binary signaling, which depends on the type of signaling and number of channels.
Abstract: Description and performance are given for two slightly different forms of an adaptive receiver that is used with binary signaling in a multichannel communication system. Each channel has a non-dispersive, nonfading propagation path and additive white gaussian noise that is independent of, and equal in intensity to, the other channel noises. Either phase-independent orthogonal signaling (such as FSK) or phase-reversal-comparison signaling is employed to convey to the receiver both the message and information about the path strengths and phases. The receiver measures the path parameters and applies the results to the detection processing as though they were perfectly accurate; it is shown that this procedure is a natural extension of non-adaptive reception. Results are given without derivation for the probability of binary decision error, which depends on the type of signaling, the ratio of the total noise-exclusive signaling energy received to the noise intensity, the number of channels, and the ratio of the effective measurement time to the binary signal duration. The distribution of the received signaling energy among the channels is immaterial. Graphs of error probability are presented for selected system parameters. A novel by-product of the study is an error-probability expression for non-adaptive multichannel reception of quite general binary signaling. Optimal adjustment of the adaptive receiver for fading-path situations is also considered, but corresponding error probabilities have not yet been derived.
TL;DR: The method of reduction of multiple integrals presented here can be generalized, and may find application in other statistical studies in which the Gaussian density form is encountered under an integral.
Abstract: Formulas for the error probabilities of equicorrelated M -ary signals under optimum phase-coherent and phase-incoherent reception are derived in the form of previously untabulated single and double integrals. These integrals are amenable to computer evaluation for arbitrary M . Two modes of reception are considered. In the first, one of M equal energy equiprobable signals is known to be transmitted during a symbol interval of T seconds through a nonfading channel with additive white Gaussian noise. The receiver is assumed to be synchronized in time and frequency with the incoming signal, and reception is on a per-symbol basis. Furthermore, the cross-correlation coefficients between all the signals are equal. The probability of correct decision in both phase-coherent and phase-incoherent reception is derived exactly, as a function of the signal-to-noise ratio, the common cross-correlation coefficient, and the size of the signal set M . In the second mode of reception, the only difference is that a threshold is incorporated in the receiver. The probability of false detection and the probability of detection and correct decision are derived exactly for both phase-coherent and phase-incoherent reception as a function of the threshold level, the signal-to-noise ratio, the common cross-correlation coefficient, and the size of the signal set M . The method of reduction of multiple integrals presented here can be generalized, and may find application in other statistical studies in which the Gaussian density form is encountered under an integral.
TL;DR: The two-dimensional analogy of perfect maps to shift register codes suggests a possible development of planar error-correcting codes and an application to the two- dimensional range-velocity ambiguity problem of radar.
Abstract: In determining location in a previously mapped region by map-matching, there arises the question of minimum submap size relative to the size of the complete map of the region for unambiguous determination of position. A lower bound for the size of the submap is obtained for quantized binary maps. It is shown that there exist maps (called perfect) such that this lower bound is realized. Of special interest is the construction of a doubly periodic 4 \times 4 perfect map for a 2 \times 2 submap. The two-dimensional analogy of perfect maps to shift register codes suggests a possible development of planar error-correcting codes and an application to the two-dimensional range-velocity ambiguity problem of radar.
TL;DR: In the period considered here the variance of the frequency distributions of pitch grows from 3 to 13 and the mean value of the kurtosis of the intervals of consecutive tones grows from 5 to 15 and splits up in recent times into two distinct values.
Abstract: Statistical properties of music and their development since about 1500 have been studied; e.g. the properties of frequency distributions of pitch and duration of tones, of intervals of consecutive tones, correlograms of various parameters up to distances of 30 of these parameters, matrices of transition probabilities of various parameters. In the period considered here the variance of the frequency distributions of pitch grows from 3 to 13 and the mean value of the kurtosis of the intervals of consecutive tones grows from 5 to 15 and splits up in recent times into two distinct values (15 and 6, 5).