Showing papers in "IEEE Transactions on Information Theory in 1960"
TL;DR: This paper discusses the problem of the minimization of the distortion of a signal by a quantizer when the number of output levels of the quantizer is fixed and an algorithm is developed to simplify their numerical solution.
Abstract: This paper discusses the problem of the minimization of the distortion of a signal by a quantizer when the number of output levels of the quantizer is fixed. The distortion is defined as the expected value of some function of the error between the input and the output of the quantizer. Equations are derived for the parameters of a quantizer with minimum distortion. The equations are not soluble without recourse to numerical methods, so an algorithm is developed to simplify their numerical solution. The case of an input signal with normally distributed amplitude and an expected squared error distortion measure is explicitly computed and values of the optimum quantizer parameters are tabulated. The optimization of a quantizer subject to the restriction that both input and output levels be equally spaced is also treated, and appropriate parameters are tabulated for the same case as above.
2,168 citations
TL;DR: In a tutorial exposition, the following topics are discussed: definition of a matched filter; where matched filters arise; properties of matched filters; matched-filter synthesis and signal specification; some forms of matched filter.
Abstract: In a tutorial exposition, the following topics are discussed: definition of a matched filter; where matched filters arise; properties of matched filters; matched-filter synthesis and signal specification; some forms of matched filters.
1,138 citations
TL;DR: This report presents data from which one may obtain the probability that a pulsed-type radar system will detect a given target at any range, in contrast to the usual method of obtaining radar range as a single number.
Abstract: This report presents data from which one may obtain the probability that a pulsed-type radar system will detect a given target at any range. This is in contrast to the usual method of obtaining radar range as a single number, which can be taken mathematically to imply that the probability of detection is zero at any range greater than this number, and certainty at any range less than this number. Three variables, which have so far received little quantitative attention in the subject of radar range, are introduced in the theory: l.The time taken to detect the target. 2.The average time interval between false alarms (false indications of targets). 3.The number of pulses integrated. It is shown briefly how the results for pulsed-type systems may be applied in the case of continuous-wave systems. Those concerned with systems analysis problems including radar performance may profitably use this work as one link in a chain involving several probabilities. For instance, the probability that a given aircraft will be detected at least once while flying any given path through a specified model radar network may be calculated using the data presented here as a basis, provided that additional probability data on such things as outage time etc., are available. The theory developed here does not take account of interference such as clutter or man-made static, but assumes only random noise at the receiver input. Also, an ideal type of electronic integrator and detector are assumed. Thus the results are the best that can be obtained under ideal conditions. It is not too difficult, however, to make reasonable assumptions which will permit application of the results to the currently available types of radar. The first part of this report is a restatement of known radar fundamentals and supplies continuity and background for what follows. The mathematical part of the theory is not contained herein, but will be issued subsequently as a Separate report(2a)
759 citations
TL;DR: The investigation shows that, for these fluctuation models, the probability of detection for a fluctuating target is less than that for a non-fluctuating target if the range is sufficiently short, and is greater if therange is sufficiently long.
Abstract: This report considers the probability of detection off a target by a pulsed search radar, when the target has a fluctuating cross section. Formulas for detection probability are derived, and curves off detection probability vs, range are given, for four different target fluctuation models. The investigation shows that, for these fluctuation models, the probability of detection for a fluctuating target is less than that for a non-fluctuating target if the range is sufficiently short, and is greater if the range is sufficiently long. The amount by which the fluctuating and non-fluctuating cases differ depends on the rapidity of fluctuation and on the statistical distribution of the fluctuations. Figure 18, p. 307, shows a comparison between the non-fluctuating case and the four fluctuating cases considered.
633 citations
TL;DR: The optical implementation of integral transforms which are of importance to communication theory is discussed; the general problems of optical filter synthesis and multichannel computation and data processing are introduced, followed by a discussion of potential applications.
Abstract: Optical systems, which inherently possess two degrees of freedom rather than the single degree of freedom available in a single electronic channel, appear to offer some advantages over their electronic counterparts for certain applications. Coherent optical systems have the added property that one may easily obtain many successive two-dimensional Fourier transforms of any given light amplitude distribution, or, by use of astigmatic optics, one-dimensional transforms can be obtained. Therefore, most linear operations of an integral transform nature are easily implemented. The optical implementation of integral transforms which are of importance to communication theory is discussed; the general problems of optical filter synthesis and multichannel computation and data processing are introduced, followed by a discussion of potential applications. Astigmatic systems, which permit multichannel operation in lieu of two-dimensional processing, are treated as a special case of general two-dimensional processors. Complex input functions are discussed with relation to their role in coherent optical systems.
346 citations
TL;DR: New results are given in the field in the form of theorems which permit systematic construction of codes for given n, d ; for some n,d , the codes contain the greatest possible numbers of points.
Abstract: Two n -digit sequences, called "points," of binary digits are said to be at distance d if exactly d corresponding digits are unlike in the two sequences. The construction of sets of points, called codes, in which some specified minimum distance is maintained between pairs of points is of interest in the design of self-checking systems for computing with or transmitting binary digits, the minimum distance being the minimum number of digital errors required to produce an undetected error in the system output. Previous work in the field had established general upper bounds for the number of n -digit points in codes of minimum distance d with certain properties. This paper gives new results in the field in the form of theorems which permit systematic construction of codes for given n, d ; for some n, d , the codes contain the greatest possible numbers of points.
334 citations
TL;DR: A simple error-correction procedure for binary codes which for arbitrary m and t are t -error correcting and have length 2^m - 1 of which no more than mt digits are redundancy is described.
Abstract: Bose and Ray-Chaudhuri have recently described a class of binary codes which for arbitrary m and t are t -error correcting and have length 2^m - 1 of which no more than mt digits are redundancy. This paper describes a simple error-correction procedure for these codes. Their cyclic structure is demonstrated and methods of exploiting it to implement the coding and correction procedure using shift registers are outlined. Closer bounds on the number of redundancy digits are derived.
320 citations
TL;DR: It is shown that the concept of correlation detection of deterministic signals in additive Gaussian noise can be extended in a natural manner to the detection of signals that are transmitted through a "Gaussian" random channel besides being corrupted by additiveGaussian noise.
Abstract: We show that the concept of correlation detection of deterministic signals in additive Gaussian noise can be extended in a natural manner to the detection of signals that are transmitted through a "Gaussian" random channel besides being corrupted by additive Gaussian noise. Such situations are typical in communication over scatter-multipath channels (with or without a specular component). In the deterministic case, the receiver essentially crosscorrelates the received signal with the signal before the additive noise was introduced. When a random channel is present, however, this latter signal, i.e., the output of the random channel, is not known at the receiver, However, knowing the statistics of the channel and the noise, the receiver can make an estimate of it from the received signal on the hypothesis that a particular signal was transmitted. The optimum receiver then crosscorrelates this estimate with the received signal.
143 citations
TL;DR: If blocks of N digits are used, the prefix should be chosen to make large the number G(N) of different blocks which satisfy the constraints, and strengthening the prefix decreases the number of "message digits" which remain in the block but also relaxes the constraints.
Abstract: When messages are transmitted as blocks of binary digits, means of locating the beginnings of blocks are provided to keep the receiver in synchronism with the transmitter. Ordinarily, one uses a special synchronizing symbol (which is really a third kind of digit, neither 0 nor 1) for this purpose. The Morse code letter space and the teletype start and stop pulses are examples. If a special synchronizing digit is not available, its function may be served by a short sequence of binary digits P which is placed as a prefix to each block. The other digits must then be constrained to keep the sequence P from appearing within a block. If blocks of N digits (including the prefix P ) are used, the prefix should be chosen to make large the number G(N) of different blocks which satisfy the constraints. Lengthening the prefix decreases the number of "message digits" which remain in the block but also relaxes the constraints. Thus, for each N , there corresponds some optimum length of prefix. For each prefix P , a generating function, a recurrence formula, and an asymptotic formula for large N are found for G(N) . Tables of G(N) are given for all prefixes of four digits or fewer. Among all prefixes P of a given length A , the one for which G(N) has the most rapid growth is P = 11 \cdots 1 . However, for this choice of P , the table of values of G(N) starts with small values; 11 \cdots 1 does not become the best A -digit prefix until N is very large. At these values of N , the (A + 1) - digit prefix 11 \cdots 10 is still better. The tables suggest that, for any N , a best prefix can always be found in the form 11 \cdots 10 , for suitable A . Taking P = 11 \cdots 10 and A = [\log_2 (N \log_2 e)] it is shown that G(N) is roughly 0.35N^{-1} X2^N . This result is near optimal since no choice of P can make G(N) exceed N^{-1}2^N .
134 citations
TL;DR: A method is described which permits the systematic construction of codes capable of error-free transmission, provided errors occur in "clusters" of limited duration.
Abstract: A method is described which permits the systematic construction of codes capable of error-free transmission, provided errors occur in "clusters" of limited duration. The method is valid for error clusters of any prescribed duration. The codes are relatively easy to implement and decoding operations are straightforward. Specific examples are given and applications to teletype transmission are discussed.
124 citations
TL;DR: A class of quaternary codes is described, and an algorithm for generating the codes is given that makes them useful for radar applications.
Abstract: A class of quaternary codes is described, and an algorithm for generating the codes is given. The codes have properties that make them useful for radar applications: 1) their auto-correlation consists of a single pulse, 2) their length can be any power of two, 3) each code can be paired with another code (its mate) of the same class in such a way that the crosscorrelation of mates is identically zero, 4) coded waveforms can be generated in a simple network the number of whose elements is proportional to the base-2 logarithm of the code length, and 5) the same network can be readily converted to a matched filter for the coded waveform.
TL;DR: Using the inverse probability approach, expressions for the minimum variances of estimators of the delay, Doppler, and doppler rate of a radar return are derived assumed to consist of a long train of pulses with independent scintillation from pulse to pulse.
Abstract: The methods of inverse probability have been used by Woodward and others to obtain lower bounds on variances of radar parameter estimators. Previous results on the lower bounds of variances of delay and Doppler estimators have assumed that the reflecting object travels with a constant line of sight velocity and does not cause scintillation in the radar return. Using the inverse probability approach, this paper derives expressions for the minimum variances of estimators of the delay, Doppler, and Doppler rate of a radar return assumed to consist of a long train of pulses with independent scintillation from pulse to pulse.
TL;DR: The Woodward ambiguity function is discussed in connection with the output of a matched filter and a formula is applied to determine the effect of signal waveform on the output signal-to-reverberation power ratio of a matching filter for a simple distribution of randomized scatterers.
Abstract: The Woodward ambiguity function is discussed in connection with the output of a matched filter. A formula for the treatment of sonar reverberation or radar clutter is set up in terms of the ambiguity function. This formula is applied to determine the effect of signal waveform on the output signal-to-reverberation power ratio of a matched filter for a simple distribution of randomized scatterers.
TL;DR: A matched filter system is described which was designed to detect complicated signals subject to a wide range of possible Doppler shifts, and the performance of the resulting signal detection system was within 5 db of that of an ideal theoretical model.
Abstract: A matched filter system is described which was designed to detect complicated signals subject to a wide range of possible Doppler shifts. A 100 tap band-pass delay line was used in conjunction with a resistor weighting matrix to synthesize signals and filter characteristics. The system could handle a signal with a duration-bandwidth product of 100 over a range of Doppler frequency shifts 17 times the reciprocal of the signal duration. A theoretical discussion of the Doppler effect is given, making use of conjugate functions or Hilbert Transforms. Various engineering compromises which simplify the construction of matched filters are suggested. The performance of the resulting signal detection system was within 5 db of that of an ideal theoretical model.
TL;DR: Z -transform theory is applied to provide a simple recursive formula for generating the values from a set of independent Gaussian random variables for simulating a sequence of sampled values of a stationary Gaussian process having a specified correlation function or power spectrum.
Abstract: A computationally convenient method is presented for simulating a sequence of sampled values of a stationary Gaussian process having a specified correlation function or power spectrum. Z -transform theory is applied to provide a simple recursive formula for generating the values from a set of independent Gaussian random variables.
TL;DR: It is shown that this "photon channel" model leads to more realistic conclusions regarding information transmission, including the maximum entropy for a narrow-band source under an average power limitation and the transmission rate through a Poisson channel with additive Poisson noise.
Abstract: In this paper the quantum nature of electromagnetic radiation is used as a basis for a mathematical model of a continuous channel It is shown that this "photon channel" model leads to more realistic conclusions regarding information transmission Among the results obtained are: 1) the maximum entropy for a narrow-band source under an average power limitation, 2) the frequency distribution (Bose-Einstein) for a wide-band power-limited source, 3) the transmission rate through a Poisson channel with additive Poisson noise
TL;DR: A matched filter communication system is described whose underlying principles are based on the Rake, where multiple propagation paths due to ionospheric reflection are resolved by the broadband signals, resulting in the appearance of the multipath pattern at the output of the Mark or Space matched filter.
Abstract: A matched filter communication system is described whose underlying principles are based on the Rake. The point-to-point synchronous teletype system employs complex lumped-parameter networks to generate and receive a pair of long-duration, broadband signals representing Mark and Space respectively. The receiver contains a pair of matched filters whose output is a narrow pulse when the matching waveform is applied. One advantage of the system, arising from the long duration of the signals, is an increase in energy per teletype baud when operating under a peak power limitation. Another is that multiple propagation paths due to ionospheric reflection are resolved by the broadband signals, resulting in the appearance of the multipath pattern at the output of the Mark or Space matched filter. Recombination of paths is achieved by means of a recirculating delay line tuned to the teletype baud rate in conjunction with parallel multiplier-integrators in the Mark and Space channels. The combination acts as a self-adjusting correlation detector for the multipath pattern.
TL;DR: This note considers the generalization of Abramson's codes to the p -nary case, where a symbol alphabet consisting of the digits 0, 1, \cdots , p - 1 is used for transmission, p being a prime number.
Abstract: Binary group codes described by Abramson permit the correction of all single errors and all double errors in adjacent digits, with the use of significantly fewer check digits than codes capable of correcting all double-bit errors. This note considers the generalization of Abramson's codes to the p -nary case, where a symbol alphabet consisting of the digits 0, 1, \cdots , p - 1 is used for transmission, p being a prime number. Examples of such p -nary codes are given, as well as necessary conditions for their existence. These codes bear the same relation to the p -nary Golay codes as Abramson's codes do to the familiar Hamming codes. Some as yet unanswered questions are raised, and suggestions for further possible generalizations are given.
TL;DR: It is shown, for incoherent reception in some important special instances, that matched filters based on maximizing output signal-to-noise ratio are also optimum from the broader, decision viewpoint.
Abstract: In this paper it is shown how the earlier concepts of the matched filter may be generalized by recognizing explicitly the decision-making character of most reception systems. Accordingly, when an approach making use of statistical decision theory is applied for both signal detection and extraction, a variety of new classes of matched filters (Bayes matched filters) can be defined. These can be described specifically in the critical situation of threshold reception, where system optimality is at a premium. It is shown, for incoherent reception in some important special instances, that matched filters based on maximizing output signal-to-noise ratio (the S/N matched filters of the earlier theory) are also optimum from the broader, decision viewpoint. The required optimum filters are themselves time-varying and nonunique, and thus permit a measure of design freedom. In all instances, realizable filters are possible, and it is shown how their weighting functions may be determined. Both discrete and continuous filtering on a finite interval, (0, T) , are considered.
TL;DR: This paper presents the first results in an unconventional approach to the problem of mean-square optimization, where instead of obtaining a representation for the optimal operator for a process, it seeks to characterize the class of processes for which the optimal operators is of specified form.
Abstract: This paper presents the first results in an unconventional approach to the problem of mean-square optimization. Instead of obtaining a representation for the optimal operator for a process, we seek to characterize the class of processes for which the optimal operator is of specified form. If the processes are given, so that the multivariate characteristic functions are known, then our results can be used to tell whether it is possible for the optimal operator to have a specified form. The bulk of the paper pertains to the signal extraction problem where the signal and noise are independent and additive, and it is desired to estimate some function of the signal. Here, with a slight shift in viewpoint, we phrase the characterization problem in the following way: Given, for example, a noise process, determine the class of signal processes for which the optimal extraction system is of specified form. The case where the noise process is Gaussian comes in for special attention.
TL;DR: The multidimensional demodulation problem is considered from the point of view of statistical estimation theory and a posteriori most probable signal estimates are derived.
Abstract: The multidimensional demodulation problem is considered from the point of view of statistical estimation theory and a posteriori most probable signal estimates are derived. Correlated signals and noises are treated. This formulation yields a set of two matrix integral equations which must be solved for the optimum estimates. For amplitude modulation, the problem reduces to that of finding a set of time varying filters which are, again, solutions to a matrix integral equation. Special cases such as two-receiver systems, quadrature modulation, and single-sideband have particularly simple representations and are considered in some detail.
TL;DR: A technique is described for finding simply computable numerical-valued functions of a received binary word whose value indicates where errors in transmission have occurred, and the construction of decoding procedures for the perfect three-error-correcting code is illustrated.
Abstract: A technique is described for finding simply computable numerical-valued functions of a received binary word whose value indicates where errors in transmission have occurred. Although it seems that a certain condition must usually be fulfilled for such functions to exist, or for our method to constitute an efficient procedure for finding them, there is, on the one hand, a strong tendency for "good" codes to satisfy the condition, while, on the other, it appears to be straightforward to construct codes which are good for a specified channel and also fulfill the condition. An advantage of the resulting decoding procedure is that it corrects and detects all possible errors; more precisely, if a word u is received and the coset \bar{u} to which u belongs has a unique leader e , the procedure concludes that u + e was sent, while if u has no unique leader, that fact, along with the weight of \bar{u} (and sometimes a little more) can be indicated. The ideas and techniques are illustrated by the construction of decoding procedures for the perfect (23, 12) three-error-correcting code.
TL;DR: An upper bound for the information rate in an additive Gaussian noise channel, based on the use of regular-simplex coded waveforms, has been calculated and indicates that for signal-to-noise energy ratios less than approximately unity and for error probabilities ranging from 10^{-2} to l0^{-8} , this rate is a sizable percentage of the ideal rate which has been derived by Shannon and Tuller.
Abstract: M equally probable symbols may be encoded as continuous waveforms whose sample values are the coordinates of the vertices of a regular simplex in N dimensions. These vertices or code points, therefore, are equally spaced, and the resulting waveforms are adapted particularly to low signal-to-noise communication systems. An upper bound for the information rate in an additive Gaussian noise channel, based on the use of regular-simplex coded waveforms, has been calculated. The results indicate that for signal-to-noise energy ratios less than approximately unity and for error probabilities ranging from 10^{-2} to l0^{-8} , this rate is a sizable percentage of the ideal rate which has been derived by Shannon and Tuller.
TL;DR: The properties of iterated binary channels are investigated, and curves allowing one to use the ordering are given, and an example using these curves is worked out.
Abstract: The properties of iterated binary channels are investigated. An ordering (defined by the symbol \supset ) of communication channels with two possible inputs and any number of possible outputs is defined. For any two such channels C_1 and C_2 , this ordering has the property that if C_1 \supset C_2 , the minimum average loss when using C_1 will be less than the minimum average loss using C_2 , independent of the losses assigned to the various errors, and independent of the statistics of the source. This ordering is applied to 1) the general binary channel, 2) the iterated binary symmetric channel, and 3) the unreliable binary symmetric channel when used with many iterations. Curves allowing one to use the ordering are given, and an example using these curves is worked out.
TL;DR: This work was supported in part by the U. S. Army (Signal Corps.), the TJ.
Abstract: * Received by the PGIT, October 15, 1959. This work was supported in part by the U. 8. Army (Signal Corps.), the TJ. S. Air Force (Oflioe of Scientific Rcscarch, Air Res. and Dcv. Command), and the U. Y. Navy (Office of Naval Research). 1 D. Middleton, “On the detection of stochastic signals in additive norm;ll noise-part I,” IRE TRANS. “iv IN~OR~~ATION THEORY, vol. IT-3, pp. 86121, eqs. (277)-(285); June, 1957. p D. Middloton, this issue, p. 349. 3 I’. M. Woodward, “Probability and Information Theory with Applications to Radar,” Pergumon PI-288, London, Ens.: 1953. 4 T I<,tilath “Sampling Models for Linear TimcVariant i%lters’,” Rcs. Lab. of Electronics, Mass. Inst. Tech., Cambridge, Mass., Tech. Rept. no. 352: May, 1959. 6 W. B. Davenport and W. L. Root, “An Introduction to Random Signals and Noise,” McGrawI-1111 nook Co., Inc., New York, N. Y.; 1958. where