In this final installment of the paper we consider the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now. To a considerable extent the continuous case can be obtained through a limiting process from the discrete case by dividing the continuum of messages and signals into a large but finite number of small regions and calculating the various parameters involved on a discrete basis. As the size of the regions is decreased these parameters in general approach as limits the proper values for the continuous case. There are, however, a few new effects that appear and also a general change of emphasis in the direction of specialization of the general results to particular cases.

A mathematical theory of communication

Scientific knowledge grows at a phenomenal pace--but few books have had as lasting an impact or played as important a role in our modern world as The Mathematical Theory of Communication, published originally as a paper on communication theory more than fifty years ago. Republished in book form shortly thereafter, it has since gone through four hardcover and sixteen paperback printings. It is a revolutionary work, astounding in its foresight and contemporaneity. The University of Illinois Press is pleased and honored to issue this commemorative reprinting of a classic.

https://www.cs.ucf.edu/~dcm/Teaching/COP5611-Spring2012/Shannon48-MathTheoryComm.pdf

The Mathematical Theory of Communication

A Mathematical Theory Communication

The probability of error in decoding an optimal convolutional code transmitted over a memoryless channel is bounded from above and below as a function of the constraint length of the code. For all but pathological channels the bounds are asymptotically (exponentially) tight for rates above R_{0} , the computational cutoff rate of sequential decoding. As a function of constraint length the performance of optimal convolutional codes is shown to be superior to that of block codes of the same length, the relative improvement increasing with rate. The upper bound is obtained for a specific probabilistic nonsequential decoding algorithm which is shown to be asymptotically optimum for rates above R_{0} and whose performance bears certain similarities to that of sequential decoding algorithms.

Error bounds for convolutional codes and an asymptotically optimum decoding algorithm

A method is developed for representing any communication system geometrically Messages and the corresponding signals are points in two "function spaces," and the modulation process is a mapping of one space into the other Using this representation, a number of results in communication theory are deduced concerning expansion and compression of bandwidth and the threshold effect Formulas are found for the maximum rate of transmission of binary digits over a system when the signal is perturbed by various types of noise Some of the properties of "ideal" systems which transmit at this maximum rate are discussed The equivalent number of binary digits per second for certain information sources is calculated

/pdf/communication-in-the-presence-of-noise-p5l6l0cm8w.pdf

Communication in the presence of noise

A quantitative measure of “information” is developed which is based on physical as contrasted with psychological considerations. How the rate of transmission of this information over a system is limited by the distortion resulting from storage of energy is discussed from the transient viewpoint. The relation between the transient and steady state viewpoints is reviewed. It is shown that when the storage of energy is used to restrict the steady state transmission to a limited range of frequencies the amount of information that can be transmitted is proportional to the product of the width of the frequency-range by the time it is available. Several illustrations of the application of this principle to practical systems are included. In the case of picture transmission and television the spacial variation of intensity is analyzed by a steady state method analogous to that commonly used for variations with time.

Transmission of information

The Fourier identity is here expressed in a more symmetrical form which leads to certain analogies between the function of the original variable and its transform. Also it permits a function of time, for example, to be analyzed into two independent sets of sinusoidal components, one of which is represented in terms of positive frequencies, and the other of negative. The steady-state treatment of transmission problems in terms of this analysis is similar to the familiar ones and may be carried out either in terms of real quantities or of complex exponentials. In the transient treatment, use is made of the analogies referred to above, and their relation to the method of "paired echoes" is discussed. A restatement is made of the condition which is known to be necessary in order that a given steady-state characteristic may represent a passive or stable active system (actual or ideal). A particular necessary condition is deduced from this as an illustration.

A More Symmetrical Fourier Analysis Applied to Transmission Problems

Theory of the Binaural Location of Pure Tones.---It is pointed out that to determine two co\"ordinates of the sound source, angular position and distance, the simultaneous observation of both phase difference and intensity ratio for the two ears is necessary. Thus, contrary to the conclusions of certain observers, intensity ratio plays an important part in sound location. Using a method of analysis due to Stokes, the authors have prepared a series of charts showing the intensity ratio and phase difference for sources at various distances and angular positions emitting sound with a frequency from 310 to 1,860 cycles and for two angular separations of the ears, 165\ifmmode^\circ\else\textdegree\fi{} and 180\ifmmode^\circ\else\textdegree\fi{}, assumed to be placed on a spherical head. These charts show the theoretical position of the source for any combination of phase difference and relative intensity which might occur with actual sources. Experiments to test this theory are suggested. When unnatural combinations occur as a result of distortion of the sound in transmission or are produced artificially in the laboratory, the interpretation of the stimuli will depend on psychological factors and cannot be explained by any purely physical theory alone.Psychological Readjustment of Unnatural Combinations of Binaural Stimuli.---Suggestions as to the nature of these adjustments are advanced and the results of laboratory experiments with such unnatural combinations are interpreted with the assistance of the charts and are shown to be in agreement with this theory.

/pdf/the-binaural-location-of-pure-tones-g503p6qesi.pdf

The Binaural Location of Pure Tones.

A theoretical study is presented of the properties of a condenser, one plate of which is free to vibrate, when it is included in a circuit containing a generator, the frequency of which is higher than the resonant frequency of the plate and unrelated thereto. It is shown that the plate may be maintained in oscillation at a frequency at or near its mechanical resonance, at the expense of the energy supplied by the generator, provided certain conditions are satisfied. The most favorable condition is one in which the plate is resonant at the frequency of its vibration and the electric circuit is resonant at that of the generator, and at the difference between the generator and plate frequencies, and is anti-resonant at their sum. Under these conditions the generator voltage must exceed a threshold value determined by the impedances and frequencies. This threshold voltage increases as the conditions become less favorable. Expressions are given for the values of the oscillations as functions of the voltage when the threshold is exceeded. When the sum frequency is absent, the energies dissipated at the plate and difference frequencies are in the ratio of the two frequencies. The oscillations described represent a special case of a class of similar oscillations, all of which depend on the presence of a non-linear reactance. Another special case is a molecular model capable of reproducing the main features of the Raman effect.

https://ieeexplore.ieee.org/iel7/6731005/6773279/06773284.pdf

Oscillations in systems with non-linear reactance

The modulation of a carrier wave by a signal wave and the subsequent reproduction of the signal wave are discussed in terms of the sustained sinusoidal components into which the various waves may be resolved. The conception of a modulated wave as being made up of a carrier component and two side-bands is explained. The saving in frequency range resulting from transmitting a single side-band is emphasized. Carrier suppression and homodyne reception are shown to save sending power, increase receiving sensitiveness and aid in suppressing interference from both stations and static. Equations are derived for the distortion of the reproduced wave in terms of that of the modulated wave. In general, both the amplitude and phase of each component of the signal depend on both the amplitudes and phases of the corresponding side-band components and of the carrier. The effect on the signal of various typical distortions of the radio wave is examined for both single and double side-band transmission, as is also that of altering the phase of the locally supplied carrier and of altering its frequency. The resulting distortion of the signal is found, in general, to be more serious for telephony when both side-bands are used and for telegraphy when only one is used. It is also shown that the reproduced signal has twice the power when both side-bands are transmitted as when the same sending power is concentrated in one. The signal to static ratio is, however, the same in both cases.

R. V. L. Hartley

Papers

Transmission of information

A More Symmetrical Fourier Analysis Applied to Transmission Problems

The Binaural Location of Pure Tones.

Oscillations in systems with non-linear reactance

Relations of Carrier and Side-Bands in Radio Transmission