Showing papers by "AT&T Labs published in 1996"

Support Vector Regression Machines

Abstract: A new regression technique based on Vapnik's concept of support vectors is introduced. We compare support vector regression (SVR) with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimensionality of the input space.

Pushing the envelope: planning, propositional logic, and stochastic search

Abstract: Planning is a notoriously hard combinatorial search problem. In many interesting domains, current planning algorithms fail to scale up gracefully. By combining a general, stochastic search algorithm and appropriate problem encodings based on propositional logic, we are able to solve hard planning problems many times faster than the best current planning systems. Although stochastic methods have been shown to be very effective on a wide range of scheduling problems, this is the first demonstration of its power on truly challenging classical planning instances. This work also provides a new perspective on representational issues in planning.

Training algorithms for linear text classifiers

Abstract: Systems for text retrieval, routing, categorization and other IR tasks rely heavily on linear classifiers. We propose that two machine learning algorithms, the Widrow-Hoff and EG algorithms, be used in training linear text classifiers. In contrast to most IR methods, theoretical analysis provides performance guarantees and guidance on parameter settings for these algorithms. Experimental data is presented showing Widrow-Hoff and EG to be more effective than the widely used Rocchio algorithm on several categorization and routing tasks.

The power of amnesia: learning probabilistic automata with variable memory length

Abstract: We propose and analyze a distribution learning algorithm for variable memory length Markov processes. These processes can be described by a subclass of probabilistic finite automata which we name Probabilistic Suffix Automata (PSA). Though hardness results are known for learning distributions generated by general probabilistic automata, we prove that the algorithm we present can efficiently learn distributions generated by PSAs. In particular, we show that for any target PSA, the KL-divergence between the distribution generated by the target and the distribution generated by the hypothesis the learning algorithm outputs, can be made small with high confidence in polynomial time and sample complexity. The learning algorithm is motivated by applications in human-machine interaction. Here we present two applications of the algorithm. In the first one we apply the algorithm in order to construct a model of the English language, and use this model to correct corrupted text. In the second application we construct a simple stochastic model for E.coli DNA.

Game theory, on-line prediction and boosting

Abstract: We study the close connections between game theory, on-line prediction and boosting. After a brief review of game theory, we describe an algorithm for learning to play repeated games based on the on-line prediction methods of Littlestone and Warmuth. The analysis of this algorithm yields a simple proof of von Neumann’s famous minmax theorem, as well as a provable method of approximately solving a game. We then show that the on-line prediction model is obtained by applying this gameplaying algorithm to an appropriate choice of game and that boosting is obtained by applying the same algorithm to the “dual” of this game.

Learning trees and rules with set-valued features

Abstract: In most learning systems examples are represented as fixed-length "feature vectors", the components of which are either real numbers or nominal values. We propose an extension of the feature-vector representation that allows the value of a feature to be a set of strings; for instance, to represent a small white and black dog with the nominal features size and species and the set-valued feature color, one might use a feature vector with size=small, species=canis-familiaris and color-{white, black}. Since we make no assumptions about the number of possible set elements, this extension of the traditional feature-vector representation is closely connected to Blum's "infinite attribute" representation. We argue that many decision tree and rule learning algorithms can be easily extended to set-valued features. We also show by example that many real-world learning problems can be efficiently and naturally represented with set-valued features; in particular, text categorization problems and problems that arise in propositionalizing first-order representations lend themselves to set-valued features.

Mining business databases

Abstract: a phenomenal rate. From the financial sector to telecommunications operations , companies increasingly rely on analysis of huge amounts of data to compete. Although ad hoc mixtures of statistical techniques and file management tools once sufficed for digging through mounds of corporate data, the size of modern data warehouses, the mission-critical nature of the data, and the speed with which analyses need to be made now call for a new approach. A new generation of techniques and tools is emerging to intelligently assist humans in analyzing mountains of data and finding critical nuggets of useful knowledge, and in some cases to perform analyses automatically. These techniques and tools are the subject of the growing field of knowledge discovery in databases (KDD) [5]. KDD is an umbrella term describing a variety of activities for making sense of data. We use the term to describe the overall process of finding useful patterns in data, including not only the data mining step of running specific discovery algorithms but also pre-and postprocessing and a host of other important activities. Our goal here is to provide a brief overview of the key issues in knowledge discovery in an industrial context and outline representative applications. The different data mining methods at the core of the KDD process can have different goals. In general, we distinguish two types: • Verification, in which the system is limited to verifying a user's hypothesis, and • Discovery, in which the system finds new patterns. Ad hoc techniques—no longer adequate for sifting through vast collections of data—are giving way to data mining and knowledge discovery for turning corporate data into competitive business advantage.

A Prosodic Analysis of Discourse Segments in Direction-Giving Monologues

Abstract: This paper reports on corpus-based research into the relationship between intonational variation and discourse structure. We examine the effects of speaking style (read versus spontaneous) and of discourse segmentation method (text-alone versus text-and-speech) on the nature of this relationship. We also compare the acoustic-prosodic features of initial, medial, and final utterances in a discourse segment.

Wavelength Division Multiplexing in Long-Haul Transmission Systems

Abstract: The Erbium-Doped Tiber Amplifier (EDFA) has had a profound impact on the design, operation and performance of transoceanic cable transmission, and is central to the expected proliferation of cable systems. Laboratory experiments have demonstrated 100 Gb/s over transoceanic distances using Wavelength Division Multiplexing (WDM) techniques. These large transmission capacity experiments have resulted from an increased understanding of the effects that can limit performance of WDM systems. Important strides have been made in areas of dispersion management, gain equalization, and modulation formats which have made possible the demonstration of large data transmission capacity. This paper reviews experimental techniques developed to improve the performance of long-haul WDM transmission systems based on the Non-Return-to-Zero (NRZ) format, and other non-soliton methods.

Statistical inference and data mining

Abstract: in a database. For many reasons—encoding errors, measurement errors, unrecorded causes of recorded features—the information in a database is almost always noisy; therefore, inference from databases invites applications of the theory of probability. From a statistical point of view, databases are usually uncontrolled convenience samples; therefore data mining poses a collection of interesting, difficult—sometimes impossible—inference problems, raising many issues, some well studied and others unexplored or at least unsettled. Data mining almost always involves a search architecture requiring evaluation of hypotheses at the stages of the search, evaluation of the search output, and appropriate use of the results. Statistics has little to offer in understanding search architectures but a great deal to offer in evaluation of hypotheses in the course of a search, in evaluating the results of a search, and in understanding the appropriate uses of the results. Statistics may have little to offer the search architectures in a data mining search, but a great deal to offer in evaluating hypotheses in the search, in evaluating the results of the search, and in applying the results.

Agent amplified communication

Abstract: We propose an agent-based framework for assisting and simplifying person-to-person communication for information gathering tasks. As an example, we focus on locating experts for any specified topic. In our approach, the informal person-to-person networks that exist within an organization are used to "referral chain" requests for expertise. User-agents help automate this process. The agents generate referrals by analyzing records of email communication patterns. Simulation results show that the higher responsiveness of an agent-based system can be effectively traded for the higher accuracy of a completely manual approach. Furthermore, preliminary experience with a group of users on a prototype system has shown that useful automatic referrals can be found in practice. Our experience with actual users has also shown that privacy concerns are central to the successful deployment of personal agents: an advanced agent-based system will therefore need to reason about issues involving trust and authority.

Multi-modal system for locating heads and faces

Abstract: We designed a modular system using a combination of shape analysis, color segmentation and motion information for locating reliably heads and faces of different sizes and orientations in complex images. The first of the system's three channels does a shape analysis on gray-level images to determine the location of individual facial features as well as the outlines of heads. In the second channel the color space is analyzed with a clustering algorithm to find areas of skin colors. The color space is first calibrated, using the results from the other channels. In the third channel motion information is extracted from frame differences. Head outlines are determined by analyzing the shapes of areas with large motion vectors. All three channels produce lists of shapes, each marking an area of the image where a facial feature or a part of the outline of a head may be present. Combinations of such shapes are evaluated with n-gram searches to produce a list of likely head positions and the locations of facial features. We tested the system for tracking faces of people sitting in front of terminals and video phones and used it to track people entering through a doorway.

Low bit-rate speech coders for multimedia communication

Abstract: The International Telecommunications Union (ITU) has standardized three speech coders which are applicable to low-bit-rate multimedia communications. ITU Rec. G.729 8 kb/s CS-ACELP has a 15 ms algorithmic codec delay and provides network-quality speech. It was originally designed for wireless applications, but is applicable to multimedia communications as well. Annex A of Rec. G.729 is a reduced-complexity version of the CS-ACELP coder. It was designed explicitly for simultaneous voice and data applications that are prevalent in low-bit-rate multimedia communications. These two coders use the same bitstream format and can interoperate. The ITU Rec. G.723.1 6.3 and 5.3 kb/s speech coder for multimedia communications was designed originally for low-bit-rate videophones. Its frame size of 30 ms and one-way algorithmic codec delay of 37.5 ms allow for a further reduction in bit rate compared to the G.729 coder. In applications where low delay is important, the delay of G.723.1 may be too large. However, if the delay is acceptable, G.723.1 provides a lower-complexity alternative to G.729 at the expense of a slight degradation in quality. This article describes the attributes of speech coders such as bit rate, complexity, delay, and quality. Then it discusses the basic concepts of the three new ITU coders by comparing their specific attributes. The second part of this article describes the standardization process for each of these coders.

Quantum Error-Correcting Codes Need Not Completely Reveal the Error Syndrome

Abstract: Quantum error-correcting codes so far proposed have not worked in the presence of noise which introduces more than one bit of entropy per qubit sent through a quantum channel, nor can any code which identifies the complete error syndrome. We describe a code which does not find the complete error syndrome and can be used for reliable transmission of quantum information through channels which add more than one bit of entropy per transmitted bit. In the case of the depolarizing channel our code can be used in a channel of fidelity .8096. The best existing code worked only down to .8107.

Algorithm 769: Fortran subroutines for approximate solution of sparse quadratic assignment problems using GRASP

Abstract: In the NP-complete quadratic assignment problem (QAP), n facilities are to be assigned to n sites at minimum cost. The contribution of assigning facility i to site k and facility j to site l to the total cost is fijdkl, where fij is the flow between facilities i and j, and dkl is the distance between sites k and l. Only very small (n≤20) instances of the QAP have been solved exactly, and heuristics are therefore used to produce approximate solutions. This article describes a set of Fortran subroutines to find approximate solutions to dense quadratic assignment problems, having at least one symmetric flow or distance matrix. A greedy, randomized, adaptive search procedure (GRASP) is used to produce the solutions. The design and implementation of the code are described in detail, and extensive computational experiments are reported, illustrating solution quality as a function of running time.

Predicting a binary sequence almost as well as the optimal biased coin

Abstract: We apply the exponential weight algorithm, introduced and Littlestone and Warmuth [26]and by Vovk [35]to the problem of predicting a binary sequence almost as well as the best biased coin. We first show that for the case of the logarithmic loss, the derived algorithm is equivalent to the Bayes algorithm with Jeffreys prior, that was studied by Xie and Barron [38]under probabilistic assumptions. We derive a uniform bound on the regret which holds for any sequence. We also show that if the empirical distribution of the sequence is bounded away from 0 and from 1, then, as the length of the sequence increases to infinity, the difference between this bound and a corresponding bound on the average case regret of the same algorithm (which is asymptotically optimal in that case) is only 1/2. We show that this gap of 1/2 is necessary by calculating the regret of the min–max optimal algorithm for this problem and showing that the asymptotic upper bound is tight. We also study the application of this algorithm to the square loss and show that the algorithm that is derived in this case is different from the Bayes algorithm and is better than it for prediction in the worstcase.

Demonstration of soliton WDM transmission at 6 and 7 × 10 Gbit/s, error free over transoceanic distances

Abstract: The authors have achieved soliton transmission in six and seven WDM channels of 10 Gbit/s each, ’error free‘ (measured BER < 10-9) over paths of 11.4 and 9.4 Mm, respectively. This result was made possible through the combined use of sliding-frequency filters and dispersion-tapered fibre spans.

Cue Phrase Classification Using Machine Learning

Abstract: Cue phrases may be used in a discourse sense to explicitly signal discourse structure, but also in a sentential sense to convey semantic rather than structural information. Correctly classifying cue phrases as discourse or sentential is critical in natural language processing systems that exploit discourse structure, e.g., for performing tasks such as anaphora resolution and plan recognition. This paper explores the use of machine learning for classifying cue phrases as discourse or sentential. Two machine learning programs (Cgrendel and C4.5) are used to induce classification models from sets of pre-classified cue phrases and their features in text and speech. Machine learning is shown to be an effective technique for not only automating the generation of classification models, but also for improving upon previous results. When compared to manually derived classification models already in the literature, the learned models often perform with higher accuracy and contain new linguistic insights into the data. In addition, the ability to automatically construct classification models makes it easier to comparatively analyze the utility of alternative feature representations of the data. Finally, the ease of retraining makes the learning approach more scalable and flexible than manual methods.

Cue phrase classification using machine learning

Abstract: Cue phrases may be used in a discourse sense to explicitly signal discourse structure, but also in a sentential sense to convey semantic rather than structural information. Correctly classifying cue phrases as discourse or sentential is critical in natural language processing systems that exploit discourse structure, e.g., for performing tasks such as anaphora resolution and plan recognition. This paper explores the use of machine learning for classifying cue phrases as discourse or sentential. Two machine learning programs (cgrendel and C4.5) are used to induce classification models from sets of pre-classified cue phrases and their features in text and speech. Machine learning is shown to be an effective technique for not only automating the generation of classification models, but also for improving upon previous results. When compared to manually derived classification models already in the literature, the learned models often perform with higher accuracy and contain new linguistic insights into the data. In addition, the ability to automatically construct classification models makes it easier to comparatively analyze the utility of alternative feature representations of the data. Finally, the ease of retraining makes the learning approach more scalable and flexible than manual methods.

Simulation run lengths to estimate blocking probabilities

Abstract: We derive formulas approximating the asymptotic variance of four estimators for steady-state blocking probability in a multiserver loss system, exploiting diffusion process limits. These formulas can be used to predict simulation run lengths required to obtain desired statistical precision before the simulation has been run, which can aid in the design of simulation experiments. They also indicate that one estimator can be much better than another, depending on the loading. An indirect estimator based on estimating the mean occupancy is significantly more (less) efficient than a direct estimator for heavy (light) loads. A major concern is the way computational effort scales with system size. For all the estimators, the asymptotic variance tends to be inversely proportional to the system size, so that the computational effort (regarded as proportional to the product of the asymptotic variance and the arrival rate) does not grow as system size increases. Indeed, holding the blocking probability fixed, the computational effort with a good estimator decreases to zero as the system size increases. The asymptotic variance formulas also reveal the impact of the arrival-process and service-time variability on the statistical precision. We validate these formulas by comparing them to exact numerical results for the special case of the classical Erlang M/M/s/0 model and simulation estimates for more general G/GI/s/0 models. It is natural to delete an initial portion of the simulation run to allow the system to approach steady state when it starts out empty. For small to moderately size systems, the time to approach steady state tends to be negligible compared to the time required to obtain good estimates in steady state. However, as the system size increases, the time to approach steady state remains approximately unchanged, or even increases slightly, so that the computational effort associated with letting the system approach steady state becomes a greater portion of the overall computational effort as system size increases.

Forward error control for MPEG-2 video transport in a wireless ATM LAN

Abstract: The performance of error control based on forward error correction (FEC) for MPEG-2 video transmission in an indoor wireless ATM LAN is studied. A multipath fading model is used to investigate the effect of errors on video transport. Combined source and channel coding techniques that employ single layer and scalable MPEG-2 coding to combat channel errors are compared. Simulation results indicate that FEC-based error control in combination with 2-layer video coding techniques can lead to acceptable quality for indoor wireless ATM video.

Challenge problems for artificial intelligence

Abstract: AI textbooks and papers of ten discuss the big questions, such as {open_quotes}how to reason with uncertainty{close_quotes}, {open_quotes}how to reason efficiently{close_quotes}, or {open_quotes}how to improve performance through learning.{close_quotes} It is more difficult, however, to find descriptions of concrete problems or challenges that are still ambitious and interesting, yet not so open-ended. The goal of this panel is to formulate a set of such challenge problems for the field. Each panelist was asked to formulate one or more challenges. The emphasis is on problems for which there is a good chance that they will be resolved within the next five to ten years.

Training Algorithms for Hidden Markov Models using Entropy Based Distance Functions

Abstract: We present new algorithms for parameter estimation of HMMs. By adapting a framework used for supervised learning, we construct iterative algorithms that maximize the likelihood of the observations while also attempting to stay "close" to the current estimated parameters. We use a bound on the relative entropy between the two HMMs as a distance measure between them. The result is new iterative training algorithms which are similar to the EM (Baum-Welch) algorithm for training HMMs. The proposed algorithms are composed of a step similar to the expectation step of Baum-Welch and a new update of the parameters which replaces the maximization (re-estimation) step. The algorithm takes only negligibly more time per iteration and an approximated version uses the same expectation step as Baum-Welch. We evaluate experimentally the new algorithms on synthetic and natural speech pronunciation data. For sparse models, i.e. models with relatively small number of non-zero parameters, the proposed algorithms require significantly fewer iterations.

Interior Point Methods for Combinatorial Optimization

Abstract: Research on using interior point algorithms to solve combinatorial optimization and integer programming problems is surveyed. This paper discusses branch and cut methods for integer programming problems, a potential reduction method based on transforming an integer programming problem to an equivalent nonconvex quadratic programming problem, interior point methods for solving network flow problems, and methods for solving multicommodity flow problems, including an interior point column generation algorithm.

Anonymous credit cards and their collusion analysis

Abstract: Communications networks are traditionally used to bring information together. They can also be used to keep information apart in order to protect personal privacy. A cryptographic protocol specifies a process by which some information is transferred among some users and hidden from others. We show how to implement anonymous credit cards using simple cryptographic protocols. We pose, and solve, a collusion problem which determines whether it is possible for a subset of users to discover information that is designed to be hidden from them during or after execution of the anonymous credit card protocol.

Escalations in workflow management systems

Abstract: Work:flow m.negement systems (WFMSs) have been used to support the modeling, execution, and monitoring of business processes. Business processes consist of multiple activities, and their enactment is carried out by human agents and software systems. "Dypically, business processes and the activities constituting them have deadlines. When an activity misses its deadline, special actions may be triggered, ~efermd to as escalation. E~c.I.Hnns ~d~ect business processes, and they may even lead to the abortion of some of them. Consequently, escalafion~ may entail a high cost to an orgAnlg~tlnU. II1 th~ paper, we present on-going research addressing wod~ow escalations. Our goal is twofold: (a) minimize the rtumbet of escalations during process execution and (b) reduce the cost associated with escalations when they cannot be avoided. 1 I n t r o d u c t i o n Workflow mmn.~ement systems (WFMSs) are increasingly used by orgfmiT~tiOll~ to S _ffem~lin~ alltomato.+ 8nd m~nm~e their business processes. These systemg provide tools to support the modeling of business processes at a conceptual level, coordinate the execution of component activities according to the model, monitor the progress of business processes, and report "important statistiCS" of both the business processes and the systems involved in the execution of them [GHS95, KS95]. WFMSs are based on the concept of a workflow, which is an abstraction of a business process. A workflow consists of activities, which correspond to Process steps, and agents that execute these f i c t i v i t i ~ . The worldiow specification desaibes the activities constituting the wotkflow and the (partial) order in which these activities mnst be executed. Activities may have dcmtline8 that d~t~minc the msxJmum allowable execution time for each of them. When an activity misses its deadline, the workflow model may specify that a special activity, referred to as escalation, be triggered Pc~rmimim m mmk~ digi~Wlmrd c~'h= °f ~U ~ Pm °f this nmm'/d fro" l~mml ~. ~room m m ~ ~i~t~ fm p~x,v/ded tlm ~ oopia m~e not made er diatn'buted fcr l ~ t or w,m,a'ci.I advanlage, the oopynot m,ica, the title of the ~bfic,tioa ,ad ~ d,te apmu', mxt mtice is 0vm em ~ s h t i. by pm.i~im e~the ACM. ~ . To oopy otK~i~., to mlmbi/sh, m po.t oa m'vm or to mlim'a, ut~ to ibm, requirm specific pamia~oa mdior fm DAM" "~6 Rodh~//e MD, USA CC4~p~ght 1997/ACM 0-89791-948-3/96/I l ..$3.50 autonmfically. The effects of an escalation may depend on the sem~mdcs of the activity that missed its dean|he and, often, btmmn intervention is required to proceed. In general, the effects of an esc~letion may be one of the following." (a) the activity that triggered the esc~lmtion is restarted, (b) a new acti~ty is ¢tecuted and the activity that triggered the escalation resumes execution, (c) a new activity replaces the one that Iriggeted the ¢scahtion, or (d) the business process is affected, and it is either aborted as a whole or some of the executed activities are compensated. In all cases, invoking esr~lAfion results in an increased cost for a l~ incss process due to the additional activities t ~ t bare to be executed, or be~-~e completed work is rolled badr. or because intervention of highly-paid workers is required. Therefore, it is desirable to reduce the nnmher of workflow executions that result in ~-~l~t/ons. In thig paper, we address the problem of reducing both the number of worktlow executions that result in ~c~l.Hons and the cost associated with escalations. The main observation behind our technique4g is thmt as long as we guarantee that activities are givtm at least as much time as originally assigned by the l~$iness AnAlyst, we can delay the invocation of escalations for some of them by giving them more time to complete nornmlly. To achieve this, we note mat deadlines 8ire usually assigned to 8ctivifie8 based on the e$fimmod execution times of these activities and on the need to meet the overall _ ~ l i n e of the business process. On the other hand, the actual time required for a partiodar activity to complete varies from one instance to the next due to variations of load, work conditions, etc. Consequently, if an activity in a giveil wofkflow ezC~.ltlon finighe$ faster dum its estimated execution fime~ the rest of the activities can be given extra time before escalation is invoked. This can be accom#ished by extending their deadlines using the available slack time. Although our work ghAlr~s similar goals with research efforts in real-time systems (n~an~ly, that of reduc:ing the number of ~ that miss their de~'Uines), the focus of our work is completely differenL Real-time. systems ~ttempt to minimiTe the number o f mbtsed _dPm'llinc8 by optimizing schoOnling of m.~kg SO that each t~gk better etiliT~ its allowable execution time, which is ass,nned to be fixed. Our work, on the Other hand, attesnpt~ to millimizc the missed