https://www.cs.rice.edu/~nakhleh/COMP571/Fall05/DatabaseSearch/blast.pdf

Basic Local Alignment Search Tool

The BLAST programs are widely used tools for searching protein and DNA databases for sequence similarities. For protein comparisons, a variety of definitional, algorithmic and statistical refinements described here permits the execution time of the BLAST programs to be decreased substantially while enhancing their sensitivity to weak similarities. A new criterion for triggering the extension of word hits, combined with a new heuristic for generating gapped alignments, yields a gapped BLAST program that runs at approximately three times the speed of the original. In addition, a method is introduced for automatically combining statistically significant alignments produced by BLAST into a position-specific score matrix, and searching the database using this matrix. The resulting Position-Specific Iterated BLAST (PSIBLAST) program runs at approximately the same speed per iteration as gapped BLAST, but in many cases is much more sensitive to weak but biologically relevant sequence similarities. PSI-BLAST is used to uncover several new and interesting members of the BRCT superfamily.

/pdf/gapped-blast-and-psi-blast-a-new-generation-of-protein-2ybl67x5yx.pdf

Gapped BLAST and PSI-BLAST: a new generation of protein database search programs.

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

This tutorial provides an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and gives practical details on methods of implementation of the theory along with a description of selected applications of the theory to distinct problems in speech recognition. Results from a number of original sources are combined to provide a single source of acquiring the background required to pursue further this area of research. The author first reviews the theory of discrete Markov chains and shows how the concept of hidden states, where the observation is a probabilistic function of the state, can be used effectively. The theory is illustrated with two simple examples, namely coin-tossing, and the classic balls-in-urns system. Three fundamental problems of HMMs are noted and several practical techniques for solving these problems are given. The various types of HMMs that have been studied, including ergodic as well as left-right models, are described. >

/pdf/a-tutorial-on-hidden-markov-models-and-selected-applications-1vlzbxbk68.pdf

A tutorial on hidden Markov models and selected applications in speech recognition

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

The class of nonnegative definite Toeplitz matrices that can be embedded in nonnegative definite circulant matrices of a larger size is characterized. An equivalent characterization in terms of the spectrum of the underlying process is also presented, together with the corresponding extremal processes. It is shown that a given finite-duration sequence rho can be extended to be the covariance of a periodic stationary processes whenever the Toeplitz matrix R generated by this sequence is strictly positive definite. The sequence rho =1, cos alpha , cos 2 alpha with ( alpha / pi ) irrational, which has a unique nonperiodic extension as a covariance sequence, demonstrates that the strictness is needed. A simple constructive proof supplies a bound on the abovementioned period in terms of the minimal eigenvalue of R. It also yields, under the same conditions, an extension of rho to covariances that eventually decay to zero. For the maximum-likelihood estimate of the covariance of a stationary Gaussian process, the extension length required for using the estimate-maximize iterative algorithm is determined. >

Embedding nonnegative definite Toeplitz matrices in nonnegative definite circulant matrices, with application to covariance estimation

An iterative approach for minimum-discrimination-information (MDI) hidden Markov modeling of information sources is proposed. The approach is developed for sources characterized by a given set of partial covariance matrices and for hidden Markov models (HMMs) with Gaussian autoregressive output probability distributions (PDs). The approach aims at estimating the HMM which yields the MDI with respect to all sources that could have produced the given set of partial covariance matrices. Each iteration of the MDI algorithm generates a new HMM as follows. First, a PD for the source is estimated by minimizing the discrimination information measure with respect to the old model over all PDs which satisfy the given set of partial covariance matrices. Then a new model that decreases the discrimination information measure between the estimated PD of the source and the PD of the old model is developed. The problem of estimating the PD of the source is formulated as a standard constrained minimization problem in the Euclidean space. The estimation of a new model given the PD of the source is done by a procedure that generalizes the Baum algorithm. The MDI approach is shown to be a descent algorithm for the discrimination information measure, and its local convergence is proved. >

A minimum discrimination information approach for hidden Markov modeling

We present a development of parts of rate-distortion theory and pattern-matching algorithms for lossy data compression, centered around a lossy version of the asymptotic equipartition property (AEP). This treatment closely parallels the corresponding development in lossless compression, a point of view that was advanced in an important paper of Wyner and Ziv in 1989. In the lossless case, we review how the AEP underlies the analysis of the Lempel-Ziv algorithm by viewing it as a random code and reducing it to the idealized Shannon code. This also provides information about the redundancy of the Lempel-Ziv algorithm and about the asymptotic behavior of several relevant quantities. In the lossy case, we give various versions of the statement of the generalized AEP and we outline the general methodology of its proof via large deviations. Its relationship with Barron (1985) and Orey's (1985, 1986) generalized AEP is also discussed. The lossy AEP is applied to (i) prove strengthened versions, of Shannon's(1948, 1974) direct source-coding theorem and universal coding theorems; (ii) characterize the performance of "mismatched" codebooks in lossy data compression; ( iii) analyze the performance of pattern-matching algorithms for lossy compression (including Lempel-Ziv schemes); and (iv) determine the first-order asymptotic of waiting times between stationary processes. A refinement to the lossy AEP is then presented, and it is used to (i) prove second-order (direct and converse) lossy source-coding theorems, including universal coding theorems; (ii) characterize which sources are quantitatively easier to compress; (iii) determine the second-order asymptotic of waiting times between stationary processes; and (iv) determine the precise asymptotic behavior of longest match-lengths between stationary processes. Finally, we discuss extensions of the above framework and results to random fields.

/pdf/source-coding-large-deviations-and-approximate-pattern-6e53bst7tz.pdf

Source coding, large deviations and approximate pattern matching

Let $X_i$ be positive i.i.d. random variables (or more generally a uniformly mixing positive-valued ergodic stationary process). The $r$-scan process induced by $\{X_i\}$ is $R_i = \sum^{i+r-1}_{k=i} X_k, i = 1, 2, \ldots, n - r + 1$. Limiting distributions for the extremal order statistics among $\{R_i\}$ suitably normalized (and appropriate threshold values $a = a_n$ and $b = b_n$) are derived as a consequence of Poisson approximations to the Bernoulli sums $N^-(a) = \sum^{n+r-1}_{i=1} W^-_i(a)$ and $N^+(b) = \sum^{n-r+1}_{i=1}W^+_i(b)$, where $W^-_i(a) \lbrack W^+_i(b) \rbrack = 1$ or 0 according as $R_i \leq a (R_i > b)$ occurs or not. Applications include limit theorems for $r$-spacings based on i.i.d. uniform $\lbrack 0, 1 \rbrack$ r.v.'s, for extremal $r$-spacings based on i.i.d. samples from a general density and for the $r$-scan process with a variable time horizon.

/pdf/poisson-approximations-for-r-scan-processes-5z74ailki0.pdf

Poisson Approximations for $r$-Scan Processes

For Erd\H{o}s-R\'enyi random graphs with average degree $\gamma$, and uniformly random $\gamma$-regular graph on $n$ vertices, we prove that with high probability the size of both the Max-Cut and maximum bisection are $n\Big(\frac{\gamma}{4} + {{\sf P}}_* \sqrt{\frac{\gamma}{4}} + o(\sqrt{\gamma})\Big) + o(n)$ while the size of the minimum bisection is $n\Big(\frac{\gamma}{4}-{{\sf P}}_*\sqrt{\frac{\gamma}{4}} + o(\sqrt{\gamma})\Big) + o(n)$. Our derivation relates the free energy of the anti-ferromagnetic Ising model on such graphs to that of the Sherrington-Kirkpatrick model, with ${{\sf P}}_* \approx 0.7632$ standing for the ground state energy of the latter, expressed analytically via Parisi's formula.

Amir Dembo

Papers

Embedding nonnegative definite Toeplitz matrices in nonnegative definite circulant matrices, with application to covariance estimation

A minimum discrimination information approach for hidden Markov modeling

Source coding, large deviations and approximate pattern matching

Poisson Approximations for $r$-Scan Processes

Extremal Cuts of Sparse Random Graphs