The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances — including the key problems of computing marginals and modes of probability distributions — are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide variety of algorithms — among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations — can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

/pdf/graphical-models-exponential-families-and-variational-3i6dwlfta1.pdf

Graphical Models, Exponential Families, and Variational Inference

https://leeds-faculty.colorado.edu/glover/TS%20-%20Future%20Paths%20for%20Integer%20Programming.pdf

Future paths for integer programming and links to artificial intelligence

ALDH1 is a marker of normal and malignant human mammary stem cells and a predictor of poor clinical outcome.

‡We describe the use of singular value decomposition in transforming genome-wide expression data from genes 3 arrays space to reduced diagonalized ‘‘eigengenes’’ 3 ‘‘eigenarrays’’ space, where the eigengenes (or eigenarrays) are unique orthonormal superpositions of the genes (or arrays). Normalizing the data by filtering out the eigengenes (and eigenarrays) that are inferred to represent noise or experimental artifacts enables meaningful comparison of the expression of different genes across different arrays in different experiments. Sorting the data according to the eigengenes and eigenarrays gives a global picture of the dynamics of gene expression, in which individual genes and arrays appear to be classified into groups of similar regulation and function, or similar cellular state and biological phenotype, respectively. After normalization and sorting, the significant eigengenes and eigenarrays can be associated with observed genome-wide effects of regulators, or with measured samples, in which these regulators are overactive or underactive, respectively.

Singular Value Decomposition for Genome-Wide Expression Data Processing and Modeling

Our personal social networks are big and cluttered, and currently there is no good way to organize them. Social networking sites allow users to manually categorize their friends into social circles (e.g. 'circles' on Google+, and 'lists' on Facebook and Twitter), however they are laborious to construct and must be updated whenever a user's network grows. We define a novel machine learning task of identifying users' social circles. We pose the problem as a node clustering problem on a user's ego-network, a network of connections between her friends. We develop a model for detecting circles that combines network structure as well as user profile information. For each circle we learn its members and the circle-specific user profile similarity metric. Modeling node membership to multiple circles allows us to detect overlapping as well as hierarchically nested circles. Experiments show that our model accurately identifies circles on a diverse set of data from Facebook, Google+, and Twitter for all of which we obtain hand-labeled ground-truth.

/pdf/learning-to-discover-social-circles-in-ego-networks-b679dv5qjl.pdf

Learning to Discover Social Circles in Ego Networks

Pseudo-boolean optimization

Given an m $\times$ n zero-one matrix $\underset\tilde\to A$ we ask whether there is a single linear inequality $\underset\tilde\to a \underset\tilde\to x \leq b$ whose zero-one solutions are precisely the zero-one solutions of $\underset\tilde\to A \underset\tilde\to x \leq e$. We develop an algorithm for answering this question in O(m$n^2$) steps and investigate other related problems. Our results may be interpreted in terms of graph theory and threshold logic.

Aggregation of inequalities in integer programming.

: Significant progress has been made with solution of location problems and in preprocessing and decomposition for discrete optimization. There has also been research on the application of techniques from combinational optimization to nonlinear problems.

http://www.dtic.mil/dtic/tr/fulltext/u2/a273552.pdf

Discrete Applied Mathematics

Written by prominent experts in the field, this monograph provides the first comprehensive, unified presentation of the structural, algorithmic and applied aspects of the theory of Boolean functions. The book focuses on algebraic representations of Boolean functions, especially disjunctive and conjunctive normal form representations. This framework looks at the fundamental elements of the theory (Boolean equations and satisfiability problems, prime implicants and associated short representations, dualization), an in-depth study of special classes of Boolean functions (quadratic, Horn, shellable, regular, threshold, read-once functions and their characterization by functional equations) and two fruitful generalizations of the concept of Boolean functions (partially defined functions and pseudo-Boolean functions). Several topics are presented here in book form for the first time. Because of the depth and breadth and its emphasis on algorithms and applications, this monograph will have special appeal for researchers and graduate students in discrete mathematics, operations research, computer science, engineering and economics.

/pdf/boolean-functions-theory-algorithms-and-applications-1n8c4ekj1n.pdf

Boolean Functions: Theory, Algorithms, and Applications

Describes a new, logic-based methodology for analyzing observations. The key features of this "logical analysis of data" (LAD) methodology are the discovery of minimal sets of features that are necessary for explaining all observations and the detection of hidden patterns in the data that are capable of distinguishing observations describing "positive" outcome events from "negative" outcome events. Combinations of such patterns are used for developing general classification procedures. An implementation of this methodology is described in this paper, along with the results of numerical experiments demonstrating the classification performance of LAD in comparison with the reported results of other procedures. In the final section, we describe three pilot studies on applications of LAD to oil exploration, psychometric testing and the analysis of developments in the Chinese transitional economy. These pilot studies demonstrate not only the classification power of LAD but also its flexibility and capability to provide solutions to various case-dependent problems.

/pdf/an-implementation-of-logical-analysis-of-data-4a21vgxfgc.pdf

Peter L. Hammer

Papers

Pseudo-boolean optimization

Aggregation of inequalities in integer programming.

Discrete Applied Mathematics

Boolean Functions: Theory, Algorithms, and Applications

An implementation of logical analysis of data