Boolean functions are perhaps the most basic objects of study in theoretical computer science. They also arise in other areas of mathematics, including combinatorics, statistical physics, and mathematical social choice. The field of analysis of Boolean functions seeks to understand them via their Fourier transform and other analytic methods. This text gives a thorough overview of the field, beginning with the most basic definitions and proceeding to advanced topics such as hypercontractivity and isoperimetry. Each chapter includes a "highlight application" such as Arrow's theorem from economics, the Goldreich-Levin algorithm from cryptography/learning theory, Hstad's NP-hardness of approximation results, and "sharp threshold" theorems for random graph properties. The book includes roughly 450 exercises and can be used as the basis of a one-semester graduate course. It should appeal to advanced undergraduates, graduate students, and researchers in computer science theory and related mathematical fields.

/pdf/analysis-of-boolean-functions-4kueie1j8m.pdf

Analysis of Boolean Functions

Analysis of Boolean Functions: List of Notation

Publisher Summary This chapter discusses algebraic complexity theory. Complexity theory, as a project of lower bounds and optimality, unites two quite different traditions. The first comes from mathematical logic and the theory of recursive functions. In this, the basic computational model is the Turing machine. The second tradition has developed from questions of numerical algebra. The problems in this typically have a fixed finite size. Consequently, the computational model is based on something like an ordinary computer that however is supplied with the ability to perform any arithmetic operation with infinite precision and that in turn is required to deliver exact results. The formal model is that of straight-line program or arithmetic circuit or computation sequence, more generally that of computation tree.

Algebraic Complexity Theory.

Learning problems form an important category of computational tasks that generalizes many of the computations researchers apply to large real-life data sets. We ask: what concept classes can be learned privately, namely, by an algorithm whose output does not depend too heavily on any one input or specific training example? More precisely, we investigate learning algorithms that satisfy differential privacy, a notion that provides strong confidentiality guarantees in contexts where aggregate information is released about a database containing sensitive information about individuals. We demonstrate that, ignoring computational constraints, it is possible to privately agnostically learn any concept class using a sample size approximately logarithmic in the cardinality of the concept class. Therefore, almost anything learnable is learnable privately: specifically, if a concept class is learnable by a (non-private) algorithm with polynomial sample complexity and output size, then it can be learned privately using a polynomial number of samples. We also present a computationally efficient private PAC learner for the class of parity functions. Local (or randomized response) algorithms are a practical class of private algorithms that have received extensive investigation. We provide a precise characterization of local private learning algorithms. We show that a concept class is learnable by a local algorithm if and only if it is learnable in the statistical query (SQ) model. Finally, we present a separation between the power of interactive and noninteractive local learning algorithms.

What Can We Learn Privately

Tensor rank is NP-complete

In this paper, monotone Boolean functions are studied using harmonic analysis on the cube. The main result is that any monotone Boolean function has most of its power spectrum on its Fourier coefficients of degree at most O(√n) under any product distribution. This is similar to a result of Linial et al. [1993], which showed that AC 0 functions have almost all of their power spectrum on the coefficients of degree, at most (log n) O(1) , under the uniform distribution. As a consequence of the main result, the following two corollaries are obtained : -For any e > 0, monotone Boolean functions are PAC learnable with error e under product distributions in time 2 O((1/e) √ n) . -Any monotone Boolean function can be approximated within error e under product distributions by a non-monotone Boolean circuit of size 2 O(1/e √ n) and depth O(1/e √n). The learning algorithm runs in time subexponential as long as the required error is Ω(1/(√n log n)). It is shown that this is tight in the sense that for any subexponential time algorithm there is a monotone Boolean function for which this algorithm cannot approximate with error better than O(1/√n). The main result is also applied to other problems in learning and complexity theory. In learning theory, several polynomial-time algorithms for learning some classes of monotone Boolean functions, such as Boolean functions with O(log 2 n/log log n) relevant variables, are presented. In complexity theory, some questions regarding monotone NP-complete problems are addressed.

/pdf/on-the-fourier-spectrum-of-monotone-functions-1mdwi1vqr2.pdf

On the Fourier spectrum of monotone functions

The Kushilevitz-Mansour (KM) algorithm is an algorithm that finds all the "large" Fourier coefficients of a Boolean function. It is the main tool for learning decision trees and DNF expressions in the PAC model with respect to the uniform distribution. The algorithm requires access to the membership query (MQ) oracle. The access is often unavailable in learning applications and thus the KM algorithm cannot be used. We significantly weaken this requirement by producing an analogue of the KM algorithm that uses extended statistical queries (SQ) (SQs in which the expectation is taken with respect to a distribution given by a learning algorithm). We restrict a set of distributions that a learning algorithm may use for its statistical queries to be a set of product distributions with each bit being 1 with probability ρ, 1/2 or 1-ρ for a constant 1/2 > ρ > 0 (we denote the resulting model by SQ-Dρ). Our analogue finds all the "large" Fourier coefficients of degree lower than clog(n) (we call it the Bounded Sieve (BS)). We use BS to learn decision trees and by adapting Freund's boosting technique we give an algorithm that learns DNF in SQ-Dρ. An important property of the model is that its algorithms can be simulated by MQs with persistent noise. With some modifications BS can also be simulated by MQs with product attribute noise (i.e., for a query x oracle changes every bit of x with some constant probability and calculates the value of the target function at the resulting point) and classification noise. This implies learnability of decision trees and weak learnability of DNF with this non-trivial noise. In the second part of this paper we develop a characterization for learnability with these extended statistical queries. We show that our characterization when applied to SQ-Dρ is tight in terms of learning parity functions. We extend the result given by Blum et al. by proving that there is a class learnable in the PAC model with random classification noise and not learnable in SQ-Dρ.

/pdf/on-using-extended-statistical-queries-to-avoid-membership-nplkz7c6mk.pdf

On using extended statistical queries to avoid membership queries

On the Fourier spectrum of Monotone Functions

The Kushilevitz-Mansour (KM)algorithm is an algorithm that finds all the "heavy" Fourier coefficients of a boolean function. It is the main tool for learning decision trees and DNF expressions in the PAC model with respect to the uniform distribution. The algorithm requires an access to the membership query (MQ)oracle.

We weaken this requirement by producing an analogue of the KM algorithm that uses extended statistical queries (SQ)(SQs in which the expectation is taken with respect to a distribution given by a learning algorithm). We restrict a set of distributions that a learning algorithm may use for its SQs to be a set of specific constant bounded product distributions. Our analogue finds all the "heavy" Fourier coefficients of degree lower than c log n (we call it BS). We use BS to learn decision trees and by adapting Freund's boosting technique we give algorithm that learns DNF in this model. Learning in this model implies learning with persistent classification noise and in some cases can be extended to learning with product attribute noise.

We develop a characterization for learnability with these extended SQs and apply it to get several negative results about the model.

On Using Extended Statistical Queries to Avoid Membership Queries

In this paper, monotone Boolean functions are studied using harmonic analysis on the cube. The main result is that any monotone Boolean function has most of its power spectrum on its Fourier coefficients of “degree” at most O(=n) under any product distribution. This is similar to a result of Linial et al. [1993], which showed that AC functions have almost all of their power spectrum on the coefficients of degree, at most (log n), under the uniform distribution. As a consequence of the main result, the following two corollaries are obtained: —For any e . 0, monotone Boolean functions are PAC learnable with error e under product distributions in time 2O((1/e)n). —Any monotone Boolean function can be approximated within error e under product distributions by a non-monotone Boolean circuit of size 2O(1/en) and depth O(1/e =n). The learning algorithm runs in time subexponential as long as the required error is V(1/(=n log n)). It is shown that this is tight in the sense that for any subexponential time algorithm there is a monotone Boolean function for which this algorithm cannot approximate with error better than O(1/=n). The main result is also applied to other problems in learning and complexity theory. In learning theory, several polynomial-time algorithms for learning some classes of monotone Boolean functions, such as Boolean functions with O(logn/log log n) relevant variables, are presented. In complexity theory, some questions regarding monotone NP-complete problems are addressed.

/pdf/on-the-fourier-spectrum-of-monotone-functions-16vx9m04sv.pdf

Nader H. Bshouty

Papers

On the Fourier spectrum of monotone functions

On using extended statistical queries to avoid membership queries

On the Fourier spectrum of Monotone Functions

On Using Extended Statistical Queries to Avoid Membership Queries

On the Fourier spectrum of monotone functions