Showing papers on "Average-case complexity published in 2014"
Book•
01 May 2014
TL;DR: This book discusses models of Computation and Complexity Classes, Probabilistically Checkable Proofs and NP-Hard Optimization Problems, and the structure of NP.
Abstract: UNIFORM COMPLEXITY. Models of Computation and Complexity Classes. NP-Completeness. The Polynomial-Time Hierarchy and Polynomial Space. Structure of NP. NONUNIFORM COMPLEXITY. Decision Trees. Circuit Complexity. Polynomial-Time Isomorphism. PROBABILISTIC COMPLEXITY. Probabilistic Machines and Complexity Classes. Complexity of Counting. Interactive Proof Systems. Probabilistically Checkable Proofs and NP-Hard Optimization Problems. Bibliography. Index.
290 citations
31 May 2014
TL;DR: In this article, the authors introduce a new technique for proving hardness of improper learning, based on reductions from problems that are hard on average, which is a generalization of Feige's assumption about the complexity of refuting random constraint satisfaction problems.
Abstract: The basic problem in the PAC model of computational learning theory is to determine which hypothesis classes are effficiently learnable. There is presently a dearth of results showing hardness of learning problems. Moreover, the existing lower bounds fall short of the best known algorithms. The biggest challenge in proving complexity results is to establish hardness of improper learning (a.k.a. representation independent learning). The difficulty in proving lower bounds for improper learning is that the standard reductions from NP-hard problems do not seem to apply in this context. There is essentially only one known approach to proving lower bounds on improper learning. It was initiated in [21] and relies on cryptographic assumptions. We introduce a new technique for proving hardness of improper learning, based on reductions from problems that are hard on average. We put forward a (fairly strong) generalization of Feige's assumption [13] about the complexity of refuting random constraint satisfaction problems. Combining this assumption with our new technique yields far reaching implications. In particular, • Learning DNF's is hard. • Agnostically learning halfspaces with a constant approximation ratio is hard. • Learning an intersection of ω(1) halfspaces is hard.
107 citations
14 Nov 2014
TL;DR: An attempt is made to develop an O(n) complexity (linear order) counterpart of the k-means, which includes a directional movement of intermediate clusters and thereby improves compactness and separability properties of cluster structures simultaneously.
Abstract: The k-means algorithm is known to have a time complexity of O (n 2), where n is the input data size. This quadratic complexity debars the algorithm from being effectively used in large applications. In this article, an attempt is made to develop an O (n) complexity (linear order) counterpart of the k-means. The underlying modification includes a directional movement of intermediate clusters and thereby improves compactness and separability properties of cluster structures simultaneously. This process also results in an improved visualization of clustered data. Comparison of results obtained with the classical k-means and the present algorithm indicates usefulness of the new approach.
101 citations
18 Oct 2014
TL;DR: In this article, the authors show an exponential gap between communication complexity and information complexity, by giving an explicit example for a communication task (relation), with information complexity ≤ O(k) and distributional communication complexity ≥ 2k.
Abstract: We show an exponential gap between communication complexity and information complexity, by giving an explicit example for a communication task (relation), with information complexity ≤ O(k), and distributional communication complexity ≥ 2k. This shows that a communication protocol cannot always be compressed to its internal information. By a result of Braverman [1], our gap is the largest possible. By a result of Braverman and Rao [2], our example shows a gap between communication complexity and amortized communication complexity, implying that a tight direct sum result for distributional communication complexity cannot hold.
59 citations
Posted Content•
TL;DR: It is shown that Shannon's information entropy, compressibility and algorithmic complexity quantify different local and global aspects of synthetic and biological data, and it is proved that the Kolmogorov complexity of a labeled graph is a good approximation of its unlabeled Kolmogsorv complexity and thus a robust definition of graph complexity.
Abstract: We survey and introduce concepts and tools located at the intersection of information theory and network biology. We show that Shannon's information entropy, compressibility and algorithmic complexity quantify different local and global aspects of synthetic and biological data. We show examples such as the emergence of giant components in Erdos-Renyi random graphs, and the recovery of topological properties from numerical kinetic properties simulating gene expression data. We provide exact theoretical calculations, numerical approximations and error estimations of entropy, algorithmic probability and Kolmogorov complexity for different types of graphs, characterizing their variant and invariant properties. We introduce formal definitions of complexity for both labeled and unlabeled graphs and prove that the Kolmogorov complexity of a labeled graph is a good approximation of its unlabeled Kolmogorov complexity and thus a robust definition of graph complexity.
53 citations
TL;DR: It is shown that no monotone circuit of size $O(n^{k/4})$ solves the k-clique problem with high probability on $\ER(n,p)$ for two sufficiently far-apart threshold functions $p(n)$ and $2n^{-2/(k-1)}$.
Abstract: We present lower and upper bounds showing that the average-case complexity of the $k$-Clique problem on monotone circuits is $n^{k/4 + O(1)}$. Similar bounds for $\mathsf{AC}^0$ circuits were shown in Rossman [Proceedings of the 40th Annual ACM Symposium on Theory of Computing, 2008, pp. 721--730] and Amano [Comput. Complexity, 19 (2010), pp. 183--210].
43 citations
Posted Content•
TL;DR: A new characteristic of a regular ideal language called reset complexity is presented and some bounds on the reset complexity in terms of the state complexity of a given language are found.
Abstract: We present a new characteristic of a regular ideal language called reset complexity We find some bounds on the reset complexity in terms of the state complexity of a given language We also compare the reset complexity and the state complexity for languages related to slowly synchronizing automata and study uniqueness question for automata yielding the minimum of reset complexity
30 citations
TL;DR: A new measure of complexity based on quantum, known asepsilon-machines, is proposed and applied to a simple system undergoing constant thermalization, which aligns more closely with the intuition of how complexity should behave.
Abstract: While we have intuitive notions of structure and complexity, the formalization of this intuition is non-trivial. The statistical complexity is a popular candidate. It is based on the idea that the complexity of a process can be quantified by the complexity of its simplest mathematical model —the model that requires the least past information for optimal future prediction. Here we review how such models, known as $ \epsilon$
-machines can be further simplified through quantum logic, and explore the resulting consequences for understanding complexity. In particular, we propose a new measure of complexity based on quantum $ \epsilon$
-machines. We apply this to a simple system undergoing constant thermalization. The resulting quantum measure of complexity aligns more closely with our intuition of how complexity should behave.
28 citations
TL;DR: In this article, an efficient quantum-adiabatic algorithm for solving Simon's problem was proposed, which is exponentially faster than its classical counterpart and has the same complexity as the corresponding circuit-based algorithm.
Abstract: We outline an efficient quantum-adiabatic algorithm that solves Simon's problem, in which one has to determine the “period”, or xor mask, of a given black-box function. We show that the proposed algorithm is exponentially faster than its classical counterpart and has the same complexity as the corresponding circuit-based algorithm. Together with other related studies, this result supports a conjecture that the complexity of adiabatic quantum computation is equivalent to the circuit-based computational model in a stronger sense than the well-known, proven polynomial equivalence between the two paradigms. We also discuss the importance of the algorithm and its theoretical and experimental implications for the existence of an adiabatic version of Shor's integer factorization algorithm that would have the same complexity as the original algorithm.
28 citations
TL;DR: It turns out that, asymptotically, and in the average case, the complexity gap between the several constructions is significantly larger than in the worst case.
22 citations
Posted Content•
TL;DR: This work shows that (the logarithm to the base two of) its communication complexity and query complexity versions form, for all relations, a quadratically tight lower bound on the public-coin randomized communication complexity or randomized query complexity respectively.
Abstract: In this work we introduce, both for classical communication complexity and query complexity, a modification of the 'partition bound' introduced by Jain and Klauck [2010]. We call it the 'public-coin partition bound'. We show that (the logarithm to the base two of) its communication complexity and query complexity versions form, for all relations, a quadratically tight lower bound on the public-coin randomized communication complexity and randomized query complexity respectively.
31 Mar 2014
TL;DR: It is shown that conjugacy in G 1,2 can be solved in polynomial time in a strongly generic setting and under a plausible assumption the average case complexity of the same problem is non-elementary.
Abstract: The conjugacy problem is the following question: given two words x, y over generators of a fixed group G, decide whether x and y are conjugated, i.e., whether there exists some z such that zx z − 1 = y in G. The conjugacy problem is more difficult than the word problem, in general. We investigate the conjugacy problem for two prominent groups: the Baumslag-Solitar group BS 1,2 and the Baumslag(-Gersten) group G 1,2. The conjugacy problem in BS 1,2 is TC 0-complete. To the best of our knowledge BS 1,2 is the first natural infinite non-commutative group where such a precise and low complexity is shown. The Baumslag group G 1,2 is an HNN-extension of BS 1,2 and its conjugacy problem is decidable G 1,2 by a result of Beese (2012). Here we show that conjugacy in G 1,2 can be solved in polynomial time in a strongly generic setting. This means that essentially for all inputs conjugacy in G 1,2 can be decided efficiently. In contrast, we show that under a plausible assumption the average case complexity of the same problem is non-elementary. Moreover, we provide a lower bound for the conjugacy problem in G 1,2 by reducing the division problem in power circuits to the conjugacy problem in G 1,2. The complexity of the division problem in power circuits is an open and interesting problem in integer arithmetic.
TL;DR: New results are derived, which allow computing an admissible range for the Rademacher complexity, given a value of the VC-entropy, and vice versa, which improve the state of the art on this research topic.
Abstract: In this paper, we derive a deep connection between the Vapnik-Chervonenkis (VC) entropy and the Rademacher complexity. For this purpose, we first refine some previously known relationships between the two notions of complexity and then derive new results, which allow computing an admissible range for the Rademacher complexity, given a value of the VC-entropy, and vice versa. The approach adopted in this paper is new and relies on the careful analysis of the combinatorial nature of the problem. The obtained results improve the state of the art on this research topic.
TL;DR: For two different formulations of an NP-hard subclass of the well-known PARTITION problem, it is shown that also the unary unbiased black-box complexity does not give a complete picture of the true difficulty of this problem for randomized search heuristics.
TL;DR: This paper focuses on the case of norms over C[0,1] and introduces the notion of dependence of a norm on a point and relate it to the query complexity of the norm, and shows that the dependence of almost every point is of the order of the query simplicity of thenorm.
12 Jul 2014
TL;DR: The results of the study indicate that the simplest configuration, in terms of operator complexity, consistently results in the best average performance, and in many cases, the result is significantly better.
Abstract: For some time, Genetic Programming research has lagged behind the wider Machine Learning community in the study of generalisation, where the decomposition of generalisation error into bias and variance components is well understood. However, recent Genetic Programming contributions focusing on complexity, size and bloat as they relate to over-fitting have opened up some interesting avenues of research. In this paper, we carry out a simple empirical study on five binary classification problems. The study is designed to discover what effects may be observed when program size and complexity are varied in combination, with the objective of gaining a better understanding of relationships which may exist between solution size, operator complexity and variance error. The results of the study indicate that the simplest configuration, in terms of operator complexity, consistently results in the best average performance, and in many cases, the result is significantly better. We further demonstrate that the best results are achieved when this minimum complexity set-up is combined with a less than parsimonious permissible size.
TL;DR: An exponential lower bound on the average time of inverting Goldreich’s function by drunken backtracking algorithms is proved; this resolves the open question stated in Cook et al.
Abstract: We prove an exponential lower bound on the average time of inverting Goldreich's function by drunken backtracking algorithms; this resolves the open question stated in Cook et al. (Proceedings of TCC, pp. 521---538, 2009). The Goldreich's function has n binary inputs and n binary outputs. Every output depends on d inputs and is computed from them by the fixed predicate of arity d. Our Goldreich's function is based on an expander graph and on the nonlinear predicates that are linear in Ω(d) variables. Drunken algorithm is a backtracking algorithm that somehow chooses a variable for splitting and randomly chooses the value for the variable to be investigated at first.
After the submission to the journal we found out that the same result was independently obtained by Rachel Miller.
01 Sep 2014
TL;DR: An implicit characterization of the complexity classes k-EXP and k-FEXP, for k ≥ 0, is given, by a type assignment system for a stratified λ-calculus, where types for programs are witnesses of the corresponding complexity class.
Abstract: In this paper an implicit characterization of the complexity classes k-EXP and k-FEXP, for k ≥ 0, is given, by a type assignment system for a stratified λ-calculus, where types for programs are witnesses of the corresponding complexity class. Types are formulae of Elementary Linear Logic (ELL), and the hierarchy of complexity classes k-EXP is characterized by a hierarchy of types.
01 Jan 2014
TL;DR: Probabilistic computable functions can be characterized by a natural generalization of Church and Kleene’s partial recursive functions, and the obtained algebra can be restricted so as to capture the notion of a polytime sampleable distribution, a key concept in average-case complexity and cryptography.
Abstract: We show that probabilistic computable functions, i.e., those functions outputting distributions and computed by probabilistic Turing machines, can be characterized by a natural generalization of Church and Kleene’s partial recursive functions. The obtained algebra, following Leivant, can be restricted so as to capture the notion of a polytime sampleable distribution, a key concept in average-case complexity and cryptography.
Posted Content•
TL;DR: Two natural parameterized analogs of efficient average-case algorithms are defined and it is shown that k-Clique?admits both analogues for Erd?s-Renyi random graphs of arbitrary density and is unlikely to admit either of these analogs for some specific computable input distribution.
Abstract: The k-Clique problem is a fundamental combinatorial problem that plays a prominent role in classical as well as in parameterized complexity theory. It is among the most well-known NP-complete and W[1]-complete problems. Moreover, its average-case complexity analysis has created a long thread of research already since the 1970s. Here, we continue this line of research by studying the dependence of the average-case complexity of the k-Clique problem on the parameter k. To this end, we define two natural parameterized analogs of efficient average-case algorithms. We then show that k-Clique admits both analogues for Erdős-Renyi random graphs of arbitrary density. We also show that k-Clique is unlikely to admit neither of these analogs for some specific computable input distribution.
TL;DR: Flott is presented, a fast, low memory T-transform algorithm which can be used to compute the string complexity measure T-complexity, which is another measure of string complexity.
Abstract: This paper presents flott, a fast, low memory T-transform algorithm which can be used to compute the string complexity measure T-complexity. The algorithm uses approximately one third of the memory of its predecessor while reducing the running time by about 20 percent. The flott implementation has the same worst-case memory requirements as state of the art suffix tree construction algorithms. A suffix tree can be used to efficiently compute the Lempel-Ziv production complexity, which is another measure of string complexity. The C-implementation of flott is available as Open Source software.
Posted Content•
TL;DR: These are the first results on the average-case complexity of pattern matching with wildcards which, as a by product, provide with first provable separation in complexity between exact pattern matching and pattern matchingwith wildcards in the word RAM model.
Abstract: Pattern matching with wildcards is the problem of finding all factors of a text $t$ of length $n$ that match a pattern $x$ of length $m$, where wildcards (characters that match everything) may be present. In this paper we present a number of fast average-case algorithms for pattern matching where wildcards are restricted to either the pattern or the text, however, the results are easily adapted to the case where wildcards are allowed in both. We analyse the \textit{average-case} complexity of these algorithms and show the first non-trivial time bounds. These are the first results on the average-case complexity of pattern matching with wildcards which, as a by product, provide with first provable separation in complexity between exact pattern matching and pattern matching with wildcards in the word RAM model.
23 Jul 2014
TL;DR: This tutorial will give an overview of algebraic complexity theory focused on bilinear complexity, and describe several powerful techniques to analyze the complexity of computational problems from linear algebra, in particular matrix multiplication.
Abstract: This tutorial will give an overview of algebraic complexity theory focused on bilinear complexity, and describe several powerful techniques to analyze the complexity of computational problems from linear algebra, in particular matrix multiplication. The presentation of these techniques will follow the history of progress on constructing asymptotically fast algorithms for matrix multiplication, and include its most recent developments.
01 Dec 2014
TL;DR: A new (4,2)-way Toom-Cook algorithm using finite field interpolation to construct a digit-serial multiplier over GF(2m) which involves subquadratic space-complexity.
Abstract: In this paper, we present a new (4,2)-way Toom-Cook algorithm using finite field interpolation. The proposed algorithm uses multi-evaluation scheme to construct a digit-serial multiplier over GF(2m) which involves subquadratic space-complexity. From theoretical analysis, it is found that the proposed architecture has O(mlog4 5) space complexity and O(mlog4 2) latency, which is significantly less than traditional digit-serial multipliers.
10 Mar 2014
TL;DR: Novel arithmetic algorithms on a canonical number representation based on the Catalan family of combinatorial objects provide super-exponential gains while their average case complexity is within constant factors of their traditional counterparts.
Abstract: We study novel arithmetic algorithms on a canonical number representation based on the Catalan family of combinatorial objects.
For numbers corresponding to Catalan objects of low structural complexity our algorithms provide super-exponential gains while their average case complexity is within constant factors of their traditional counterparts.
TL;DR: It is shown that when the authors restrict ourselves to quantified conjunctions of linear inequalities, i.e., quantified linear systems, the complexity classes collapse to polynomial time, which reinforces the importance of sentence formats from the perspective of computational complexity.
01 Jun 2014
TL;DR: The sample complexity bound to O(n) measurements for sufficiently large n is improved, using a variant of Matrix Bernstein concentration inequality that exploits the intrinsic dimension, together with properties of one bit phase retrieval.
Abstract: In this paper we show that the problem of phase retrieval can be efficiently and provably solved via an alternating minimization algorithm suitably initialized. Our initialization is based on One Bit Phase Retrieval that we introduced in [1], where we showed that O(n log(n)) Gaussian phase-less measurements ensure robust recovery of the phase. In this paper we improve the sample complexity bound to O(n) measurements for sufficiently large n, using a variant of Matrix Bernstein concentration inequality that exploits the intrinsic dimension, together with properties of one bit phase retrieval.
14 Jul 2014
TL;DR: This paper introduces and analyzes some fundamental concepts of computational complexity, and discusses complete problems of time complexity and space complexity by examples; and the relation among complexity classes is analyzed in detail.
Abstract: Computational complexity is a branch of the theory of computation. It is used to measure how hard a problem is solved and the common measures include time and space. The classes of time complexity generally include: P, NP, NP-hard, NP-complete and EXPTIME; the classes of space complexity generally include: PSPACE, NPSPACE, PSPACE-hard and PSPACE-complete. Researching computational complexity of a problem can make it explicit whether there is an effective solving algorithm of the problem or not. This paper introduces and analyzes some fundamental concepts of computational complexity, and discusses complete problems of time complexity and space complexity by examples; What's more, the relation among complexity classes is analyzed in detail.
08 Sep 2014
TL;DR: The used criteria are adequate for predicting the mean execution time and its confidence intervals for given input types in an algorithm complexity as a random value research.
Abstract: A statistical research of an algorithm complexity as a random value was carried out via numerical experimentation using parallel computation. For a segment of input data sizes point characteristics for this random value and its confidence interval are obtained. Confidence complexity function value based on gamma-distribution is determined. The following result have been obtained: the used criteria are adequate for predicting the mean execution time and its confidence intervals for given input types.
TL;DR: An article on codiagnosability verification of discrete event systems was reported by Moreira et al, claiming an improvement in complexity over a paper by Qiu and Kumar, but the results were obtained in a more restricted setting of “projection mask”, in contrast to the more general “non-projection masks” allowed in Qiu & Kumar's paper.
Abstract: An article on codiagnosability verification of discrete event systems was reported by Moreira et al, claiming an improvement in complexity over a paper by Qiu and Kumar. This note clarifies an oversight in the complexity analysis of Moreira et al's paper. Further the results of Moreira et al's paper were obtained in a more restricted setting of “projection masks”, in contrast to the more general “non-projection masks” allowed in Qiu and Kumar's paper, which was overlooked. Finally in the special case when the projection masks are used, the complexity of Qiu and Kumar's paper is lower compared to the non-projection masks case, and equals the corrected complexity of Moreira et al's paper.