What is modal logic

Thompson Sampling is one of the oldest heuristics for multi-armed bandit problems. It is a randomized algorithm based on Bayesian ideas, and has recently generated significant interest after several studies demonstrated it to have comparable or better empirical performance compared to the state of the art methods. In this paper, we provide a novel regret analysis for Thompson Sampling that proves the first near-optimal problem-independent bound of O( √ NT lnT ) on the expected regret of this algorithm. Our novel martingale-based analysis techniques are conceptually simple, and easily extend to distributions other than the Beta distribution. For the version of Thompson Sampling that uses Gaussian priors, we prove a problem-independent bound of O( √ NT lnN) on the expected regret, and demonstrate the optimality of this bound by providing a matching lower bound. This lower bound of Ω( √ NT lnN) is the first lower bound on the performance of a natural version of Thompson Sampling that is away from the general lower bound of O( √ NT ) for the multi-armed bandit problem. Our near-optimal problem-independent bounds for Thompson Sampling solve a COLT 2012 open problem of Chapelle and Li. Additionally, our techniques simultaneously provide the optimal problem-dependent bound of (1+ ǫ) ∑ i lnT d(μi,μ1) +O(Nǫ2 ) on the expected regret. The optimal problem-dependent regret bound for this problem was first proven recently by Kaufmann et al. [2012b]. Appearing in Proceedings of the 16 International Conference on Artificial Intelligence and Statistics (AISTATS) 2013, Scottsdale, AZ, USA. Volume 31 of JMLR: W&CP 31. Copyright 2013 by the authors.

/pdf/further-optimal-regret-bounds-for-thompson-sampling-1bpja2rtxz.pdf

Further Optimal Regret Bounds for Thompson Sampling

59.4 T h e C u t Elimination Theorem for Second Order Logic 171 (respectively, v . ) . In particular, (a) guarantees t h a t *"-PIND or c ~ ' ~ I N D , and C ~ ' ~ C R inferences are still valid after the substitution of V for a. However, P ( V ) may fail t o be a proof in t h a t there may be initial sequents of P ( V ) of the form where V={Z')A(Z'). However, sequents of this form are easy t o prove without free cuts. S o we merely tack onto P ( V ) free cut free proofs of these initial sequents and thus obtain a proof Q of I'(V)+A(V). Q is not necessarily free cu t free, as Q may contain free cu ts with principal formula A(3). B u t since V is a first order abstract, every free cut inference in Q has a first order principal formula. Hence, by (b), there is a free cut free proof of I'(V)--+A(V). Theorem 10: (Cut Elimination Theorem). Let P be a firproof o r a VTproof. Then there is a proof P * in the same theory such tha t P * has the same endsequent as P and there are no free cu ts in P * . Furthermore, each principal formula of an induction inference in P * is a substitution instance of a principal formula of an induction inference in P and each principal abstract of a comprehension inference in P * is a substitution instance of a principal abstract of a comprehension inference in P. Hence, for all i 2 0 , if P is a fii(or vi-) proof then so is P*. Proofi We shall modify Takeuti 's exposition on pages 22-29, 112, 143-144 of [28]. T h e reader should have [28] available as he reads the proof. Following [28], we define the grade of a formula A t o be the number of logical symbols in A . T h e level of A is the number of second order quantifiers in A . A mix inference with principal formula A is of the form where l l* and A * are obtained from rI and A by removing all occurrences of A . A mix inference is free iff all of the occurrences of A in A and II are free. Since a mix inference and a cut inference are so similar, i t suffices t o prove Theorem 10 for proofs which use mix inferences instead of cu t inferences. Suppose P is a proof whose last inference is a mix with principal formula A as shown above. Define the distance of a sequent in P t o be the number of inferences separating i t from the endsequent of P. T h e right rank of P is defined t o be the maximum distance of a sequent containing a direct ancestor of an occurrence of A in the cedent ll. T h e left rank of P is the maximum distance of a sequent containing a direct ancestor of an occurrence of A in the cedent 172 Foundations of Second Order Bounded Arithmetic A . T h e rank of P is the sum of the right rank and left rank. I t suffices t o consider P with a single mix inference as the last inference. T h e proof is by ordinal induction on where level(P) and grade(P) are the level and grade of the principal formula of t he final mix of P, and w is the first infinite ordinal. T h u s it suffices t o show tha t if P is a proof with no free mixes except for the final inference of P and if Theorem 10 is satisfied for all proofs P' with ord(P' )<ord(P) , then P satisfies Theorem 10. W e modify the proof of Lemma 5.4 of Takeut i [28]: Case (1): rank(P)=2. Cases (1 .I)-(1.5. ii): Similar t o pages 2 4 2 7 of [28]. Case (1.5.iii): Suppose A=(V+)B(q5) and the last inferences of P are where V is a ~ d ~ ~ a b s t r a c t , and since rank(P)=f the indicated occurrences of (V+)B(+) are the only ones. By Lemma 9(c), we can obtain a free mix free proof of rA,B( V) from the free mix free proof of I'-+A,B(cr). T h u s we have a proof Q such t h a t the only free mix in Q is i ts last inference: where II# and A# are II and A minus all occurrences of B(V). By the induction hypothesis, there is a free mix free proof Q* of ~,II#-A#,A since ord(Q)<ord(P). By adding weak inferences t o the end of Q* we obtain the desired proof P* . Case (1.5. i~) : Suppose A=(34)B(4). This case is very similar t o Case (1.5.iii). Case (2): rank(P) >2. Case (2.1): T h e right rank of P is >l. Cases (2.1.1)-(2.1.9. ii): Similar t o 1281. T h e C u t Elimination Theorem for Second Order Logic 173 Case (2.1.J.iii): Suppose A=(V4)B(4), V is a ~:p~-abstract and the last inferences of P are: where now A and II, b u t not r , may contain occurrences of (Vc$)B(4). T h e cedents TI* and A* are II and A minus all occurrences of (V+)B(c$). Modify the end of P t o obtain a proof P1 which ends as T h e right rank of P1 is one less than the right rank of P so by the induction hypothesis there is a free mix free proof Pg of the endsequent of P I . Now consider the proof which ends T h e right rank of this is one, so by the induction hypothesis and some exchanges and contractions we obtain a free mix free proof of r , n * + A*,A. T h e rest of the cases are similar. Q.E.D. 9.5. C:yb-Defined Functions and A:yb-Defined Predicates. T h e second order theories of Bounded Arithmetic are in many respects analogous t o the first order theories s~~ and T;. One of the most fundamental properties of second order Bounded Arithmetic is t ha t new function and predicate symbols may be introduced into the language of Bounded Arithmetic; under certain conditions, these new function and predicate symbols may be used freely in the principal formulae of induction axioms and comprehension axioms. 174 Foundations of Second Order Bounded Arithmetic Definition: Let R be a second order theory of Bounded Arithmetic. Suppose A(2,y) is a formula with all free variables indicated and tha t Then we say t h a t R can define the function j such t h a t N t= ('d$)A(Z',j($)). T h e ~ t ) ~ d e f i n e d function symbols and the Atpb-defined predicate symbols play the same role in the second order theories of Bounded Arithmetic as the C:-defined function symbols and the A/-defined predicate symbols d o in the first order theories Si and T;'. In particular, the analogues of Theorems 2.2, 2.3 and 2.4 hold for second order Bounded Arithmetic. Definition: Let 7 and ji be new function and predicate symbols. T h e sets C,'tb(7,ji) and ll,'*b(7,j?) are sets of bounded formulae in the language of second order Bounded Arithmetic plus the symbols f' and ji. These sets are defined by counting alternations of second order quantifiers ignoring the first order, bounded quantifiers. Theorem 11: Let R be a second order theory of Bounded Arithmetic. Suppose R can C;lb-define each of the functions f' and can define each of the predicates ji. Let R * be the theory obtained from R by adjoining the new symbols 7 and j3 and their defining axioms. Then, if i > O and B is a C;'lb(7,j?)(or a Il,'lb@,j?)-) formula, then there is a formula B*Ec;'~~ (or II,'pb, respectively) such t h a t R * t B** B. T h e proof of Theorem 11 is similar t o the proofs of Theorems 2.2 and 2.4. Definition: Let R be a theory of Bounded Arithmetic and let 7 be a vector of defined function symbols of R and j3 be a vector of defined predicate symbols. Then ~(7, j i ) is the conservative extension of R obtained by enlarging the language t o include 7 and 3 and including the defining axioms for these symbols. Corollary 12: ( i 2 1): (a) Let f' be a vector of ~ t * ~ d e f i n e d function symbols of U; (respectively, v;) and let ji be a vector of At*b-defined predicate symbol-. of u;' (respectively, v;' ). Then U;@,j3) (respectively, vfl,j?) ) has as theorems the E : ~ ~ ~ , ~ ~ ) P I N D axioms (respectively, the c,"~(~!,~~)-IND axioms). (b) Let 7 be a vector of C:rb-defined function symbols of fi; (respectively, F;' ) and let ji be a vector of A:.b-defined predicate symbols of 6; (respectively, ) Then fi;'(],j?) (respectively, ?i@,j?)) has as theorems the E:I*(~!,~I)-PIND axioms (respectively, the E,'P~@,~~)-IND axioms). (c) Let f' be a vector of C:lb-defined function symbols of 8; (respectively, 8;:) and let j? be a vector of A:rb-defined predicate symbols of 8; (respectively, ) Then 8i(],jI) s9.5 zipb-Defined Functions and airb-Defined Predicates 175 (respectively, ) has as theorems the E:V~-PIND v , a ) axioms (respectively, the c~'~~-IND (7,jJ) axioms) and the i :9b(7,$)-~~ axioms. Corollary 12 tells us t h a t C:~~-defined function symbols and a:1b-defined predicate symbols may be used freely in the principal formulae of induction inferences. Furthermore, if we are working in the theory 82 or 82 we may use such function and predicate symbols freely in principal abstracts of comprehension inferences. T h e next two theorems give an application of ~ : ~ ~ c o m ~ r e h e n s i o n t show t h a t 82 and 8: can define the iteration of defined predicates. I t is an open question whether these theorems hold for the theories U; and v;. Theorem 18: Let ~(a,i?,?) and ~ ( a , b,t,a:,q) be formulae of 8:, where a: is a unary predicate variable. Let t (b , t ) be a term which contains only the free variables b and t. Then the predicate ~ ( a , b,i?,;) which satisfies if b=O and a < t ( b , t ) ~ ( a , b , t , < ) if a > t ( b , t ) B ( ~ , b , t , {z}K(z, [ ;b ~, t , ; j ) ,3 otherwise Proofi T h e idea, of course, is t o define K ( a , b , t , 3 by induction on the length of b . Let B*(a, b , t , c u ~ , ~ ) be the formula and let D(u ,4) be the formula I t is easy t o see t h a t

Bounded Arithmetic と計算量の根本問題

Thompson Sampling (TS) is one of the oldest heuristics for multiarmed bandit problems. It is a randomized algorithm based on Bayesian ideas and has recently generated significant interest after several studies demonstrated that it has favorable empirical performance compared to the state-of-the-art methods. In this article, a novel and almost tight martingale-based regret analysis for Thompson Sampling is presented. Our technique simultaneously yields both problem-dependent and problem-independent bounds: (1) the first near-optimal problem-independent bound of O(√ NT ln T) on the expected regret and (2) the optimal problem-dependent bound of (1 + ϵ)Σi ln T / d(μi,μ1) + O(N/ϵ2) on the expected regret (this bound was first proven by Kaufmann et al. (2012b)).Our technique is conceptually simple and easily extends to distributions other than the Beta distribution used in the original TS algorithm. For the version of TS that uses Gaussian priors, we prove a problem-independent bound of O(√ NT ln N) on the expected regret and show the optimality of this bound by providing a matching lower bound. This is the first lower bound on the performance of a natural version of Thompson Sampling that is away from the general lower bound of Ω (√ NT) for the multiarmed bandit problem.

Near-Optimal Regret Bounds for Thompson Sampling

The exact hardness of computing a Nash equilibrium is a fundamental open question in algorithmic game theory. This problem is complete for the complexity class PPAD. It is well known that problems in PPAD cannot be $$\mathrm {NP}$$ -complete unless $$\mathrm {NP}=\mathrm {coNP}$$ . Therefore, a natural direction is to reduce the hardness of PPAD to the hardness of problems used in cryptography.

Bitansky, Paneth, and Rosen [FOCS 2015] prove the hardness of PPAD assuming the existence of quasi-polynomially hard indistinguishability obfuscation and sub-exponentially hard one-way functions. This leaves open the possibility of basing PPAD hardness on simpler, polynomially hard, computational assumptions.

We make further progress in this direction and reduce PPAD hardness directly to polynomially hard assumptions. Our first result proves hardness of PPAD assuming the existence of polynomially hard indistinguishability obfuscation $$i\mathcal {O}$$ and one-way permutations. While this improves upon Bitansky et al.'s work, it does not give us a reduction to simpler, polynomially hard computational assumption because constructions of $$i\mathcal {O}$$ inherently seems to require assumptions with sub-exponential hardness. In contrast, public key functional encryption is a much simpler primitive and does not suffer from this drawback. Our second result shows that $$\mathsf{PPAD}$$ hardness can be based on polynomially hard compact public key functional encryption and one-way permutations. Our results further demonstrate the power of polynomially hard compact public key functional encryption which is believed to be weaker than indistinguishability obfuscation. Our techniques are general and we expect them to have various applications.

Revisiting the Cryptographic Hardness of Finding a Nash Equilibrium

https://users.math.cas.cz/~jerabek/papers/wphp.pdf

Dual weak pigeonhole principle, Boolean complexity, and derandomization

We construct explicit bases of admissible rules for a representative class of normal modal logics (including the systems K4, GL, S4, Grz, and GL.3), by extending the methods of S. Ghilardi and R. Iemhoff. We also investigate the notion of admissible multiple conclusion rules.

/pdf/admissible-rules-of-modal-logics-54nq0bf2d1.pdf

Admissible Rules of Modal Logics

We investigate admissible rules of Łukasiewicz multi-valued propositional logic. We show that admissibility of multiple-conclusion rules in Łukasiewicz logic, as well as validity of universal sentences in free MV-algebras, is decidable (in PSPACE).

/pdf/admissible-rules-of-lukasiewicz-logic-7vip4zq6gf.pdf

Admissible Rules of Łukasiewicz Logic

https://users.math.cas.cz/~jerabek/papers/factor.pdf

Integer factoring and modular square roots

We construct explicit bases of single-conclusion and multiple-conclusion admissible rules of propositional Łukasiewicz logic, and we prove that every formula has an admissibly saturated approximation. We also show that Łukasiewicz logic has no finite basis of admissible rules.

/pdf/bases-of-admissible-rules-of-lukasiewicz-logic-16ehafwt92.pdf

Emil Jeřábek

Papers

Dual weak pigeonhole principle, Boolean complexity, and derandomization

Admissible Rules of Modal Logics

Admissible Rules of Łukasiewicz Logic

Integer factoring and modular square roots

Bases of Admissible Rules of Łukasiewicz Logic