Upper and lower bounds

About: Upper and lower bounds is a research topic. Over the lifetime, 56902 publications have been published within this topic receiving 1143379 citations. The topic is also known as: majoring or minoring element.

Approximately Optimal Solutions to the Finite-Dimensional Witsenhausen Counterexample

Abstract: Recently, a vector version of Witsenhausen's counterexample was considered and it was shown that in the asymptotic limit of infinite vector length, certain vector-quantization-based control strategies are provably within a constant factor of the asymptotically optimal cost for all possible problem parameters. While suggestive, a constant factor result for the finite-dimensional problem has remained elusive. In this paper, we provide a resolution to this issue. By applying a large-deviation “sphere-packing” philosophy, we derive a lower bound to the optimal cost for the finite dimensional case that uses appropriate shadows of an existing vector lower bound that is the same for all dimensions. Using this new lower bound, we show that good lattice-quantization-based control strategies achieve within a constant factor of the optimal cost uniformly over all possible problem parameters, including the vector length. For Witsenhausen's original problem-which is the scalar case-the gap between regular lattice-quantization-based strategies and the lower bound is provably never more than a factor of 100, and computer calculations strongly suggest that the factor in fact may be no larger than 8. Finally, to obtain a numerical understanding of the possible room for improvement in costs using alternative strategies, we also include numerical comparison with strategies that are conjectured to be optimal. Using this comparison, we posit that there is more room for improvement in our lower bounds than in our upper bounds.

On the conductivity of fiber reinforced materials

Abstract: A two‐phase material in which the phase boundaries are cylindrical surfaces is considered. A technique exists for finding upper and lower bounds on the effective thermal conductivity (or electrical conductivity, permittivity, or magnetic permeability) of the composite in the direction perpendicular to the generators of the phase boundaries in terms of two different three point correlation functions. It is shown how a phase interchange theorem can be introduced into these bounds enabling us to express them in terms of a single geometrical constant of phase geometry. We determine what range of values of this factor is realizable for real phase geometries, and we show that the bounds thus obtained span exactly all realizable effective conductivities for such composites. Finally, we show that the bounds as expressed here enable us to use a knowledge of the effective conductivity of a composite for one ratio of constituent conductivities to narrow the bounds for some other ratio.

Decision algorithms for multiplayer noncooperative games of incomplete information

Abstract: Extending the complexity results of Reif [1,2] for two player games of incomplete information, this paper (see also [3]) presents algorithms for deciding the outcome for various classes of multiplayer games of incomplete information, i.e., deciding whether or not a team has a winning strategy for a particular game. Our companion paper, [4] shows that these algorithms are indeed asymptotically optimal by providing matching lower bounds. The classes of games to which our algorithms are applicable include games which were not previously known to be decidable. We apply our algorithms to provide alternative upper bounds, and new time-space trade-offs on the complexity of multiperson alternating Turing machines [3]. We analyze the algorithms to characterize the space complexity of multiplayer games in terms of the complexity of deterministic computation on Turing machines. In hierarchical multiplayer games, each additional clique (subset of players with the same information) increases the complexity of the outcome problem by a further exponential. We show that an S ( n ) space bounded k -player game of incomplete information has a deterministic time upper bound of k + 1 repeated exponentials of S ( n ). Furthermore, S ( n ) space bounded k -player blindfold games have a deterministic space upper bound of k repeated exponentials of S ( n ). This paper proves that this exponential blow-up can occur. We also show that time bounded games do not exhibit such hierarchy. A T ( n ) time bounded blindfold multiplayer game, as well as a T ( n ) time bounded multiplayer game of incomplete information, has a deterministic space bound of T ( n ).

Improved Model Selection Method for a Regression Function with Dependent Noise

Abstract: This paper is devoted to nonparametric estimation, through the \(\mathcal{L}_2\)-risk, of a regression function based on observations with spherically symmetric errors, which are dependent random variables (except in the normal case). We apply a model selection approach using improved estimates. In a nonasymptotic setting, an upper bound for the risk is obtained (oracle inequality). Moreover asymptotic properties are given, such as upper and lower bounds for the risk, which provide optimal rate of convergence for penalized estimators.

Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses

Abstract: Uniform stability is a notion of algorithmic stability that bounds the worst case change in the model output by the algorithm when a single data point in the dataset is replaced. An influential work of Hardt et al. (2016) provides strong upper bounds on the uniform stability of the stochastic gradient descent (SGD) algorithm on sufficiently smooth convex losses. These results led to important progress in understanding of the generalization properties of SGD and several applications to differentially private convex optimization for smooth losses. Our work is the first to address uniform stability of SGD on {\em nonsmooth} convex losses. Specifically, we provide sharp upper and lower bounds for several forms of SGD and full-batch GD on arbitrary Lipschitz nonsmooth convex losses. Our lower bounds show that, in the nonsmooth case, (S)GD can be inherently less stable than in the smooth case. On the other hand, our upper bounds show that (S)GD is sufficiently stable for deriving new and useful bounds on generalization error. Most notably, we obtain the first dimension-independent generalization bounds for multi-pass SGD in the nonsmooth case. In addition, our bounds allow us to derive a new algorithm for differentially private nonsmooth stochastic convex optimization with optimal excess population risk. Our algorithm is simpler and more efficient than the best known algorithm for the nonsmooth case Feldman et al. (2020).