Showing papers by "Richard Cole published in 2021"

Fully asynchronous stochastic coordinate descent: a tight lower bound on the parallelism achieving linear speedup

Abstract: We seek tight bounds on the viable parallelism in asynchronous implementations of coordinate descent that achieves linear speedup. We focus on asynchronous coordinate descent (ACD) algorithms on convex functions which consist of the sum of a smooth convex part and a possibly non-smooth separable convex part. We quantify the shortfall in progress compared to the standard sequential stochastic gradient descent. This leads to a simple yet tight analysis of the standard stochastic ACD in a partially asynchronous environment, generalizing and improving the bounds in prior work. We also give a considerably more involved analysis for general asynchronous environments in which the only constraint is that each update can overlap with at most q others. The new lower bound on the maximum degree of parallelism attaining linear speedup is tight and improves the best prior bound almost quadratically.

Selecting a Match: Exploration vs Decision.

Abstract: In a dynamic matching market, such as a marriage or job market, how should agents balance accepting a proposed match with the cost of continuing their search? We consider this problem in a discrete setting, in which agents have cardinal values and finite lifetimes, and proposed matches are random. We seek to quantify how well the agents can do. We provide upper and lower bounds on the collective losses of the agents, with a polynomially small failure probability, where the notion of loss is with respect to a plausible baseline we define. These bounds are tight up to constant factors. We highlight two aspects of this work. First, in our model, agents have a finite time in which to enjoy their matches, namely the minimum of their remaining lifetime and that of their partner; this implies that unmatched agents become less desirable over time, and suggests that their decision rules should change over time. Second, we use a discrete rather than a continuum model for the population. The discreteness causes variance which induces localized imbalances in the two sides of the market. One of the main technical challenges we face is to bound these imbalances. In addition, we present the results of simulations on moderate-sized problems for both the discrete and continuum versions. For these size problems, there are substantial ongoing fluctuations in the discrete setting whereas the continuum version converges reasonably quickly.

Parallel Stochastic Asynchronous Coordinate Descent: Tight Bounds on the Possible Parallelism

Abstract: Several works have shown linear speedup is achieved by an asynchronous parallel implementation of stochastic coordinate descent so long as there is not too much parallelism. More specifically, it i...