https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

https://www.cs.jhu.edu/~misha/ReadingSeminar/Papers/Bay08.pdf

Speeded-Up Robust Features (SURF)

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

Models for the processes by which ideas and influence propagate through a social network have been studied in a number of domains, including the diffusion of medical and technological innovations, the sudden and widespread adoption of various strategies in game-theoretic settings, and the effects of "word of mouth" in the promotion of new products. Recently, motivated by the design of viral marketing strategies, Domingos and Richardson posed a fundamental algorithmic problem for such social network processes: if we can try to convince a subset of individuals to adopt a new product or innovation, and the goal is to trigger a large cascade of further adoptions, which set of individuals should we target?We consider this problem in several of the most widely studied models in social network analysis. The optimization problem of selecting the most influential nodes is NP-hard here, and we provide the first provable approximation guarantees for efficient algorithms. Using an analysis framework based on submodular functions, we show that a natural greedy strategy obtains a solution that is provably within 63% of optimal for several classes of models; our framework suggests a general approach for reasoning about the performance guarantees of algorithms for these types of influence problems in social networks.We also provide computational experiments on large collaboration networks, showing that in addition to their provable guarantees, our approximation algorithms significantly out-perform node-selection heuristics based on the well-studied notions of degree centrality and distance centrality from the field of social networks.

/pdf/maximizing-the-spread-of-influence-through-a-social-network-47y1ehuvwi.pdf

Maximizing the Spread of Influence through a Social Network

In settings where a platform must allocate finite supplies of goods to buyers, balancing overall platform revenues with the fairness of the individual allocations to platform participants is paramo...

Fair Resource Allocation in a Volatile Marketplace

Obtaining scalable algorithms for hierarchical agglomerative clustering (HAC) is of significant interest due to the massive size of real-world datasets. At the same time, efficiently parallelizing HAC is difficult due to the seemingly sequential nature of the algorithm. In this paper, we address this issue and present ParHAC, the first efficient parallel HAC algorithm with sublinear depth for the widely-used average-linkage function. In particular, we provide a $(1+\epsilon)$-approximation algorithm for this problem on $m$ edge graphs using $\tilde{O}(m)$ work and poly-logarithmic depth. Moreover, we show that obtaining similar bounds for exact average-linkage HAC is not possible under standard complexity-theoretic assumptions. We complement our theoretical results with a comprehensive study of the ParHAC algorithm in terms of its scalability, performance, and quality, and compare with several state-of-the-art sequential and parallel baselines. On a broad set of large publicly-available real-world datasets, we find that ParHAC obtains a 50.1x speedup on average over the best sequential baseline, while achieving quality similar to the exact HAC algorithm. We also show that ParHAC can cluster one of the largest publicly available graph datasets with 124 billion edges in a little over three hours using a commodity multicore machine.

Hierarchical Agglomerative Graph Clustering in Poly-Logarithmic Depth

In the era of big data, learning from categorical features with very large vocabularies (e.g., 28 million for the Criteo click prediction dataset) has become a practical challenge for machine learning researchers and practitioners. We design a highly-scalable vocabulary compression algorithm that seeks to maximize the mutual information between the compressed categorical feature and the target binary labels and we furthermore show that its solution is guaranteed to be within a $1-1/e \approx 63\%$ factor of the global optimal solution. To achieve this, we introduce a novel re-parametrization of the mutual information objective, which we prove is submodular, and design a data structure to query the submodular function in amortized $O(\log n )$ time (where $n$ is the input vocabulary size). Our complete algorithm is shown to operate in $O(n \log n )$ time. Additionally, we design a distributed implementation in which the query data structure is decomposed across $O(k)$ machines such that each machine only requires $O(\frac n k)$ space, while still preserving the approximation guarantee and using only logarithmic rounds of computation. We also provide analysis of simple alternative heuristic compression methods to demonstrate they cannot achieve any approximation guarantee. Using the large-scale Criteo learning task, we demonstrate better performance in retaining mutual information and also verify competitive learning performance compared to other baseline methods.

Categorical Feature Compression via Submodular Optimization

Minimax optimization has become a central tool in machine learning with applications in robust optimization, reinforcement learning, GANs, etc. These applications are often nonconvex-nonconcave, but the existing theory is unable to identify and deal with the fundamental difficulties this poses. In this paper, we study the classic proximal point method (PPM) applied to nonconvex-nonconcave minimax problems. We find that a classic generalization of the Moreau envelope by Attouch and Wets provides key insights. Critically, we show this envelope not only smooths the objective but can convexify and concavify it based on the level of interaction present between the minimizing and maximizing variables. From this, we identify three distinct regions of nonconvex-nonconcave problems. When interaction is sufficiently strong, we derive global linear convergence guarantees. Conversely when the interaction is fairly weak, we derive local linear convergence guarantees with a proper initialization. Between these two settings, we show that PPM may diverge or converge to a limit cycle.

The Landscape of the Proximal Point Method for Nonconvex-Nonconcave Minimax Optimization

We study policies aiming to minimize the weighted sum of completion times of
jobs in the context of coordination mechanisms for selfish scheduling problems.
Our goal is to design local policies that achieve a good price of anarchy in
the resulting equilibria for unrelated machine scheduling. In short, we present
the first constant-factor-approximate coordination mechanisms for this model
and show that our bounds imply a new combinatorial constant-factor
approximation algorithm for the underlying optimization problem. More
specifically: We present a generalization of the ShortestFirst policy for weighted jobs,
called SmithRule policy; we prove that it achieves an approximation ratio of 4
and show that any set of strongly local ordering policies can result in
equilibria with approximation ratio at least 4 even for unweighted jobs. The main result of our paper is ProportionalSharing, a preemptive strongly
local policy that generalizes the EqualSharing policy for weighted jobs and
beats this lower bound of 4; we show that this policy achieves an approximation
ratio of 2.619 for the weighted sum of completion times and an approximation
ratio of 2.5 for the (unweighted) sum of completion times. Again, we observe
that these bounds are tight. Furthermore, we show that the ProportionalSharing
policy induces potential games (in which best-response dynamics converge to
pure Nash equilibria). All of our upper bounds are for the robust price of anarchy, defined by
Roughgarden [39], so they naturally extend to mixed Nash equilibria, correlated
equilibria, and regret minimization dynamics. Finally, we prove that the games induced by ProportionalSharing are
beta?-nice, which yields the first combinatorial constant-factor approximation
algorithm minimizing weighted completion time for unrelated machine scheduling.

Vahab Mirrokni

Papers

Fair Resource Allocation in a Volatile Marketplace

Hierarchical Agglomerative Graph Clustering in Poly-Logarithmic Depth

Categorical Feature Compression via Submodular Optimization

The Landscape of the Proximal Point Method for Nonconvex-Nonconcave Minimax Optimization

Coordination Mechanisms for Weighted Sum of Completion Times in Machine Scheduling