Greedy algorithm

About: Greedy algorithm is a research topic. Over the lifetime, 15347 publications have been published within this topic receiving 393945 citations.

Optimal resource allocation in uplink SC-FDMA systems

Abstract: We present algorithms for resource allocation in Single Carrier Frequency Division Multiple Access (SC-FDMA) systems, which is the uplink multiple access scheme considered in the Third Generation Partnership Project-Long Term Evolution (3GPP-LTE) standard. Unlike the well-studied problem of Orthogonal Frequency Division Multiple Access (OFDMA) resource allocation, the "subchannel adjacency" restriction, whereby users can only be assigned multiple subchannels that are adjacent to each other, makes the problem much harder to solve. We present a novel reformulation of this problem as a pure binary-integer program called the set partitioning problem, which is a well studied problem in operations research. We also present a greedy heuristic algorithm that approaches the optimal performance in cases of practical interest. We present simulation results for 3GPP-LTE uplink scenarios.

A Highly Efficient Gradient Boosting Decision Tree

Abstract: Gradient Boosting Decision Tree (GBDT) is a popular machine learning algorithm, and has quite a few effective implementations such as XGBoost and pGBRT. Although many engineering optimizations have been adopted in these implementations, the efficiency and scalability are still unsatisfactory when the feature dimension is high and data size is large. A major reason is that for each feature, they need to scan all the data instances to estimate the information gain of all possible split points, which is very time consuming. To tackle this problem, we propose two novel techniques: \emph{Gradient-based One-Side Sampling} (GOSS) and \emph{Exclusive Feature Bundling} (EFB). With GOSS, we exclude a significant proportion of data instances with small gradients, and only use the rest to estimate the information gain. We prove that, since the data instances with larger gradients play a more important role in the computation of information gain, GOSS can obtain quite accurate estimation of the information gain with a much smaller data size. With EFB, we bundle mutually exclusive features (i.e., they rarely take nonzero values simultaneously), to reduce the number of features. We prove that finding the optimal bundling of exclusive features is NP-hard, but a greedy algorithm can achieve quite good approximation ratio (and thus can effectively reduce the number of features without hurting the accuracy of split point determination by much). We call our new GBDT implementation with GOSS and EFB \emph{LightGBM}. Our experiments on multiple public datasets show that, LightGBM speeds up the training process of conventional GBDT by up to over 20 times while achieving almost the same accuracy.

Worst-Case Analysis of Greedy Heuristics for Integer Programming with Nonnegative Data

Abstract: We give a worst-case analysis for two greedy heuristics for the integer programming problem minimize cx, Ax ≥ b, 0 ≤ x ≤ u, x integer, where the entries in A, b, and c are all nonnegative. The first heuristic is for the case where the entries in A and b are integral, the second only assumes the rows are scaled so that the smallest nonzero entry is at least 1. In both cases we compare the ratio of the value of the greedy solution to that of the integer optimal. The error bound grows logarithmically in the maximum column sum of A for both heuristics.

Analysis of the greedy approach in problems of maximum k‐coverage

Abstract: In this paper, we consider a general covering problem in which k subsets are to be selected such that their union covers as large a weight of objects from a universal set of elements as possible. Each subset selected must satisfy some structural constraints. We analyze the quality of a k-stage covering algorithm that relies, at each stage, on greedily selecting a subset that gives maximum improvement in terms of overall coverage. We show that such greedily constructed solutions are guaranteed to be within a factor of 1 − 1/e of the optimal solution. In some cases, selecting a best solution at each stage may itself be difficult; we show that if a β-approximate best solution is chosen at each stage, then the overall solution constructed is guaranteed to be within a factor of 1 − 1/eβ of the optimal. Our results also yield a simple proof that the number of subsets used by the greedy approach to achieve entire coverage of the universal set is within a logarithmic factor of the optimal number of subsets. Examples of problems that fall into the family of general covering problems considered, and for which the algorithmic results apply, are discussed. © 1998 John Wiley & Sons, Inc. Naval Research Logistics 45: 615–627, 1998