Showing papers on "Greedy algorithm published in 1997"

Inducing features of random fields

Abstract: We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the field and an iterative scaling algorithm is used to estimate the optimal values of the weights. The random field models and techniques introduced in this paper differ from those common to much of the computer vision literature in that the underlying random fields are non-Markovian and have a large number of parameters that must be estimated. Relations to other learning approaches, including decision trees, are given. As a demonstration of the method, we describe its application to the problem of automatic word classification in natural language processing.

A new rank based version of the ant system: a computational study

Abstract: The ant system is a new meta-heuristic for hard combinatorial optimization problems. It is a population-based approach that uses exploitation of positive feedback as well as greedy search. It was first proposed for tackling the well known Traveling Salesman Problem (TSP), but has been also successfully applied to problems such as quadratic assignment, job-shop scheduling, vehicle routing and graph coloring.In this paper we introduce a new rank based version of the ant system and present results of a computational study, where we compare the ant system with simulated annealing and a genetic algorithm on several TSP instances. It turns out that our rank based ant system can compete with the other methods in terms of average behavior, and shows even better worst case behavior. (author's abstract)

Algebra of programming

Abstract: 1. Programs. 2. Functions and Categories. 3. Applications. 4. Relationships and Allegories. 5. Datatypes in Allegories. 6. Optimisation Problems. 7. Thinning Algorithms. 8. Dynamic Programming. 9. Greedy Algorithms. Appendices.

Incremental clustering and dynamic information retrieval

Abstract: Motivated by applications such as document and image classification in information retrieval, we consider the problem of clustering dynamic point sets in a metric space. We propose a model called incremental clustering which is based on a careful analysis of the requirements of the information retrieval application, and which should also be useful in other applications. The goal is to efficiently maintain clusters of small diameter as new points are inserted. We analyze several natural greedy algorithms and demonstrate that they perform poorly. We propose new deterministic and randomized incremental clustering algorithms which have a provably good performance, and which we believe should also perform well in practice. We complement our positive results with lower bounds on the performance of incremental algorithms. Finally, we consider the dual clustering problem where the clusters are of fixed diameter, and the goal is to minimize the number of clusters.

Greed is good: Approximating independent sets in sparse and bounded-degree graphs

Abstract: Theminimum-degree greedy algorithm, or Greedy for short, is a simple and well-studied method for finding independent sets in graphs. We show that it achieves a performance ratio of (Δ+2)/3 for approximating independent sets in graphs with degree bounded by Δ. The analysis yields a precise characterization of the size of the independent sets found by the algorithm as a function of the independence number, as well as a generalization of Turan's bound. We also analyze the algorithm when run in combination with a known preprocessing technique, and obtain an improved $$(2\bar d + 3)/5$$ performance ratio on graphs with average degree $$\bar d$$ , improving on the previous best $$(\bar d + 1)/2$$ of Hochbaum. Finally, we present an efficient parallel and distributed algorithm attaining the performance guarantees of Greedy.

A fast algorithm for constructing sparse euclidean spanners

Abstract: Let G = (V,E) be a n-vertex connected graph with positive edge weights, and let t > 1 be a real constant. A subgraph G' is a t-spanner if for all u,v ∊ V, the weight of the shortest path between u and v in G' is at most t times the weight of the corresponding shortest path in G. We design an O(n log2 n) time algorithm which, given a set V of n points in k-dimensional space (for any fixed k), and any real constant t > 1, produces a t-spanner of the complete Euclidean graph of V. This algorithm retains the spirit of a recent O(n3logn) time greedy algorithm which produces t-spanners; we use graph clustering techniques to achieve a more efficient implementation. Our spanners have O(n) edges and weight O(1)· wt(MST), which is similar to the size and weight of spanners constructed by the greedy algorithm. The constants implicit in the O-notation depend upon t and k.

Approximating Disjoint-Path Problems Using Greedy Algorithms and Packing Integer Programs

Abstract: In the edge(vertex)-disjoint path problem we are given a graph $G$ and a set ${\cal T}$ of connection requests. Every connection request in ${\cal T}$ is a vertex pair $(s_i,t_i),$ $1 \leq i \leq K.$ The objective is to connect a maximum number of the pairs via edge(vertex)-disjoint paths. The edge-disjoint path problem can be generalized to the multiple-source unsplittable flow problem where connection request $i$ has a demand $\rho_i$ and every edge $e$ a capacity $u_e.$ All these problems are NP-hard and have a multitude of applications in areas such as routing, scheduling and bin packing. Given the hardness of the problem, we study polynomial-time approximation algorithms. In this context, a $\rho$-approximation algorithm is able to route at least a $1/\rho$ fraction of the connection requests. Although the edge- and vertex-disjoint path problems, and more recently the unsplittable flow generalization, have been extensively studied, they remain notoriously hard to approximate with a bounded performance guarantee. For example, even for the simple edge-disjoint path problem, no $o(\sqrt{|E|})$-approximation algorithm is known. Moreover some of the best existing approximation ratios are obtained through sophisticated and non-standard randomized rounding schemes. In this paper we introduce techniques which yield algorithms for a wide range of disjoint-path and unsplittable flow problems. For the general unsplittable flow problem, even with weights on the commodities, our techniques lead to the first approximation algorithm and obtain an approximation ratio that matches, to within logarithmic factors, the $O(\sqrt{|E|})$ approximation ratio for the simple edge-disjoint path problem. In addition to this result and to improved bounds for several disjoint-path problems, our techniques simplify and unify the derivation of many existing approximation results. We use two basic techniques. First, we propose simple greedy algorithms for edge- and vertex-disjoint paths and second, we propose the use of a framework based on packing integer programs for more general problems such as unsplittable flow. A packing integer program is of the form maximize $c^{T}\cdot x,$ subject to $Ax \leq b,$ $A,b,c \geq 0.$ As part of our tools we develop improved approximation algorithms for a class of packing integer programs, a result that we believe is of independent interest.

Combination of Chaotic Neurodynamics with the 2-opt Algorithm to Solve Traveling Salesman Problems

Abstract: We propose a novel approach for combinatorial optimization problems. For solving the traveling salesman problems, we combine chaotic neurodynamics with heuristic algorithm. We select the heuristic algorithm of 2-opt as a basic part, because it is well understood that this simple algorithm is very effective for the traveling salesman problems. Although the conventional approaches with chaotic neurodynamics were only applied to such very small problems as 10 cities, our method exhibits higher performance for larger size problems with the order of 1${0}^{2}$.

Finding solvable subsets of constraint graphs

Abstract: We present a network flow based, degree of freedom analysis for graphs that arise in geometric constraint systems. For a vertex and edge weighted constraint graph with m edges and n vertices, we give an O(n(m + n)) time max-flow based algorithm to isolate a subgraph that can be solved separately. Such a subgraph is called dense. If the constraint problem is not overconstrained, the subgraph will be minimal. For certain overconstrained problems, finding minimal dense subgraphs may require up to O(n 2 (m + n)) steps. Finding a minimum dense subgraph is NP-hard. The algorithm has been implemented and consistently outperforms a simple but fast, greedy algorithm.

Learning to Order Things

Abstract: There are many applications in which it is desirable to order rather than classify instances. Here we consider the problem of learning how to order, given feedback in the form of preference judgments, i.e., statements to the effect that one instance should be ranked ahead of another. We outline a two-stage approach in which one first learns by conventional means a preference Junction, of the form PREF(u, v), which indicates whether it is advisable to rank u before v. New instances are then ordered so as to maximize agreements with the learned preference function. We show that the problem of finding the ordering that agrees best with a preference function is NP-complete, even under very restrictive assumptions. Nevertheless, we describe a simple greedy algorithm that is guaranteed to find a good approximation. We then discuss an on-line learning algorithm, based on the "Hedge" algorithm, for finding a good linear combination of ranking "experts." We use the ordering algorithm combined with the on-line learning algorithm to find a combination of "search experts," each of which is a domain-specific query expansion strategy for a WWW search engine, and present experimental results that demonstrate the merits of our approach.

Hardware software partitioning using genetic algorithm

Abstract: Hardware software co-design is gaining importance with the advent of CAD for embedded systems. A key phase in such designs is partitioning the specification into hardware and software implementation sets. The problem being combinatorically explosive, several greedy search algorithms have been proposed for hardware software partitioning. In this paper, we model the hardware software partitioning problem as a Constraint Satisfaction Problem (CSP), and present a genetic algorithm based approach to solve the CSP in order to obtain the partitioning solution.

General floorplanning with L-shaped, T-shaped and soft blocks based on bounded slicing grid structure

Abstract: A new method of non-slicing floorplanning is proposed, which is based on the new representation for non-slicing floorplans, called bounded slicing grid (BSG) structure. We developed a new greedy algorithm based on the BSG structure, running in linear time, to select the alternative shape for each soft block so as to minimize the overall area for general floorplan, including non-slicing structures. We propose a new stochastic optimization method, named genetic simulated annealing (GSA) for general floorplanning. Based on BSG structure, we extend SA-based local search and GA-based global crossover to L-shaped, T-shaped blocks and obtain high density packing of rectilinear blocks.

On the reconfiguration of degradable VLSI/WSI arrays

Abstract: This paper consider the problem of reconfiguring two dimensional very large scale integration (VLSI/WSI) arrays via the degradation approach. In this approach, all elements are treated uniformly and no elements are dedicated as spares. The goal is to derive a fault-free subarray T from the defective host array such that the dimensions of T are larger than some specified minimum. This problem has been shown to be NP-complete under various switching and routing constraints. However, we show that a special case of the reconfiguration problem with row bypass and column rerouting capabilities is optimally solvable in linear time. Using this result, a new fast and efficient reconfiguration algorithm is proposed. Empirical study shows that the new algorithm indeed produces good results in terms of the percentages of harvest and degradation of VLSI/WSI arrays.

MPEG coding performance improvement by jointly optimizing coding mode decisions and rate control

Abstract: This paper presents a new algorithm for determining the optimal MPEG coding strategy in terms of the selection of macroblock coding modes and quantizer scales. In the algorithm proposed in the test model the rate control operates independently from the coding mode selection for each macroblock. The coding mode is decided based only upon the energy of predictive residues. Actually, the two processes of coding mode decision and rate control are intimately related to each other and should be determined jointly in order to achieve optimal coding performance. We formulate the constrained optimization problem and present solutions based upon rate-distortion characteristics, or R(D) curves, for all the macroblocks that compose the picture being coded. Distortion for the entire picture is assumed to be decomposable and expressible as a function of individual macroblock distortions, with this being the objective function to minimize. The determination of the optimal solution is complicated by the MPEG differential encoding of motion vectors and DC coefficients, which introduce dependencies that carry over from macroblock to macroblock for a duration equal to the slice length. As an approximation, a near optimum greedy algorithm is proposed. Once the upper bound in performance is calculated, it can be used to assess how well practical suboptimum methods perform. Finally, such a practical suboptimum algorithm is proposed and evaluated.

Near-optimal intraprocedural branch alignment

Abstract: Branch alignment reorders the basic blocks of a program to minimize pipeline penalties due to control-transfer instructions. Prior work in branch alignment has produced useful heuristic methods. We present a branch alignment algorithm that usually achieves the minimum possible pipeline penalty and on our benchmarks averages within 0.3% of a provable optimum. We compare the control penalties and running times of our algorithm to an older, greedy approach and observe that both the greedy method and our method are close to the lower bound on control penalties, suggesting that greedy is good enough. Surprisingly, in actual execution our method produces programs that run noticeably faster than the greedy method. We also report results from training and testing on different data sets, validating that our results can be achieved in real-world usage. Training and testing on different data sets slightly reduced the benefits from both branch alignment algorithms, but the ranking of the algorithms does not change, and the bulk of the benefits remain.

Image segmentation by a deformable contour model incorporating region analysis

Abstract: Deformable contour models are useful tools for image segmentation. However, many models depend mainly on local edge-based image features to guide the convergence of the contour. This makes the models sensitive to noise and the initial estimate. Our model incorporates region-based image features to improve its convergence and to reduce its dependence on initial estimation. Computational efficiency is achieved by an optimization strategy, modified from the greedy algorithm of Williams and Shah. The model allows a simultaneous optimization of multiple contours, making it useful for a large variety of segmentation problems.

Task scheduling in distributed computing systems with a genetic algorithm

Abstract: Scheduling a directed acyclic graph (DAG) which represents the precedence relations of the tasks of a parallel program in a distributed computing system (DCS) is known as an NP-complete problem except for some special cases. Many heuristic-based methods have been proposed under various models and assumptions. A DCS can be classified in two types according to the characteristics of the processors on a network: a distributed homogeneous system (DHOS) and a distributed heterogeneous system (DHES). The paper defines a general model for a DHOS and a DHES and presents a genetic algorithm (GA) to solve the task scheduling problem in the defined DCS. The performance of the proposed GA is compared with the list scheduling algorithm in a DHOS and with the one-level reach-out greedy algorithm (OLROG) in a DHES. The proposed GA has shown better performance in various environments than other scheduling methods.

Mesh partitioning and load balancing for distributed memory parallel systems

Abstract: A method is outlined for optimising graph partitions which arise in mapping unstructured mesh calculations to parallel computers. The method employs a relative gain iterative technique to both evenly balance the workload and minimise the number and volume of interprocessor communications. A parallel graph reduction technique is also briefly described and can be used to give a global perspective to the optimisation. The algorithms work efficiently in parallel as well as sequentially and when combined with a fast direct partitioning technique (such as the Greedy algorithm) to give an initial partition, the resulting two-stage process proves itself to be both a powerful and flexible solution to the static graph-partitioning problem. Experiments indicate that the resulting parallel code can provide high quality partitions, independent of the initial partition, within a few seconds. The algorithms can also be used for dynamic load-balancing, reusing existing partitions and in this case the procedures are much faster than static techniques, provide partitions of similar or higher quality and, in comparison, involve the migration of a fraction of the data.

Investigation of effect of different run-time distributions on SmartNet performance

Abstract: : This thesis investigates, using in-line simulation, the effect of non-deterministic runtime distributions on the performance of SmartNet's schedule execution using the Opportunistic Load Balancing (OLB) Algorithm, the Limited Best Assignment (LBA) Algorithm, an O(mn squared) Greedy Algorithm, and an O(mn) Greedy Algorithm. SmartNet is a framework for scheduling jobs and machines in a heterogeneous computing environment. Its major strength is its use of both current machine loads and predicted job/machine performance when generating schedules. Schedules are built to meet various Quality of Service requirements using the above algorithms among others. We enhanced SmartNet's simulator so that the runtime distributions could be used for experimentation. The distributions were generated using derivations from our study on NAS Benchmarks. Experiments were run for various categories of job/machine heterogeneity to compare the algorithms which account for both load and expected performance (the Greedy algorithms) against OLB and LBA. For all categories of heterogeneity, the greedy algorithms outperformed the other two algorithms for both truncated Gaussian and exponential distributions. For these same distributions, the O(mn) Greedy algorithm performed as well as the O(mn2) Greedy algorithm when the heterogeneity of jobs and machines was high.

Solving large travelling salesman problems with small populations

Abstract: A new genetic algorithm for the solution of the travelling salesman problem is presented in this paper. The approach is to introduce several knowledge-augmented genetic operators which guide the genetic algorithm more directly towards better quality of the population but are not trapped in local optima prematurely. The algorithm applies a greedy crossover and two advanced mutation operations based on the 2-opt and 3-opt heuristics. One of our particular interests is to investigate whether small populations are adequate for solving large problems. We also want to see how the quality of the initial population and the quality of the final solution are related, especially when the population is small. Far this purpose, we designed a selective initialization scheme to generate a better initial population. The algorithm has been implemented in C and tested on several sets of data. The largest data instance used is 2392 cities (i.e., pr2392). The actual population size used is only 32. For small sets of data (less than 100 cities), our algorithm can find the optimal solutions. For large data sets, the quality of the best solutions found is about 1.3-2.3% worse than optimal. The empirical results show that combining the knowledge from the heuristic method and the genetic algorithm is a good approach for solving the large traveling salesman problem. By incorporating the heuristic method, we can develop greedy genetic operators to improve the efficiency of genetic algorithms. Moreover, it makes small population sizes sufficient to solve large problems. By incorporating the genetic algorithm technique, we can escape from local optima in many cases so that much better results can be obtained than with heuristic methods alone.

Differential greedy for the 0-1 equicut problem.

Abstract: Given the success of recent greedy schemes with tie-breaking rules proposed by the authors, this paper extends the investigation by considering diierent greedy approaches for the graph bi-partitioning problem. In particular , a new method is proposed (Diff-Greedy) that ameliorates in a signiicant way the performance of the previously proposed Min-Max-Greedy, while requiring less computation. The computational complexity is demonstrated to be O(jEj), E being the edge set of the graph. Experimental results are presented for a large sample of two classes of randomly generated graphs, one with a connection probability independent of the vertices, the second one with a two-dimensional geometric structure. Remarkably enough, the cut sizes obtained through independent repetitions are close to, and in some cases better than the cut sizes obtainedby more sophisticatedtechniquesbased on standard greedy construction followed by local search with prohibition-based diversii-cation (Tabu Search), for comparable running times.