The soft set theory offers a general mathematical tool for dealing with uncertain, fuzzy, not clearly defined objects. The main purpose of this paper is to introduce the basic notions of the theory of soft sets, to present the first results of the theory, and to discuss some problems of the future.

Soft set theory—First results

(This article originally appeared in Management Science, January 1981, Volume 27, Number 1, pp. 1-18, published by The Institute of Management Sciences.)

One of the most computationally useful ideas of the 1970s is the observation that many hard integer programming problems can be viewed as easy problems complicated by a relatively small set of side constraints. Dualizing the side constraints produces a Lagrangian problem that is easy to solve and whose optimal value is a lower bound (for minimization problems) on the optimal value of the original problem. The Lagrangian problem can thus be used in place of a linear programming relaxation to provide bounds in a branch and bound algorithm. This approach has led to dramatically improved algorithms for a number of important problems in the areas of routing, location, scheduling, assignment and set covering. This paper is a review of Lagrangian relaxation based on what has been learned in the last decade.

https://www.jstor.org/stable/info/2631139

The Lagrangian Relaxation Method for Solving Integer Programming Problems

We consider a class of subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions. This type of minimization arises in a dual context from Lagrangian relaxation of the coupling constraints of large scale separable problems. The idea is to perform the subgradient iteration incrementally, by sequentially taking steps along the subgradients of the component functions, with intermediate adjustment of the variables after processing each component function. This incremental approach has been very successful in solving large differentiable least squares problems, such as those arising in the training of neural networks, and it has resulted in a much better practical rate of convergence than the steepest descent method.
In this paper, we establish the convergence properties of a number of variants of incremental subgradient methods, including some that are stochastic. Based on the analysis and computational experiments, the methods appear very promising and effective for important classes of large problems. A particularly interesting discovery is that by randomizing the order of selection of component functions for iteration, the convergence rate is substantially improved.

/pdf/incremental-subgradient-methods-for-nondifferentiable-46sp42lt1w.pdf

Incremental Subgradient Methods for Nondifferentiable Optimization

We study subgradient methods for computing the saddle points of a convex-concave function. Our motivation comes from networking applications where dual and primal-dual subgradient methods have attracted much attention in the design of decentralized network protocols. We first present a subgradient algorithm for generating approximate saddle points and provide per-iteration convergence rate estimates on the constructed solutions. We then focus on Lagrangian duality, where we consider a convex primal optimization problem and its Lagrangian dual problem, and generate approximate primal-dual optimal solutions as approximate saddle points of the Lagrangian function. We present a variation of our subgradient method under the Slater constraint qualification and provide stronger estimates on the convergence rate of the generated primal sequences. In particular, we provide bounds on the amount of feasibility violation and on the primal objective function values at the approximate solutions. Our algorithm is particularly well-suited for problems where the subgradient of the dual function cannot be evaluated easily (equivalently, the minimum of the Lagrangian function at a dual solution cannot be computed efficiently), thus impeding the use of dual subgradient methods.

http://www.ifp.illinois.edu/~angelia/saddle_revised.pdf

Subgradient Methods for Saddle-Point Problems

Proximal bundle methods for minimizing a convex functionf generate a sequence {x

 k
 } by takingx

k+1 to be the minimizer of
 
$$\hat f^k (x) + u^k |x - x^k |^2 /2$$

 , where
 
$$\hat f^k $$

 is a sufficiently accurate polyhedral approximation tof andu

 k
 > 0. The usual choice ofu

 k
 = 1 may yield very slow convergence. A technique is given for choosing {u

 k
 } adaptively that eliminates sensitivity to objective scaling. Some encouraging numerical experience is reported.

V.F. Dem'yanov

Papers

Nondifferentiable Optimization