Showing papers on "Iterative method published in 2015"

An Introduction to Domain Decomposition Methods: Algorithms, Theory, and Parallel Implementation

Abstract: The purpose of this book is to offer an overview of the most popular domain decomposition methods for partial differential equations (PDEs). These methods are widely used for numerical simulations in solid mechanics, electromagnetism, flow in porous media, etc., on parallel machines from tens to hundreds of thousands of cores. The appealing feature of domain decomposition methods is that, contrary to direct methods, they are naturally parallel. The authors focus on parallel linear solvers. The authors present all popular algorithms, both at the PDE level and at the discrete level in terms of matrices, along with systematic scripts for sequential implementation in a free open-source finite element package as well as some parallel scripts. Also included is a new coarse space construction (two-level method) that adapts to highly heterogeneous problems. Audience: This book is intended for those working in domain decomposition methods, parallel computing, and iterative methods, in particular those who need to implement parallel solvers for PDEs. It will also be of interest to mechanical, civil, and aeronautical engineers, physical and environmental scientists, and physicists in need of parallel PDE solvers.

Minimization of $\ell_{1-2}$ for Compressed Sensing

Abstract: We study minimization of the difference of $\ell_1$ and $\ell_2$ norms as a nonconvex and Lipschitz continuous metric for solving constrained and unconstrained compressed sensing problems. We establish exact (stable) sparse recovery results under a restricted isometry property (RIP) condition for the constrained problem, and a full-rank theorem of the sensing matrix restricted to the support of the sparse solution. We present an iterative method for $\ell_{1-2}$ minimization based on the difference of convex functions algorithm and prove that it converges to a stationary point satisfying the first-order optimality condition. We propose a sparsity oriented simulated annealing procedure with non-Gaussian random perturbation and prove the almost sure convergence of the combined algorithm (DCASA) to a global minimum. Computation examples on success rates of sparse solution recovery show that if the sensing matrix is ill-conditioned (non RIP satisfying), then our method is better than existing nonconvex compre...

Randomized Iterative Methods for Linear Systems

Abstract: We develop a novel, fundamental, and surprisingly simple randomized iterative method for solving consistent linear systems. Our method has six different but equivalent interpretations: sketch-and-project, constrain-and-approximate, random intersect, random linear solve, random update, and random fixed point. By varying its two parameters---a positive definite matrix (defining geometry), and a random matrix (sampled in an independent and identically distributed fashion in each iteration)---we recover a comprehensive array of well-known algorithms as special cases, including the randomized Kaczmarz method, randomized Newton method, randomized coordinate descent method, and random Gaussian pursuit. We naturally also obtain variants of all these methods using blocks and importance sampling. However, our method allows for a much wider selection of these two parameters, which leads to a number of new specific methods. We prove exponential convergence of the expected norm of the error in a single theorem, from w...

Hyperspectral Image Denoising via Noise-Adjusted Iterative Low-Rank Matrix Approximation

Abstract: Due to the low-dimensional property of clean hyperspectral images (HSIs), many low-rank-based methods have been proposed to denoise HSIs. However, in an HSI, the noise intensity in different bands is often different, and most of the existing methods do not take this fact into consideration. In this paper, a noise-adjusted iterative low-rank matrix approximation (NAILRMA) method is proposed for HSI denoising. Based on the low-rank property of HSIs, the patchwise low-rank matrix approximation (LRMA) is established. To further separate the noise from the signal subspaces, an iterative regularization framework is proposed. Considering that the noise intensity in different bands is different, an adaptive iteration factor selection based on the noise variance of each HSI band is adopted. This noise-adjusted iteration strategy can effectively preserve the high-SNR bands and denoise the low-SNR bands. The randomized singular value decomposition (RSVD) method is then utilized to solve the NAILRMA optimization problem. A number of experiments were conducted in both simulated and real data conditions to illustrate the performance of the proposed NAILRMA method for HSI denoising.

A Novel Point-Matching Algorithm Based on Fast Sample Consensus for Image Registration

Abstract: Robustness and accuracy are the two main challenging problems in feature-based remote sensing image registration. In this letter, a novel point-matching algorithm is proposed. An improved random sample consensus (RANSAC) algorithm called fast sample consensus (FSC) is proposed. It divides the data set in RANSAC into two parts: the sample set and the consensus set. Sample set has high correct rate and consensus set has a large number of correct matches. An iterative method is put forward to increase the number of correct correspondences. A set of measures has been used to evaluate the registration result. The performance of the proposed method is validated on the evaluation of these measures and the mosaic images. FSC can get more correct matches than RANSAC in less number of iterations, iterative selection of correct matches algorithm and removal of the imprecise points algorithm effectively increase the accuracy of the result. Extensive experimental studies compared with three state-of-the-art methods prove that the proposed algorithm is robust and accurate.

New Algorithms for Secure Outsourcing of Large-Scale Systems of Linear Equations

Abstract: With the rapid development in availability of cloud services, the techniques for securely outsourcing the prohibitively expensive computations to untrusted servers are getting more and more attentions in the scientific community. In this paper, we investigate secure outsourcing for large-scale systems of linear equations, which are the most popular problems in various engineering disciplines. For the first time, we utilize the sparse matrix to propose a new secure outsourcing algorithm of large-scale linear equations in the fully malicious model. Compared with the state-of-the-art algorithm, the proposed algorithm only requires ( optimal ) one round communication (while the algorithm requires $L$ rounds of interactions between the client and cloud server, where $L$ denotes the number of iteration in iterative methods). Furthermore, the client in our algorithm can detect the misbehavior of cloud server with the ( optimal ) probability 1. Therefore, our proposed algorithm is superior in both efficiency and checkability. We also provide the experimental evaluation that demonstrates the efficiency and effectiveness of our algorithm.

Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping

Abstract: This work proposes block-coordinate fixed point algorithms with applications to nonlinear analysis and optimization in Hilbert spaces. The asymptotic analysis relies on a notion of stochastic quasi-Fejer monotonicity, which is thoroughly investigated. The iterative methods under consideration feature random sweeping rules to select arbitrarily the blocks of variables that are activated over the course of the iterations and they allow for stochastic errors in the evaluation of the operators. Algorithms using quasi-nonexpansive operators or compositions of averaged nonexpansive operators are constructed, and weak and strong convergence results are established for the sequences they generate. As a by-product, novel block-coordinate operator splitting methods are obtained for solving structured monotone inclusion and convex minimization problems. In particular, the proposed framework leads to random block-coordinate versions of the Douglas--Rachford and forward-backward algorithms and of some of their variant...

Master–Slave-Splitting Based Distributed Global Power Flow Method for Integrated Transmission and Distribution Analysis

Abstract: With the recent rapid development of smart grid technology, the distribution grids become more active, and the interaction between transmission and distribution grids becomes more significant. However, in traditional power flow calculations, transmission and distribution grids are separated, which is not suitable for such future smart grids. To achieve a global unified power flow solution to support an integrated analysis for both transmission and distribution grids, we propose a global power flow (GPF) method that considers transmission and distribution grids as a whole in this paper. We construct GPF equations, and develop a master-slave-splitting (MSS) iterative method with convergence guarantee to alleviate boundary mismatches between the transmission and distribution grids. In our method, the GPF problem is split into a transmission power flow and a number of distribution power flow sub-problems, which supports on-line geographically distributed computation. Each sub-problem can be solved using a different power flow algorithm to capture the different features of transmission and distribution grids. An equivalent method is proposed to improve the convergence of the MSS-based GPF calculation for distribution grids that include loops. Numerical simulations validate the effectiveness of the proposed method, in particular when the distribution grid has loops or distributed generators.

Global Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems

Abstract: This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB) equation to an optimization problem, which is solved via a new policy iteration method. The proposed method distinguishes from previously known nonlinear ADP methods in that the neural network approximation is avoided, giving rise to significant computational improvement. Instead of semiglobally or locally stabilizing, the resultant control policy is globally stabilizing for a general class of nonlinear polynomial systems. Furthermore, in the absence of the a priori knowledge of the system dynamics, an online learning method is devised to implement the proposed policy iteration technique by generalizing the current ADP theory. Finally, three numerical examples are provided to validate the effectiveness of the proposed method.

Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals

Abstract: We consider the problem of recovering block-sparse signals whose cluster patterns are unknown a priori. Block-sparse signals with nonzero coefficients occurring in clusters arise naturally in many practical scenarios. However, the knowledge of the block partition is usually unavailable in practice. In this paper, we develop a new sparse Bayesian learning method for recovery of block-sparse signals with unknown cluster patterns. A pattern-coupled hierarchical Gaussian prior is introduced to characterize the pattern dependencies among neighboring coefficients, where a set of hyperparameters are employed to control the sparsity of signal coefficients. The proposed hierarchical model is similar to that for the conventional sparse Bayesian learning. However, unlike the conventional sparse Bayesian learning framework in which each individual hyperparameter is associated independently with each coefficient, in this paper, the prior for each coefficient not only involves its own hyperparameter, but also its immediate neighbor hyperparameters. In doing this way, the sparsity patterns of neighboring coefficients are related to each other and the hierarchical model has the potential to encourage structured-sparse solutions. The hyperparameters are learned by maximizing their posterior probability. We exploit an expectation-maximization (EM) formulation to develop an iterative algorithm that treats the signal as hidden variables and iteratively maximizes a lower bound on the posterior probability. In the M-step, a simple suboptimal solution is employed to replace a gradient-based search to maximize the lower bound. Numerical results are provided to illustrate the effectiveness of the proposed algorithm.

A Novel Dual Iterative $Q$ -Learning Method for Optimal Battery Management in Smart Residential Environments

Abstract: In this paper, a novel iterative $Q$ -learning method called “dual iterative $Q$ -learning algorithm” is developed to solve the optimal battery management and control problem in smart residential environments. In the developed algorithm, two iterations are introduced, which are internal and external iterations, where internal iteration minimizes the total cost of power loads in each period, and the external iteration makes the iterative $Q$ -function converge to the optimum. Based on the dual iterative $Q$ -learning algorithm, the convergence property of the iterative $Q$ -learning method for the optimal battery management and control problem is proven for the first time, which guarantees that both the iterative $Q$ -function and the iterative control law reach the optimum. Implementing the algorithm by neural networks, numerical results and comparisons are given to illustrate the performance of the developed algorithm.

Local-Set-Based Graph Signal Reconstruction

Abstract: Signal processing on graph is attracting more and more attentions. For a graph signal in the low-frequency subspace, the missing data associated with unsampled vertices can be reconstructed through the sampled data by exploiting the smoothness of the graph signal. In this paper, the concept of local set is introduced and two local-set-based iterative methods are proposed to reconstruct bandlimited graph signal from sampled data. In each iteration, one of the proposed methods reweights the sampled residuals for different vertices, while the other propagates the sampled residuals in their respective local sets. These algorithms are built on frame theory and the concept of local sets, based on which several frames and contraction operators are proposed. We then prove that the reconstruction methods converge to the original signal under certain conditions and demonstrate the new methods lead to a significantly faster convergence compared with the baseline method. Furthermore, the correspondence between graph signal sampling and time-domain irregular sampling is analyzed comprehensively, which may be helpful to future works on graph signals. Computer simulations are conducted. The experimental results demonstrate the effectiveness of the reconstruction methods in various sampling geometries, imprecise priori knowledge of cutoff frequency, and noisy scenarios.

A multi-threaded version of MCFM

Abstract: We report on our findings modifying MCFM using OpenMP to implement multi-threading. By using OpenMP, the modified MCFM will execute on any processor, automatically adjusting to the number of available threads. We modified the integration routine VEGAS to distribute the event evaluation over the threads, while combining all events at the end of every iteration to optimize the numerical integration. Special care has been taken that the results of the Monte Carlo integration are independent of the number of threads used, to facilitate the validation of the OpenMP version of MCFM.

Neural Network and Jacobian Method for Solving the Inverse Statics of a Cable-Driven Soft Arm With Nonconstant Curvature

Abstract: The solution of the inverse kinematics problem of soft manipulators is essential to generate paths in the task space. The inverse kinematics problem of constant curvature or piecewise constant curvature manipulators has already been solved by using different methods, which include closed-form analytical approaches and iterative methods based on the Jacobian method. On the other hand, the inverse kinematics problem of nonconstant curvature manipulators remains unsolved. This study represents one of the first attempts in this direction. It presents both a model-based method and a supervised learning method to solve the inverse statics of nonconstant curvature soft manipulators. In particular, a Jacobian-based method and a feedforward neural network are chosen and tested experimentally. A comparative analysis has been conducted in terms of accuracy and computational time.

A study on iterative methods for solving Richards` equation

Abstract: This work concerns linearization methods for efficiently solving the Richards` equation,a degenerate elliptic-parabolic equation which models flow in saturated/unsaturated porous media.The discretization of Richards` equation is based on backward Euler in time and Galerkin finite el-ements in space. The most valuable linearization schemes for Richards` equation, i.e. the Newtonmethod, the Picard method, the Picard/Newton method and theLscheme are presented and theirperformance is comparatively studied. The convergence, the computational time and the conditionnumbers for the underlying linear systems are recorded. The convergence of theLscheme is theo-retically proved and the convergence of the other methods is discussed. A new scheme is proposed,theLscheme/Newton method which is more robust and quadratically convergent. The linearizationmethods are tested on illustrative numerical examples.

Optimal Energy-Efficient Transmit Beamforming for Multi-User MISO Downlink

Abstract: This paper studies beamforming techniques for energy efficiency maximization (EEmax) in multiuser multiple-input single-output (MISO) downlink system. For this challenging nonconvex problem, we first derive an optimal solution using branch-and-reduce-and-bound (BRB) approach. We also propose two low-complexity approximate designs. The first one uses the well-known zero-forcing beamforming (ZFBF) to eliminate inter-user interference so that the EEmax problem reduces to a concave-convex fractional program. Particularly, the problem is then efficiently solved by closed-form expressions in combination with the Dinkelbach's approach. In the second design, we aim at finding a stationary point using the sequential convex approximation (SCA) method. By proper transformations, we arrive at a fast converging iterative algorithm where a convex program is solved in each iteration. We further show that the problem in each iteration can also be approximated as a second-order cone program (SOCP), allowing for exploiting computationally efficient state-of-the-art SOCP solvers. Numerical experiments demonstrate that the second design converges quickly and achieves a near-optimal performance. To further increase the energy efficiency, we also consider the joint beamforming and antenna selection (JBAS) problem for which two designs are proposed. In the first approach, we capitalize on the perspective reformulation in combination with continuous relaxation to solve the JBAS problem. In the second one, sparsity-inducing regularization is introduced to approximate the JBAS problem, which is then solved by the SCA method. Numerical results show that joint beamforming and antenna selection offers significant energy efficiency improvement for large numbers of transmit antennas.

Learning Efficient Sparse and Low Rank Models

Abstract: Parsimony, including sparsity and low rank, has been shown to successfully model data in numerous machine learning and signal processing tasks. Traditionally, such modeling approaches rely on an iterative algorithm that minimizes an objective function with parsimony-promoting terms. The inherently sequential structure and data-dependent complexity and latency of iterative optimization constitute a major limitation in many applications requiring real-time performance or involving large-scale data. Another limitation encountered by these modeling techniques is the difficulty of their inclusion in discriminative learning scenarios. In this work, we propose to move the emphasis from the model to the pursuit algorithm, and develop a process-centric view of parsimonious modeling, in which a learned deterministic fixed-complexity pursuit process is used in lieu of iterative optimization. We show a principled way to construct learnable pursuit process architectures for structured sparse and robust low rank models, derived from the iteration of proximal descent algorithms. These architectures learn to approximate the exact parsimonious representation at a fraction of the complexity of the standard optimization methods. We also show that appropriate training regimes allow to naturally extend parsimonious models to discriminative settings. State-of-the-art results are demonstrated on several challenging problems in image and audio processing with several orders of magnitude speed-up compared to the exact optimization algorithms.

Distributed Bisection Method for Economic Power Dispatch in Smart Grid

Abstract: In this paper, we present a fully distributed bisection algorithm for the economic dispatch problem (EDP) in a smart grid scenario, with the goal to minimize the aggregated cost of a network of generators, which cooperatively furnish a given amount of power within their individual capacity constraints. Our distributed algorithm adopts the method of bisection, and is based on a consensus-like iterative method, with no need for a central decision maker or a leader node. Under strong connectivity conditions and allowance for local communications, we show that the iterative solution converges to the globally optimal solution. Furthermore, two stopping criteria are presented for the practical implementation of the proposed algorithm, for which sign consensus is defined. Finally, numerical simulations based on the IEEE 14-bus and 118-bus systems are given to illustrate the performance of the algorithm.

Iterative Dynamic Water-Filling for Fading Multiple-Access Channels With Energy Harvesting

Abstract: In this paper, we develop optimal energy scheduling algorithms for N-user fading multiple-access channels with energy harvesting to maximize the channel sum-rate, assuming that the side information of both the channel states and energy harvesting states for K time slots is known a priori, and the battery capacity and the maximum energy consumption in each time slot are bounded. The problem is formulated as a convex optimization problem with O (NK) constraints making it hard to solve using a general convex solver since the computational complexity of a generic convex solver becomes impractically high when the number of constraints is large. This paper gives an efficient energy scheduling algorithm, called the iterative dynamic water-filling algorithm, that has a computational complexity of O(NK 2 ) per iteration. For the single-user case, a dynamic water-filling method is shown to be optimal. Unlike the traditional water-filling algorithm, in dynamic water-filling, the water level is not constant but changes when the battery overflows or depletes. An iterative version of the dynamic water-filling algorithm is shown to be optimal for the case of multiple users. Even though in principle the optimality is achieved under large number of iterations, in practice convergence is reached in only a few iterations. Moreover, a single iteration of the dynamic water-filling algorithm achieves a sum-rate that is within (N-1)K nats of the optimal sum-rate.

Efficient convolutional sparse coding

Abstract: Computationally efficient algorithms may be applied for fast dictionary learning solving the convolutional sparse coding problem in the Fourier domain. More specifically, efficient convolutional sparse coding may be derived within an alternating direction method of multipliers (ADMM) framework that utilizes fast Fourier transforms (FFT) to solve the main linear system in the frequency domain. Such algorithms may enable a significant reduction in computational cost over conventional approaches by implementing a linear solver for the most critical and computationally expensive component of the conventional iterative algorithm. The theoretical computational cost of the algorithm may be reduced from O(M 3 N) to O(MN log N), where N is the dimensionality of the data and M is the number of elements in the dictionary. This significant improvement in efficiency may greatly increase the range of problems that can practically be addressed via convolutional sparse representations.

Revisiting Lambert’s problem

Abstract: The orbital boundary value problem, also known as Lambert problem, is revisited. Building upon Lancaster and Blanchard approach, new relations are revealed and a new variable representing all problem classes, under L-similarity, is used to express the time of flight equation. In the new variable, the time of flight curves have two oblique asymptotes and they mostly appear to be conveniently approximated by piecewise continuous lines. We use and invert such a simple approximation to provide an efficient initial guess to an Householder iterative method that is then able to converge, for the single revolution case, in only two iterations. The resulting algorithm is compared, for single and multiple revolutions, to Gooding’s procedure revealing to be numerically as accurate, while having a significantly smaller computational complexity.

Rational Basis Functions in Iterative Learning Control—With Experimental Verification on a Motion System

Abstract: Iterative learning control (ILC) approaches often exhibit poor extrapolation properties with respect to exogenous signals, such as setpoint variations. This brief introduces rational basis functions in ILC. Such rational basis functions have the potential to both increase performance and enhance the extrapolation properties. The key difficulty that is associated with these rational basis functions lies in a significantly more complex optimization problem when compared with using preexisting polynomial basis functions. In this brief, a new iterative optimization algorithm is proposed that enables the use of rational basis functions in ILC for single-input single-output systems. An experimental case study confirms the advantages of rational basis functions compared with preexisting results, as well as the effectiveness of the proposed iterative algorithm.

A Two-Phase Iterative Heuristic Approach for the Production Routing Problem

Abstract: This paper investigates the integrated optimization of production, distribution, and inventory decisions related to supplying multiple retailers from a central production facility. A single-item capacitated lot-sizing problem is defined for optimizing production decisions and inventory management. The optimization of daily distribution is modeled as a traveling salesman problem or a vehicle routing problem depending on the number of vehicles. A two-phase iterative method, from which several heuristics are derived, is proposed that iteratively focuses on lot-sizing and distribution decisions. Computational results show that our best heuristic outperforms existing methods.

A chebyshev semi-iterative approach for accelerating projective and position-based dynamics

Abstract: In this paper, we study the use of the Chebyshev semi-iterative approach in projective and position-based dynamics. Although projective dynamics is fundamentally nonlinear, its convergence behavior is similar to that of an iterative method solving a linear system. Because of that, we can estimate the "spectral radius" and use it in the Chebyshev approach to accelerate the convergence by at least one order of magnitude, when the global step is handled by the direct solver, the Jacobi solver, or even the Gauss-Seidel solver. Our experiment shows that the combination of the Chebyshev approach and the direct solver runs fastest on CPU, while the combination of the Chebyshev approach and the Jacobi solver outperforms any other combination on GPU, as it is highly compatible with parallel computing. Our experiment further shows position-based dynamics can be accelerated by the Chebyshev approach as well, although the effect is less obvious for tetrahedral meshes. The whole approach is simple, fast, effective, GPU-friendly, and has a small memory cost.

Energy-Efficient Subcarrier Assignment and Power Allocation in OFDMA Systems With Max-Min Fairness Guarantees

Abstract: In next-generation wireless networks, energy efficiency optimization needs to take individual link fairness into account. In this paper, we investigate a max-min energy efficiency-optimal problem (MEP) to ensure fairness among links in terms of energy efficiency in OFDMA systems. In particular, we maximize the energy efficiency of the worst-case link subject to the rate requirements, transmit power, and subcarrier assignment constraints. Due to the nonsmooth and mixed combinatorial features of the formulation, we focus on low-complexity suboptimal algorithms design. Using a generalized fractional programming theory and the Lagrangian dual decomposition, we first propose an iterative algorithm to solve the problem. We then devise algorithms to separate the subcarrier assignment and power allocation to further reduce the computational cost. Our simulation results verify the convergence performance and the fairness achieved among links, and particularly reveal a new tradeoff between the network energy efficiency and fairness by comparing the MEP with the existing algorithms.