An artificial neural network (ANN) is a flexible mathematical structure which is capable of identifying complex nonlinear relationships between input and output data sets. ANN models have been found useful and efficient, particularly in problems for which the characteristics of the processes are difficult to describe using physical equations. This study presents a new procedure (entitled linear least squares simplex, or LLSSIM) for identifying the structure and parameters of three-layer feed forward ANN models and demonstrates the potential of such models for simulating the nonlinear hydrologic behavior of watersheds. The nonlinear ANN model approach is shown to provide a better representation of the rainfall-runoff relationship of the medium-size Leaf River basin near Collins, Mississippi, than the linear ARMAX (autoregressive moving average with exogenous inputs) time series approach or the conceptual SAC-SMA (Sacramento soil moisture accounting) model. Because the ANN approach presented here does not provide models that have physically realistic components and parameters, it is by no means a substitute for conceptual watershed modeling. However, the ANN approach does provide a viable and effective alternative to the ARMAX time series approach for developing input-output simulation and forecasting models in situations that do not require modeling of the internal structure of the watershed.

Artificial Neural Network Modeling of the Rainfall‐Runoff Process

We review algorithms developed for nonnegative matrix factorization (NMF) and nonnegative tensor factorization (NTF) from a unified view based on the block coordinate descent (BCD) framework. NMF and NTF are low-rank approximation methods for matrices and tensors in which the low-rank factors are constrained to have only nonnegative elements. The nonnegativity constraints have been shown to enable natural interpretations and allow better solutions in numerous applications including text analysis, computer vision, and bioinformatics. However, the computation of NMF and NTF remains challenging and expensive due the constraints. Numerous algorithmic approaches have been proposed to efficiently compute NMF and NTF. The BCD framework in constrained non-linear optimization readily explains the theoretical convergence properties of several efficient NMF and NTF algorithms, which are consistent with experimental observations reported in literature. In addition, we discuss algorithms that do not fit in the BCD framework contrasting them from those based on the BCD framework. With insights acquired from the unified perspective, we also propose efficient algorithms for updating NMF when there is a small change in the reduced dimension or in the data. The effectiveness of the proposed updating algorithms are validated experimentally with synthetic and real-world data sets.

/pdf/algorithms-for-nonnegative-matrix-and-tensor-factorizations-2lc6tn5b1i.pdf

Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

Erratum to Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints

A survey of the development of algorithms for enforcing nonnegativity constraints in scientific computation is given. Special emphasis is placed on such constraints in least squares computations in numerical linear algebra and in nonlinear optimization. Techniques involving nonnegative low-rank matrix and tensor factorizations are also emphasized. Details are provided for some important classical and modern applications in science and engineering. For completeness, this report also includes an effort toward a literature survey of the various algorithms and applications of nonnegativity constraints in numerical analysis.

http://users.wfu.edu/plemmons/papers/nonneg.pdf

Nonnegativity constraints in numerical analysis.

Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties

An affine scaling trust-region approach to bound-constrained nonlinear systems

In this paper a Matlab solver for constrained nonlinear equations is presented. The code, called STRSCNE, is based on the affine scaling trust-region method STRN, recently proposed by the authors. The approach taken in implementing the key steps of the method is discussed. The structure and the usage of STRSCNE are described and its features and capabilities are illustrated by numerical experiments. The results of a comparison with high quality codes for nonlinear optimization are shown.

STRSCNE: A Scaled Trust-Region Solver for Constrained Nonlinear Equations

Trust-region quadratic methods for nonlinear systems of mixed equalities and inequalities

An interior point approach for medium and large non-negative linear least-squares problems is proposed. Global and locally quadratic convergence is shown even if a degenerate solution is approached. Viable approaches for implementation are discussed and numerical results are provided. Copyright © 2006 John Wiley & Sons, Ltd.

http://www.optimization-online.org/DB_FILE/2005/12/1259.pdf

An interior point Newton‐like method for non‐negative least‐squares problems with degenerate solution

In this paper, we present a numerical method for solving linear and nonlinear second-order singularly perturbed boundary-value-problems. For linear problems, the method comes from the well-known WKB method. The required approximate solution is obtained by solving the reduced problem and one or two suitable initial-value problems, directly deduced from the given problem. For nonlinear problems, the quasilinearization method is applied. Numerical results are given showing the accuracy and feasibility of the proposed method.

M. Macconi

Papers

An affine scaling trust-region approach to bound-constrained nonlinear systems

STRSCNE: A Scaled Trust-Region Solver for Constrained Nonlinear Equations

Trust-region quadratic methods for nonlinear systems of mixed equalities and inequalities

An interior point Newton‐like method for non‐negative least‐squares problems with degenerate solution

Initial-value methods for second-order singularly perturbed boundary-value problems