Registration is a fundamental task in image processing used to match two or more pictures taken, for example, at different times, from different sensors, or from different viewpoints. Virtually all large systems which evaluate images require the registration of images, or a closely related operation, as an intermediate step. Specific examples of systems where image registration is a significant component include matching a target with a real-time image of a scene for target recognition, monitoring global land usage using satellite images, matching stereo images to recover shape for autonomous navigation, and aligning images from different medical modalities for diagnosis.Over the years, a broad range of techniques has been developed for various types of data and problems. These techniques have been independently studied for several different applications, resulting in a large body of research. This paper organizes this material by establishing the relationship between the variations in the images and the type of registration techniques which can most appropriately be applied. Three major types of variations are distinguished. The first type are the variations due to the differences in acquisition which cause the images to be misaligned. To register images, a spatial transformation is found which will remove these variations. The class of transformations which must be searched to find the optimal transformation is determined by knowledge about the variations of this type. The transformation class in turn influences the general technique that should be taken. The second type of variations are those which are also due to differences in acquisition, but cannot be modeled easily such as lighting and atmospheric conditions. This type usually effects intensity values, but they may also be spatial, such as perspective distortions. The third type of variations are differences in the images that are of interest such as object movements, growths, or other scene changes. Variations of the second and third type are not directly removed by registration, but they make registration more difficult since an exact match is no longer possible. In particular, it is critical that variations of the third type are not removed. Knowledge about the characteristics of each type of variation effect the choice of feature space, similarity measure, search space, and search strategy which will make up the final technique. All registration techniques can be viewed as different combinations of these choices. This framework is useful for understanding the merits and relationships between the wide variety of existing techniques and for assisting in the selection of the most suitable technique for a specific problem.

A survey of image registration techniques

Big Data applications are typically associated with systems involving large numbers of users, massive complex software systems, and large-scale heterogeneous computing and storage architectures. The construction of such systems involves many distributed design choices. The end products (e.g., recommendation systems, medical analysis tools, real-time game engines, speech recognizers) thus involve many tunable configuration parameters. These parameters are often specified and hard-coded into the software by various developers or teams. If optimized jointly, these parameters can result in significant improvements. Bayesian optimization is a powerful tool for the joint optimization of design choices that is gaining great popularity in recent years. It promises greater automation so as to increase both product quality and human productivity. This review paper introduces Bayesian optimization, highlights some of its methodological aspects, and showcases a wide range of applications.

/pdf/taking-the-human-out-of-the-loop-a-review-of-bayesian-4olcznoudw.pdf

Taking the Human Out of the Loop: A Review of Bayesian Optimization

http://ee.sharif.edu/~miap/Files/A%20Survey%20of%20Medical%20Image%20Registration.pdf

A survey of medical image registration.

Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel approaches to feature learning. Traditionally, hyper-parameter optimization has been the job of humans because they can be very efficient in regimes where only a few trials are possible. Presently, computer clusters and GPU processors make it possible to run more trials and we show that algorithmic approaches can find better results. We present hyper-parameter optimization results on tasks of training neural networks and deep belief networks (DBNs). We optimize hyper-parameters using random search and two new greedy sequential methods based on the expected improvement criterion. Random search has been shown to be sufficiently efficient for learning neural networks for several datasets, but we show it is unreliable for training DBNs. The sequential algorithms are applied to the most difficult DBN learning problems from [1] and find significantly better results than the best previously reported. This work contributes novel techniques for making response surface models P(y|x) in which many elements of hyper-parameter assignment (x) are known to be irrelevant given particular values of other elements.

/pdf/algorithms-for-hyper-parameter-optimization-5bvk0b44nx.pdf

Algorithms for Hyper-Parameter Optimization

The successful application of a conceptual rainfall-runoff (CRR) model depends on how well it is calibrated. Despite the popularity of CRR models, reports in the literature indicate that it is typically difficult, if not impossible, to obtain unique optimal values for their parameters using automatic calibration methods. Unless the best set of parameters associated with a given calibration data set can be found, it is difficult to determine how sensitive the parameter estimates (and hence the model forecasts) are to factors such as input and output data error, model error, quantity and quality of data, objective function used, and so on. Results are presented that establish clearly the nature of the multiple optima problem for the research CRR model SIXPAR. These results suggest that the CRR model optimization problem is more difficult than had been previously thought and that currently used local search procedures have a very low probability of successfully finding the optimal parameter sets. Next, the performance of three existing global search procedures are evaluated on the model SIXPAR. Finally, a powerful new global optimization procedure is presented, entitled the shuffled complex evolution (SCE-UA) method, which was able to consistently locate the global optimum of the SIXPAR model, and appears to be capable of efficiently and effectively solving the CRR model optimization problem.

Effective and efficient global optimization for conceptual rainfall‐runoff models

Applied Interval Analysis

Identification of parametric models : from experimental data

I. Introduction.- 1. Introduction.- 1.1 What Are the Key Concepts?.- 1.2 How Did the Story Start?.- 1.3 What About Complexity?.- 1.4 How is the Book Organized?.- II. Tools.- 2. Interval Analysis.- 2.1 Introduction.- 2.2 Operations on Sets.- 2.2.1 Purely set-theoretic operations.- 2.2.2 Extended operations.- 2.2.3 Properties of set operators.- 2.2.4 Wrappers.- 2.3 Interval Analysis.- 2.3.1 Intervals.- 2.3.2 Interval computation.- 2.3.3 Closed intervals.- 2.3.4 Interval vectors.- 2.3.5 Interval matrices.- 2.4 Inclusion Functions.- 2.4.1 Definitions.- 2.4.2 Natural inclusion functions.- 2.4.3 Centred inclusion functions.- 2.4.4 Mixed centred inclusion functions.- 2.4.5 Taylor inclusion functions.- 2.4.6 Comparison.- 2.5 Inclusion Tests.- 2.5.1 Interval Booleans.- 2.5.2 Tests.- 2.5.3 Inclusion tests for sets.- 2.6 Conclusions.- 3. Subpavings.- 3.1 Introduction.- 3.2 Set Topology.- 3.2.1 Distances between compact sets.- 3.2.2 Enclosure of compact sets between subpavings.- 3.3 Regular Subpavings.- 3.3.1 Pavings and subpavings.- 3.3.2 Representing a regular subpaving as a binary tree.- 3.3.3 Basic operations on regular subpavings.- 3.4 Implementation of Set Computation.- 3.4.1 Set inversion.- 3.4.2 Image evaluation.- 3.5 Conclusions.- 4. Contractors.- 4.1 Introduction.- 4.2 Basic Contractors.- 4.2.1 Finite subsolvers.- 4.2.2 Intervalization of finite subsolvers.- 4.2.3 Fixed-point methods.- 4.2.4 Forward-backward propagation.- 4.2.5 Linear programming approach.- 4.3 External Approximation.- 4.3.1 Principle.- 4.3.2 Preconditioning.- 4.3.3 Newton contractor.- 4.3.4 Parallel linearization.- 4.3.5 Using formal transformations.- 4.4 Collaboration Between Contractors.- 4.4.1 Principle.- 4.4.2 Contractors and inclusion functions.- 4.5 Contractors for Sets.- 4.5.1 Definitions.- 4.5.2 Sets defined by equality and inequality constraints.- 4.5.3 Improving contractors using local search.- 4.6 Conclusions.- 5. Solvers.- 5.1 Introduction.- 5.2 Solving Square Systems of Non-linear Equations.- 5.3 Characterizing Sets Defined by Inequalities.- 5.4 Interval Hull of a Set Defined by Inequalities.- 5.4.1 First approach.- 5.4.2 Second approach.- 5.5 Global Optimization.- 5.5.1 The Moore-Skelboe algorithm.- 5.5.2 Hansen's algorithm.- 5.5.3 Using interval constraint propagation.- 5.6 Minimax Optimization.- 5.6.1 Unconstrained case.- 5.6.2 Constrained case.- 5.6.3 Dealing with quantifiers.- 5.7 Cost Contours.- 5.8 Conclusions.- III. Applications.- 6. Estimation.- 6.1 Introduction.- 6.2 Parameter Estimation Via Optimization.- 6.2.1 Least-square parameter estimation in compartmental modelling.- 6.2.2 Minimax parameter estimation.- 6.3 Parameter Bounding.- 6.3.1 Introduction.- 6.3.2 The values of the independent variables are known.- 6.3.3 Robustification against outliers.- 6.3.4 The values of the independent variables are uncertain.- 6.3.5 Computation of the interval hull of the posterior feasible set.- 6.4 State Bounding.- 6.4.1 Introduction.- 6.4.2 Bounding the initial state.- 6.4.3 Bounding all variables.- 6.4.4 Bounding by constraint propagation.- 6.5 Conclusions.- 7. Robust Control.- 7.1 Introduction.- 7.2 Stability of Deterministic Linear Systems.- 7.2.1 Characteristic polynomial.- 7.2.2 Routh criterion.- 7.2.3 Stability degree.- 7.3 Basic Tests for Robust Stability.- 7.3.1 Interval polynomials.- 7.3.2 Polytope polynomials.- 7.3.3 Image-set polynomials.- 7.3.4 Conclusion.- 7.4 Robust Stability Analysis.- 7.4.1 Stability domains.- 7.4.2 Stability degree.- 7.4.3 Value-set approach.- 7.4.4 Robust stability margins.- 7.4.5 Stability radius.- 7.5 Controller Design.- 7.6 Conclusions.- 8. Robotics.- 8.1 Introduction.- 8.2 Forward Kinematics Problem for Stewart-Gough Platforms.- 8.2.1 Stewart-Gough platforms.- 8.2.2 From the frame of the mobile plate to that of the base.- 8.2.3 Equations to be solved.- 8.2.4 Solution.- 8.3 Path Planning.- 8.3.1 Graph discretization of configuration space.- 8.3.2 Algorithms for finding a feasible path.- 8.3.3 Test case.- 8.4 Localization and Tracking of a Mobile Robot.- 8.4.1 Formulation of the static localization problem.- 8.4.2 Model of the measurement process.- 8.4.3 Set inversion.- 8.4.4 Dealing with outliers.- 8.4.5 Static localization example.- 8.4.6 Tracking.- 8.4.7 Example.- 8.5 Conclusions.- IV. Implementation.- 9. Automatic Differentiation.- 9.1 Introduction.- 9.2 Forward and Backward Differentiations.- 9.2.1 Forward differentiation.- 9.2.2 Backward differentiation.- 9.3 Differentiation of Algorithms.- 9.3.1 First assumption.- 9.3.2 Second assumption.- 9.3.3 Third assumption.- 9.4 Examples.- 9.4.1 Example 1.- 9.4.2 Example 2.- 9.5 Conclusions.- 10. Guaranteed Computation with Floating-point Numbers.- 10.1 Introduction.- 10.2 Floating-point Numbers and IEEE 754.- 10.2.1 Representation.- 10.2.2 Rounding.- 10.2.3 Special quantities.- 10.3 Intervals and IEEE 754.- 10.3.1 Machine intervals.- 10.3.2 Closed interval arithmetic.- 10.3.3 Handling elementary functions.- 10.3.4 Improvements.- 10.4 Interval Resources.- 10.5 Conclusions.- 11. Do It Yourself.- 11.1 Introduction.- 11.2 Notions of C++.- 11.2.1 Program structure.- 11.2.2 Standard types.- 11.2.3 Pointers.- 11.2.4 Passing parameters to a function.- 11.3 INTERVAL Class.- 11.3.1 Constructors and destructor.- 11.3.2 Other member functions.- 11.3.3 Mathematical functions.- 11.4 Intervals with PROFIL/BIAS.- 11.4.1 BIAS.- 11.4.2 PROFIL.- 11.4.3 Getting started.- 11.5 Exercises on Intervals.- 11.6 Interval Vectors.- 11.6.1 INTERVAL_VECTOR class.- 11.6.2 Constructors, assignment and function call operators.- 11.6.3 Friend functions.- 11.6.4 Utilities.- 11.7 Vectors with PROFIL/BIAS.- 11.8 Exercises on Interval Vectors.- 11.9 Interval Matrices.- 11.10 Matrices with PROFIL/BIAS.- 11.11 Exercises on Interval Matrices.- 11.12 Regular Subpavings with PROFIL/BIAS.- 11.12.1 NODE class.- 11.12.2 Set inversion with subpavings.- 11.12.3 Image evaluation with subpavings.- 11.12.4 System simulation and state estimation with subpavings.- 11.13 Error Handling.- 11.13.1 Using exit.- 11.13.2 Exception handling.- 11.13.3 Mathematical errors.- References.

https://hal.archives-ouvertes.fr/hal-00845131/document

Applied interval analysis : with examples in parameter and state estimation, robust control and robotics

Overview of the Volume J. Norton. Optimal Estimation Theory for Dynamic System with Set Membership Uncertainty: An Overview M. Milanese, A. Vicino. Solving Linear Problems in the Presence of Bounded Data Perturbations B.Z. Kacewicz. A Review and a Comparison of Ellipsoidal Bounding Algorithms G. Favier, L.V.R. Arruda. On the Deadzone in System Identification K. Forsman, L. Ljung. Recursive Estimation Algorithms for Linear Models with Set Membership Error G. Belforte, T.T. Tay. Transfer Function Parameter Interval Estimation Using Recursive Least Squares in the Time and Frequency Domains P.O. Gutman. Volume-optimal Inner and Outer Ellipsoids L. Pronzato, E. Walter. Linear Interpolation and Estimation Using Interval Analysis S.M. Markov, E.D. Popova. Adaptive Approximation of Uncertainty Sets for Linear Regression Models A. Vicino, G. Zappa. Worstcase l1 Identification M. Milanese. Recursive Robust Minimax Estimation E. Walter, H. Piet-Lahanier. Robustness to Outliers of Bounded-error Estimators, Consequences on Experiment Design L. Pronzato, E. Walter. Ellipsoidal State Estimation for Uncertain Dynamical Systems T.F. Filipova, et al. Set-valued Estimation of State and Parameter Vectors within Adaptive Control-Systems V.M. Kuntsevich. Limited-complexity Polyhedric Tracking H. Piet-Lahanier, E. Walter. Parameterbounding Algorithms for Linear Errors in Variables Models S.M. Veres, J.P. Norton. Errors-invariables Models in Parameter Bounding V. Cerone. Identification of Linear Objects with Bounded Disturbances in Both Input and Output Channels Yu.A. Merkuryev. Identification of Nonlinear Statespace Models by Deterministic Search J.P. Norton, S.M. Veres. Robust Identification and Prediction for Nonlinear State-Space Models with Bounded Output Error K.J. Keesman. Estimation Theory for Nonlinear Models and Set Membership Uncertainty M. Milanese, A. Vicino. Guaranteed Nonlinear Set Estimation via Interval Analysis L. Jaulin, E. Walter. On Adaptive Control of Systems Subjected to Bounded Disturbances L.S. Zhitecki. Predictive Selftuning Control by Parameter Bounding and Worstcase Design S.M. Veres, J.P. Norton. Estimation of a Mobile Robot Localization: Geometric Approaches D. Meizel, et al. Improved Image Compression Using Bounded Error Parameter Estimation Concepts A.K. Rao. Application of OBE Algorithms to Speech Analysis, Recognition and Coding J.R. Deller Jr., et al. 2 additional articles. Index.

Bounding Approaches to System Identification

https://www.ensta-bretagne.fr/jaulin/paper_automatica93.pdf

Eric Walter

Papers

Applied Interval Analysis

Identification of parametric models : from experimental data

Applied interval analysis : with examples in parameter and state estimation, robust control and robotics

Bounding Approaches to System Identification

Set inversion via interval analysis for nonlinear bounded-error estimation