Showing papers on "Affine transformation published in 2019"

Semantic Image Synthesis With Spatially-Adaptive Normalization

Abstract: We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout. Previous methods directly feed the semantic layout as input to the network, forcing the network to memorize the information throughout all the layers. Instead, we propose using the input layout for modulating the activations in normalization layers through a spatially-adaptive, learned affine transformation. Experiments on several challenging datasets demonstrate the superiority of our method compared to existing approaches, regarding both visual fidelity and alignment with input layouts. Finally, our model allows users to easily control the style and content of image synthesis results as well as create multi-modal results. Code is available upon publication.

An abstract domain for certifying neural networks

Abstract: We present a novel method for scalable and precise certification of deep neural networks. The key technical insight behind our approach is a new abstract domain which combines floating point polyhedra with intervals and is equipped with abstract transformers specifically tailored to the setting of neural networks. Concretely, we introduce new transformers for affine transforms, the rectified linear unit (ReLU), sigmoid, tanh, and maxpool functions. We implemented our method in a system called DeepPoly and evaluated it extensively on a range of datasets, neural architectures (including defended networks), and specifications. Our experimental results indicate that DeepPoly is more precise than prior work while scaling to large networks. We also show how to combine DeepPoly with a form of abstraction refinement based on trace partitioning. This enables us to prove, for the first time, the robustness of the network when the input image is subjected to complex perturbations such as rotations that employ linear interpolation.

First Order Motion Model for Image Animation

Abstract: Image animation consists of generating a video sequence so that an object in a source image is animated according to the motion of a driving video. Our framework addresses this problem without using any annotation or prior information about the specific object to animate. Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. To achieve this, we decouple appearance and motion information using a self-supervised formulation. To support complex motions, we use a representation consisting of a set of learned keypoints along with their local affine transformations. A generator network models occlusions arising during target motions and combines the appearance extracted from the source image and the motion derived from the driving video. Our framework scores best on diverse benchmarks and on a variety of object categories.

Formal verification of neural network controlled autonomous systems

Abstract: In this paper, we consider the problem of formally verifying the safety of an autonomous robot equipped with a Neural Network (NN) controller that processes LiDAR images to produce control actions. Given a workspace that is characterized by a set of polytopic obstacles, our objective is to compute the set of safe initial states such that a robot trajectory starting from these initial states is guaranteed to avoid the obstacles. Our approach is to construct a finite state abstraction of the system and use standard reachability analysis over the finite state abstraction to compute the set of safe initial states. To mathematically model the imaging function, that maps the robot position to the LiDAR image, we introduce the notion of imaging-adapted partitions of the workspace in which the imaging function is guaranteed to be affine. Given this workspace partitioning, a discrete-time linear dynamics of the robot, and a pre-trained NN controller with Rectified Linear Unit (ReLU) non-linearity, we utilize a Satisfiability Modulo Convex (SMC) encoding to enumerate all the possible assignments of different ReLUs. To accelerate this process, we develop a pre-processing algorithm that could rapidly prune the space of feasible ReLU assignments. Finally, we demonstrate the efficiency of the proposed algorithms using numerical simulations with the increasing complexity of the neural network controller.

Off-Policy Interleaved $Q$ -Learning: Optimal Control for Affine Nonlinear Discrete-Time Systems

Abstract: In this paper, a novel off-policy interleaved Q-learning algorithm is presented for solving optimal control problem of affine nonlinear discrete-time (DT) systems, using only the measured data along the system trajectories. Affine nonlinear feature of systems, unknown dynamics, and off-policy learning approach pose tremendous challenges on approximating optimal controllers. To this end, on-policy Q-learning method for optimal control of affine nonlinear DT systems is reviewed first, and its convergence is rigorously proven. The bias of solution to Q-function-based Bellman equation caused by adding probing noises to systems for satisfying persistent excitation is also analyzed when using on-policy Q-learning approach. Then, a behavior control policy is introduced followed by proposing an off-policy Q-learning algorithm. Meanwhile, the convergence of algorithm and no bias of solution to optimal control problem when adding probing noise to systems are investigated. Third, three neural networks run by the interleaved Q-learning approach in the actor-critic framework. Thus, a novel off-policy interleaved Q-learning algorithm is derived, and its convergence is proven. Simulation results are given to verify the effectiveness of the proposed method.

Affine Volterra processes

Abstract: We introduce affine Volterra processes, defined as solutions of certain stochastic convolution equations with affine coefficients. Classical affine diffusions constitute a special case, but affine Volterra processes are neither semimartingales, nor Markov processes in general. We provide explicit exponential-affine representations of the Fourier--Laplace functional in terms of the solution of an associated system of deterministic integral equations, extending well-known formulas for classical affine diffusions. For specific state spaces, we prove existence, uniqueness, and invariance properties of solutions of the corresponding stochastic convolution equations. Our arguments avoid infinite-dimensional stochastic analysis as well as stochastic integration with respect to non-semimartingales, relying instead on tools from the theory of finite-dimensional deterministic convolution equations. Our findings generalize and simplify recent results in the literature on rough volatility models in finance.

Guided Image-to-Image Translation With Bi-Directional Feature Transformation

Abstract: We address the problem of guided image-to-image translation where we translate an input image into another while respecting the constraints provided by an external, user-provided guidance image. Various types of conditioning mechanisms for leveraging the given guidance image have been explored, including input concatenation, feature concatenation, and conditional affine transformation of feature activations. All these conditioning mechanisms, however, are uni-directional, i.e., no information flow from the input image back to the guidance. To better utilize the constraints of the guidance image, we present a bi-directional feature transformation (bFT) scheme. We show that our novel bFT scheme outperforms other conditioning schemes and has comparable results to state-of-the-art methods on different tasks.

Practical Stabilization of Switched Affine Systems With Dwell-Time Guarantees

Abstract: Using a hybrid systems approach, we address the practical stabilization of operating points for switched affine systems, ensuring a minimum dwell time and an admissible chattering around the operating point. Two different solutions are shown to induce uniform dwell time, based on time or space regularization. The proposed solutions provide useful tuning knobs to separately adjust the switching frequency during transients and at the steady state. The strengths of the method are illustrated by simulating a boost converter.

Motion vector prediction for affine motion models in video coding

Abstract: A video decoder selects a source affine block. The source affine block is an affine-coded block that spatially neighbors a current block. Additionally, the video decoder extrapolates motion vectors of control points of the source affine block to determine motion vector predictors for control points of the current block. The video decoder inserts, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block. The video decoder also determines, based on an index signaled in a bitstream, a selected affine MVP set in the affine MVP set candidate list. The video decoder obtains, from the bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set.

Networks for Joint Affine and Non-Parametric Image Registration

Abstract: We introduce an end-to-end deep-learning framework for 3D medical image registration. In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model. Specifically, it consists of three stages. In the first stage, a multi-step affine network predicts affine transform parameters. In the second stage, we use a U-Net-like network to generate a momentum, from which a velocity field can be computed via smoothing. Finally, in the third stage, we employ a self-iterable map-based vSVF component to provide a non-parametric refinement based on the current estimate of the transformation map. Once the model is trained, a registration is completed in one forward pass. To evaluate the performance, we conducted longitudinal and cross-subject experiments on 3D magnetic resonance images (MRI) of the knee of the Osteoarthritis Initiative (OAI) dataset. Results show that our framework achieves comparable performance to state-of-the-art medical image registration approaches, but it is much faster, with a better control of transformation regularity including the ability to produce approximately symmetric transformations, and combining affine as well as non-parametric registration.

An Improved Framework of Affine Motion Compensation in Video Coding

Abstract: Affine motion compensation (AMC) is a promising coding tool in Joint Exploration Model (JEM) developed by the Joint Video Exploration Team. AMC in JEM employs a 4-parameter affine model between the current block and its reference block. With this model, the motion vectors (MV) of each sub-block can be derived from the MVs at two control points. In this paper, we present a practical framework to further improve the AMC in JEM. First, we introduce a multi-model AMC approach, which allows the encoder to select either the 4-parameter affine model or the 6-parameter affine model adaptively. Second, we improve the affine inter-mode in two aspects. For the normative part, we present an efficient affine motion coding method, which replaces the affine MV Prediction (MVP) candidates in JEM with more accurate but simpler ones, and employs a second-order MVP. For the non-normative part, we enhance the motion estimation process for AMC, by regulating the optimization algorithm. Finally, we propose to unify the affine merge-mode and the normal merge-mode into a unified merge-mode, which combine affine merge candidates and normal merge candidates in a single merge candidate list. Partial of these methods have been adopted into the next generation video coding standard named Versatile Video Coding. Simulation results show that the proposed methods can achieve 1.67% BD rate savings in average for the random access configurations.

Second-Order Continuous-Time Algorithm for Optimal Resource Allocation in Power Systems

Abstract: In this paper, based on differential inclusions and the saddle point dynamics, a novel second-order continuous-time algorithm is proposed to solve the optimal resource allocation problem in power systems. The considered cost function is the sum of all local cost functions with a set of affine equality demand constraints and an inequality constraint on generating capacity of the generator. In virtue of nonsmooth analysis, geometric graph theory, and Lyapunov stability theory, all generators achieve consensus on the Lagrange multipliers associated with a set of affine equality constraints while the proposed algorithm converges exponentially to the optimal solution of the resource allocation problem starting from any initial states over an undirected and connected graph. Moreover, the obtained results can be further extended to the optimal resource allocation problem in case of switching communication topologies. Finally, two numerical examples involving a smart grid system composed of five generators and the IEEE 30-bus system demonstrate the effectiveness and the performance of the theoretical results.

Breast cancer detection using synthetic mammograms from generative adversarial networks in convolutional neural networks.

Abstract: The convolutional neural network (CNN) is a promising technique to detect breast cancer based on mammograms. Training the CNN from scratch, however, requires a large amount of labeled data. Such a requirement usually is infeasible for some kinds of medical image data such as mammographic tumor images. Because improvement of the performance of a CNN classifier requires more training data, the creation of new training images, image augmentation, is one solution to this problem. We applied the generative adversarial network (GAN) to generate synthetic mammographic images from the digital database for screening mammography (DDSM). From the DDSM, we cropped two sets of regions of interest (ROIs) from the images: normal and abnormal (cancer/tumor). Those ROIs were used to train the GAN, and the GAN then generated synthetic images. For comparison with the affine transformation augmentation methods, such as rotation, shifting, scaling, etc., we used six groups of ROIs [three simple groups: affine augmented, GAN synthetic, real (original), and three mixture groups of any two of the three simple groups] for each to train a CNN classifier from scratch. And, we used real ROIs that were not used in training to validate classification outcomes. Our results show that, to classify the normal ROIs and abnormal ROIs from DDSM, adding GAN-generated ROIs in the training data can help the classifier prevent overfitting, and on validation accuracy, the GAN performs about 3.6% better than affine transformations for image augmentation. Therefore, GAN could be an ideal augmentation approach. The images augmented by GAN or affine transformation cannot substitute for real images to train CNN classifiers because the absence of real images in the training set will cause over-fitting.

Universal approximations of permutation invariant/equivariant functions by deep neural networks

Abstract: In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine transformations are $G$-equivariant/invariant. Due to representation theory, we can show that this approximator has exponentially fewer free parameters than usual models.

Assembling integrable sigma-models as affine Gaudin models

Abstract: We explain how to obtain new classical integrable field theories by assembling two affine Gaudin models into a single one. We show that the resulting affine Gaudin model depends on a parameter γ in such a way that the limit γ → 0 corresponds to the decoupling limit. Simple conditions ensuring Lorentz invariance are also presented. A first application of this method for σ-models leads to the action announced in [1] and which couples an arbitrary number N of principal chiral model fields on the same Lie group, each with a Wess-Zumino term. The affine Gaudin model descriptions of various integrable σ-models that can be used as elementary building blocks in the assembling construction are then given. This is in particular used in a second application of the method which consists in assembling N − 1 copies of the principal chiral model each with a Wess-Zumino term and one homogeneous Yang-Baxter deformation of the principal chiral model.

Convolutional Neural Network Architecture for Geometric Matching

Abstract: We address the problem of determining correspondences between two images in agreement with a geometric model such as an affine, homography or thin-plate spline transformation, and estimating its parameters. The contributions of this work are three-fold. First, we propose a convolutional neural network architecture for geometric matching. The architecture is based on three main components that mimic the standard steps of feature extraction, matching and simultaneous inlier detection and model parameter estimation, while being trainable end-to-end. Second, we demonstrate that the network parameters can be trained from synthetically generated imagery without the need for manual annotation and that our matching layer significantly increases generalization capabilities to never seen before images. Finally, we show that the same model can perform both instance-level and category-level matching giving state-of-the-art results on the challenging PF, TSS and Caltech-101 datasets.

Integrable Coupled σ Models.

Abstract: A systematic procedure for constructing classical integrable field theories with arbitrarily many free parameters is outlined. It is based on the recent interpretation of integrable field theories as realizations of affine Gaudin models. In this language, one can associate integrable field theories with affine Gaudin models having arbitrarily many sites. We present the result of applying this general procedure to couple together an arbitrary number of principal chiral model fields on the same Lie group, each with a Wess-Zumino term.

Extreme Learning Machine With Affine Transformation Inputs in an Activation Function

Abstract: The extreme learning machine (ELM) has attracted much attention over the past decade due to its fast learning speed and convincing generalization performance. However, there still remains a practical issue to be approached when applying the ELM: the randomly generated hidden node parameters without tuning can lead to the hidden node outputs being nonuniformly distributed, thus giving rise to poor generalization performance. To address this deficiency, a novel activation function with an affine transformation (AT) on its input is introduced into the ELM, which leads to an improved ELM algorithm that is referred to as an AT-ELM in this paper. The scaling and translation parameters of the AT activation function are computed based on the maximum entropy principle in such a way that the hidden layer outputs approximately obey a uniform distribution. Application of the AT-ELM algorithm in nonlinear function regression shows its robustness to the range scaling of the network inputs. Experiments on nonlinear function regression, real-world data set classification, and benchmark image recognition demonstrate better performance for the AT-ELM compared with the original ELM, the regularized ELM, and the kernel ELM. Recognition results on benchmark image data sets also reveal that the AT-ELM outperforms several other state-of-the-art algorithms in general.

Non-intrusive Sparse Subspace Learning for Parametrized Problems

Abstract: We discuss the use of hierarchical collocation to approximate the numerical solution of parametric models. With respect to traditional projection-based reduced order modeling, the use of a collocation enables non-intrusive approach based on sparse adaptive sampling of the parametric space. This allows to recover the low-dimensional structure of the parametric solution subspace while also learning the functional dependency from the parameters in explicit form. A sparse low-rank approximate tensor representation of the parametric solution can be built through an incremental strategy that only needs to have access to the output of a deterministic solver. Non-intrusiveness makes this approach straightforwardly applicable to challenging problems characterized by nonlinearity or non affine weak forms. As we show in the various examples presented in the paper, the method can be interfaced with no particular effort to existing third party simulation software making the proposed approach particularly appealing and adapted to practical engineering problems of industrial interest.

Numerical verification of affine systems with up to a billion dimensions

Abstract: Affine systems reachability is the basis of many verification methods. With further computation, methods exist to reason about richer models with inputs, nonlinear differential equations, and hybrid dynamics. As such, the scalability of affine systems verification is a prerequisite to scalable analysis for more complex systems. In this paper, we improve the scalability of affine systems verification, in terms of the number of dimensions (variables) in the system. The reachable states of affine systems can be written in terms of the matrix exponential, and safety checking can be performed at specific time steps with linear programming. Unfortunately, for large systems with many state variables, this direct approach requires an intractable amount of memory while using an intractable amount of computation time. We overcome these challenges by combining several methods that leverage common problem structure. Memory is reduced by exploiting initial states that are not full-dimensional and safety properties (outputs) over a few linear projections of the state variables. Computation time is saved by using numerical simulations to compute only projections of the matrix exponential relevant for the verification problem. Since large systems often have sparse dynamics, we use fast Krylov-subspace simulation methods based on the Arnoldi or Lanczos iterations. Our implementation produces accurate counter-examples when properties are violated and, in the extreme case with sufficient problem structure, is shown to analyze a system with one billion real-valued state variables.

Loading Factor Estimation Under Affine Constraints on the Covariance Eigenvalues With Application to Radar Target Detection

Abstract: Maximum likelihood (ML) estimation of the loading factor under affine constraints on the covariance eigenvalues is addressed. Several situations of practical interest for radar are considered, and the corresponding ML solutions to the loading factor estimation problem are derived in closed form. Furthermore, it is shown that the constrained ML problem, the constrained geometric approach, and the constrained problem of mean square error minimization (with respect to the loading factor) all lead to the same solution. At the analysis stage, the effectiveness of the resulting covariance estimators is evaluated in terms of both the signal-to-interference-plus-noise ratio and the receiving beampattern shape and compared with that of other covariance estimation methods available in the literature. Finally, a receiving architecture based on the adaptive matched filter that exploits the new loaded covariance estimators is also considered to assess the benefits of the new strategies in terms of detection probability.

Fusion categories for affine vertex algebras at admissible levels

Abstract: The main result is that the category of ordinary modules of an affine vertex operator algebra of a simply laced Lie algebra at admissible level is rigid and thus a braided fusion category. If the level satisfies a certain coprime property then it is even a modular tensor category. In all cases open Hopf links coincide with the corresponding normalized S-matrix entries of torus one-point functions. This is interpreted as a Verlinde formula beyond rational vertex operator algebras. A preparatory Theorem is a convenient formula for the fusion rules of rational principal W-algebras of any type.

Affine forward variance models

Abstract: We introduce the class of affine forward variance (AFV) models of which both the conventional Heston model and the rough Heston model are special cases. We show that AFV models can be characterised by the affine form of their cumulant-generating function, which can be obtained as solution of a convolution Riccati equation. We further introduce the class of affine forward order flow intensity (AFI) models, which are structurally similar to AFV models, but driven by jump processes, and which include Hawkes-type models. We show that the cumulant-generating function of an AFI model satisfies a generalised convolution Riccati equation and that a high-frequency limit of AFI models converges in distribution to an AFV model.

Networks for Joint Affine and Non-parametric Image Registration

Abstract: We introduce an end-to-end deep-learning framework for 3D medical image registration. In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model. Specifically, it consists of three stages. In the first stage, a multi-step affine network predicts affine transform parameters. In the second stage, we use a Unet-like network to generate a momentum, from which a velocity field can be computed via smoothing. Finally, in the third stage, we employ a self-iterable map-based vSVF component to provide a non-parametric refinement based on the current estimate of the transformation map. Once the model is trained, a registration is completed in one forward pass. To evaluate the performance, we conducted longitudinal and cross-subject experiments on 3D magnetic resonance images (MRI) of the knee of the Osteoarthritis Initiative (OAI) dataset. Results show that our framework achieves comparable performance to state-of-the-art medical image registration approaches, but it is much faster, with a better control of transformation regularity including the ability to produce approximately symmetric transformations, and combining affine and non-parametric registration.

Graphical Affine Algebra

Abstract: Graphical linear algebra is a diagrammatic language allowing to reason compositionally about different types of linear computing devices. In this paper, we extend this formalism with a connector for affine behaviour. The extension, which we call graphical affine algebra, is simple but remarkably powerful: it can model systems with richer patterns of behaviour such as mutual exclusion-with modules over the natural numbers as semantic domain-or non-passive electrical components-when considering modules over a certain field. Our main technical contribution is a complete axiomatisation for graphical affine algebra over these two interpretations. We also show, as case studies, how graphical affine algebra captures electrical circuits and the calculus of stateless connectors-a coordination language for distributed systems.