Showing papers on "Bilinear interpolation published in 2018"

Bilinear Attention Networks

Abstract: Attention networks in multimodal learning provide an efficient way to utilize given visual information selectively. However, the computational cost to learn attention distributions for every pair of multimodal input channels is prohibitively expensive. To solve this problem, co-attention builds two separate attention distributions for each modality neglecting the interaction between multimodal inputs. In this paper, we propose bilinear attention networks (BAN) that find bilinear attention distributions to utilize given vision-language information seamlessly. BAN considers bilinear interactions among two groups of input channels, while low-rank bilinear pooling extracts the joint representations for each pair of channels. Furthermore, we propose a variant of multimodal residual networks to exploit eight-attention maps of the BAN efficiently. We quantitatively and qualitatively evaluate our model on visual question answering (VQA 2.0) and Flickr30k Entities datasets, showing that BAN significantly outperforms previous methods and achieves new state-of-the-arts on both datasets.

Bilinear Attention Networks

Abstract: Attention networks in multimodal learning provide an efficient way to utilize given visual information selectively. However, the computational cost to learn attention distributions for every pair of multimodal input channels is prohibitively expensive. To solve this problem, co-attention builds two separate attention distributions for each modality neglecting the interaction between multimodal inputs. In this paper, we propose bilinear attention networks (BAN) that find bilinear attention distributions to utilize given vision-language information seamlessly. BAN considers bilinear interactions among two groups of input channels, while low-rank bilinear pooling extracts the joint representations for each pair of channels. Furthermore, we propose a variant of multimodal residual networks to exploit eight-attention maps of the BAN efficiently. We quantitatively and qualitatively evaluate our model on visual question answering (VQA 2.0) and Flickr30k Entities datasets, showing that BAN significantly outperforms previous methods and achieves new state-of-the-arts on both datasets.

Bilinear Convolutional Neural Networks for Fine-Grained Visual Recognition

Abstract: We present a simple and effective architecture for fine-grained recognition called Bilinear Convolutional Neural Networks (B-CNNs) . These networks represent an image as a pooled outer product of features derived from two CNNs and capture localized feature interactions in a translationally invariant manner. B-CNNs are related to orderless texture representations built on deep features but can be trained in an end-to-end manner. Our most accurate model obtains 84.1, 79.4, 84.5 and 91.3 percent per-image accuracy on the Caltech-UCSD birds [1] , NABirds [2] , FGVC aircraft [3] , and Stanford cars [4] dataset respectively and runs at 30 frames-per-second on a NVIDIA Titan X GPU. We then present a systematic analysis of these networks and show that (1) the bilinear features are highly redundant and can be reduced by an order of magnitude in size without significant loss in accuracy, (2) are also effective for other image classification tasks such as texture and scene recognition, and (3) can be trained from scratch on the ImageNet dataset offering consistent improvements over the baseline architecture. Finally, we present visualizations of these models on various datasets using top activations of neural units and gradient-based inversion techniques. The source code for the complete system is available at http://vis-www.cs.umass.edu/bcnn .

Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition

Abstract: Fine-grained visual recognition is challenging because it highly relies on the modeling of various semantic parts and fine-grained feature learning. Bilinear pooling based models have been shown to be effective at fine-grained recognition, while most previous approaches neglect the fact that inter-layer part feature interaction and fine-grained feature learning are mutually correlated and can reinforce each other. In this paper, we present a novel model to address these issues. First, a cross-layer bilinear pooling approach is proposed to capture the inter-layer part feature relations, which results in superior performance compared with other bilinear pooling based approaches. Second, we propose a novel hierarchical bilinear pooling framework to integrate multiple cross-layer bilinear features to enhance their representation capability. Our formulation is intuitive, efficient and achieves state-of-the-art results on the widely used fine-grained recognition datasets.

Multi-view Harmonized Bilinear Network for 3D Object Recognition

Abstract: View-based methods have achieved considerable success in 3D object recognition tasks. Different from existing view-based methods pooling the view-wise features, we tackle this problem from the perspective of patches-to-patches similarity measurement. By exploiting the relationship between polynomial kernel and bilinear pooling, we obtain an effective 3D object representation by aggregating local convolutional features through bilinear pooling. Meanwhile, we harmonize different components inherited in the bilinear feature to obtain a more discriminative representation. To achieve an end-to-end trainable framework, we incorporate the harmonized bilinear pooling as a layer of a network, constituting the proposed Multi-view Harmonized Bilinear Network (MHBN). Systematic experiments conducted on two public benchmark datasets demonstrate the efficacy of the proposed methods in 3D object recognition.

Diversity of exact solutions to a (3+1)-dimensional nonlinear evolution equation and its reduction

Abstract: In this paper, a (3+1)-dimensional nonlinear evolution equation and its reduction is studied by use of the Hirota bilinear method and the test function method. With symbolic computation, diversity of exact solutions is obtained by solving the under-determined nonlinear system of algebraic equations for the associated parameters. Finally, analysis and graphical simulation are given to reveal the propagation and dynamical behavior of the solutions.

Novel solitary wave solutions for the (3+1)-dimensional extended Jimbo–Miwa equations

Abstract: This paper retrieves new periodic solitary wave solutions for the ( 3 + 1 ) -dimensional extended Jimbo–Miwa equations, based on the Hirota bilinear method, by utilizing Maple software. As a result, the Hirota bilinear method is successfully employed and acquired several classes of solitary wave solutions in terms of a new combination of exponential function, trigonometric function and hyperbolic functions. All solutions have been verified back into its corresponding equation by Maple. We depicted the physical explanation of the extracted solutions with the free choice of the different parameters by plotting some 3D and 2D illustrations. Finally, it has been illustrated that the executed method is robust and more efficient than other methods and the obtained solutions are trustworthy in the applied sciences.

State filtering-based least squares parameter estimation for bilinear systems using the hierarchical identification principle

Abstract: This study presents a combined parameter and state estimation algorithm for a bilinear system described by its observer canonical state-space model based on the hierarchical identification principle. The Kalman filter is known as the best state filter for linear systems, but not applicable for bilinear systems. Thus, a bilinear state observer (BSO) is designed to give the state estimates using the extremum principle. Then a BSO-based recursive least squares (BSO-RLS) algorithm is developed. For comparison with the BSO-RLS algorithm, by dividing the system into three fictitious subsystems on the basis of the decomposition–coordination principle, a BSO-based hierarchical least squares algorithm is proposed to reduce the computation burden. Moreover, a BSO-based forgetting factor recursive least squares algorithm is presented to improve the parameter tracking capability. Finally, a numerical example illustrates the effectiveness of the proposed algorithms.

Combined state and parameter estimation for a bilinear state space system with moving average noise

Abstract: This paper considers the identification problem of bilinear systems with measurement noise in the form of the moving average model. In particular, we present an interactive estimation algorithm for unmeasurable states and parameters based on the hierarchical identification principle. For unknown states, we formulate a novel bilinear state observer from input-output measurements using the Kalman filter. Then a bilinear state observer based multi-innovation extended stochastic gradient (BSO-MI-ESG) algorithm is proposed to estimate the unknown system parameters. A linear filter is utilized to improve the parameter estimation accuracy and a filtering based BSO-MI-ESG algorithm is presented using the data filtering technique. In the numerical example, we illustrate the effectiveness of the proposed identification methods.

Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications

Abstract: The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment. And they can be well modeled by a hyper-Laplacian. However, the use of such distributions generally leads to challenging non-convex, non-smooth and non-Lipschitz problems, and makes existing algorithms very slow for large-scale applications. Together with the analytic solutions to $\ell _{p}$ -norm minimization with two specific values of $p$ , i.e., $p=1/2$ and $p=2/3$ , we propose two novel bilinear factor matrix norm minimization models for robust principal component analysis. We first define the double nuclear norm and Frobenius/nuclear hybrid norm penalties, and then prove that they are in essence the Schatten- $1/2$ and $2/3$ quasi-norms, respectively, which lead to much more tractable and scalable Lipschitz optimization problems. Our experimental analysis shows that both our methods yield more accurate solutions than original Schatten quasi-norm minimization, even when the number of observations is very limited. Finally, we apply our penalties to various low-level vision problems, e.g., text removal, moving object detection, image alignment and inpainting, and show that our methods usually outperform the state-of-the-art methods.

Lump solution and its interaction to (3+1)-D potential-YTSF equation

Abstract: This paper studies the $$(3+1)$$ -dimensional potential-Yu–Toda–Sasa–Fukuyama (YTSF) equation by implementing the Hirota bilinear method. As a consequence, the Hirota bilinear method is successfully employed and acquired a type of the lump solution and five types of interaction solutions in terms of a new merge of positive quadratic functions, trigonometric functions and hyperbolic functions. All solutions have been verified back into its corresponding equation by Maple. We depicted the physical explanation of the extracted solutions with the free choice of the different parameters by plotting some 3D and 2D illustrations. Finally, we believe that the executed method is robust and efficient than other methods and the obtained solutions are trustworthy in the applied sciences.

Grassmann Pooling as Compact Homogeneous Bilinear Pooling for Fine-Grained Visual Classification

Abstract: Designing discriminative and invariant features is the key to visual recognition. Recently, the bilinear pooled feature matrix of Convolutional Neural Network (CNN) has shown to achieve state-of-the-art performance on a range of fine-grained visual recognition tasks. The bilinear feature matrix collects second-order statistics and is closely related to the covariance matrix descriptor. However, the bilinear feature could suffer from the visual burstiness phenomenon similar to other visual representations such as VLAD and Fisher Vector. The reason is that the bilinear feature matrix is sensitive to the magnitudes and correlations of local CNN feature elements which can be measured by its singular values. On the other hand, the singular vectors are more invariant and reasonable to be adopted as the feature representation. Motivated by this point, we advocate an alternative pooling method which transforms the CNN feature matrix to an orthonormal matrix consists of its principal singular vectors. Geometrically, such orthonormal matrix lies on the Grassmann manifold, a Riemannian manifold whose points represent subspaces of the Euclidean space. Similarity measurement of images reduces to comparing the principal angles between these “homogeneous” subspaces and thus is independent of the magnitudes and correlations of local CNN activations. In particular, we demonstrate that the projection distance on the Grassmann manifold deduces a bilinear feature mapping without explicitly computing the bilinear feature matrix, which enables a very compact feature and classifier representation. Experimental results show that our method achieves an excellent balance of model complexity and accuracy on a variety of fine-grained image classification datasets.

Deep Bilinear Learning for RGB-D Action Recognition

Abstract: In this paper, we focus on exploring modality-temporal mutual information for RGB-D action recognition. In order to learn time-varying information and multi-modal features jointly, we propose a novel deep bilinear learning framework. In the framework, we propose bilinear blocks that consist of two linear pooling layers for pooling the input cube features from both modality and temporal directions, separately. To capture rich modality-temporal information and facilitate our deep bilinear learning, a new action feature called modality-temporal cube is presented in a tensor structure for characterizing RGB-D actions from a comprehensive perspective. Our method is extensively tested on two public datasets with four different evaluation settings, and the results show that the proposed method outperforms the state-of-the-art approaches.

Domination of multilinear singular integrals by positive sparse forms

Abstract: We establish a uniform domination of the family of trilinear multiplier forms with singularity over a one-dimensional subspace by positive sparse forms involving Lp -averages. This class includes the adjoint forms to the bilinear Hilbert transforms. Our result strengthens the Lp -boundedness proved by Muscalu, Tao and Thiele, and entails as a corollary a novel rich multilinear weighted theory. A particular case of this theory is the Lq1 (v1) × Lq2 (v2)boundedness of the bilinear Hilbert transform when the weights vj belong to the class A q+1 2 ∩ RH2. Our proof relies on a stopping time construction based on newly developed localized outer-Lp embedding theorems for the wave packet transform. In an Appendix, we show how our domination principle can be applied to recover the vector-valued bounds for the bilinear Hilbert transforms recently proved by Benea and Muscalu.

Backward Fuzzy Rule Interpolation

Abstract: Fuzzy rule interpolation offers a useful means to enhancing the robustness of fuzzy models by making inference possible in sparse rule-based systems. However, in real-world applications of interconnected rule bases, situations may arise when certain crucial antecedents are absent from given observations. If such missing antecedents were involved in the subsequent interpolation process, the final conclusion would not be deducible using conventional means. To address this important issue, a new approach named backward fuzzy rule interpolation and extrapolation (BFRIE) is proposed in this paper, allowing the observations, which directly relate to the conclusion to be inferred or interpolated from the known antecedents and conclusion. This approach supports both backward interpolation and extrapolation which involve multiple fuzzy rules, with each having multiple antecedents. As such, it significantly extends the existing fuzzy rule interpolation techniques. In particular, considering that there may be more than one antecedent value missing in an application problem, two methods are proposed in an attempt to perform backward interpolation with multiple missing antecedent values. Algorithms are given to implement the approaches via the use of the scale and move transformation-based fuzzy interpolation. Experimental studies that are based on a real-world scenario are provided to demonstrate the potential and efficacy of the proposed work.

Bilinear Residual Neural Network for the Identification and Forecasting of Geophysical Dynamics

Abstract: Due to the increasing availability of large-scale observations and simulation datasets, data-driven representations arise as efficient and relevant computation representations of geophysical systems for a wide range of applications, where model-driven models based on ordinary differential equations remain the state-of-the-art approaches. In this work, we investigate neural networks (NN) as physically-sound data-driven representations of such systems. Viewing Runge-Kutta methods as graphical models, we consider a residual NN architecture and introduce bilinear layers to embed non-linearities which are intrinsic features of geophysical systems. From numerical experiments for synthetic and real datasets, we demonstrate the relevance of the proposed NN-based architecture both in terms of forecasting performance and model identification.

On multi-relational link prediction with bilinear models

Abstract: We study bilinear embedding models for the task of multi-relational link prediction and knowledge graph completion. Bilinear models belong to the most basic models for this task, they are comparably efficient to train and use, and they can provide good prediction performance. The main goal of this paper is to explore the expressiveness of and the connections between various bilinear models proposed in the literature. In particular, a substantial number of models can be represented as bilinear models with certain additional constraints enforced on the embeddings. We explore whether or not these constraints lead to universal models, which can in principle represent every set of relations, and whether or not there are subsumption relationships between various models. We report results of an independent experimental study that evaluates recent bilinear models in a common experimental setup. Finally, we provide evidence that relation-level ensembles of multiple bilinear models can achieve state-of-the-art prediction performance.

Fast and Guaranteed Blind Multichannel Deconvolution Under a Bilinear System Model

Abstract: We consider the multichannel blind deconvolution problem where we observe the output of multiple channels that are all excited with the same unknown input. From these observations, we wish to estimate the impulse responses of each of the channels. We show that this problem is well-posed if the channels follow a bilinear model where the ensemble of channel responses is modeled as lying in a low-dimensional subspace but with each channel modulated by an independent gain. Under this model, we show how the channel estimates can be found by minimizing a quadratic function over a non-convex set. We analyze two methods for solving this non-convex program, and provide performance guarantees for each. The first is a method of alternating eigenvectors that breaks the program down into a series of eigenvalue problems. The second is a truncated power iteration, which can roughly be interpreted as a method for finding the largest eigenvector of a symmetric matrix with the additional constraint that it adheres to our bilinear model. As with most non-convex optimization algorithms, the performance of both of these algorithms is highly dependent on having a good starting point. We show how such a starting point can be constructed from the channel measurements. Our performance guarantees are non-asymptotic, and provide a sufficient condition on the number of samples observed per channel in order to guarantee channel estimates of certain accuracy. Our analysis uses a model with a “generic” subspace that is drawn at random, and we show the performance bounds hold with high probability. Mathematically, the key estimates are derived by quantifying how well the eigenvectors of certain random matrices approximate the eigenvectors of their mean. We also present a series of numerical results demonstrating that the empirical performance is consistent with the presented theory.

Feedback Stabilization Using Koopman Operator

Abstract: In this paper, we provide a systematic approach for the design of stabilizing feedback controllers for nonlinear control systems using the Koopman operator framework. The Koopman operator approach provides a linear representation for a nonlinear dynamical system and a bilinear representation for a nonlinear control system. The problem of feedback stabilization of a nonlinear control system is then transformed to the stabilization of a bilinear control system. We propose a control Lyapunov function (CLF)-based approach for the design of stabilizing feedback controllers for the bilinear system. The search for finding a CLF for the bilinear control system is formulated as a convex optimization problem. This leads to a schematic procedure for designing CLF-based stabilizing feedback controllers for the bilinear system and hence the original nonlinear system. Another advantage of the proposed controller design approach outlined in this paper is that it does not require explicit knowledge of system dynamics. In particular, the bilinear representation of a nonlinear control system in the Koopman eigenfunction space can be obtained from time-series data. Simulation results are presented to verify the main results on the design of stabilizing feedback controllers and the data-driven aspect of the proposed approach.

Conformal image warping

Abstract: Numerical and computer-graphic methods for conformal image mapping between two simply connected regions are described. The immediate motivation for this application is that the visual field is represented in the brain by mappings which are, at least approximately, conformal. Thus, to simulate the imaging properties of the human visual system (and perhaps other sensory systems), conformal image mapping is a necessary technique. For generating the conformal map, a method for analytic mappings and an implementation of the Symm algorithm for numerical conformal mapping are shown. The first method evaluates the inverse mapping function at each pixel of the range, with antialiasing by multiresolution texture prefiltering and bilinear interpolation. The second method is based on constructing a piecewise affine approximation of the mapping in the form of a joint triangulation, or triangulation map, in which only the nodes of the triangulation are conformally mapped. The texture is then mapped by a local affine transformation on each pixel of the range triangulation with the same antialiasing used in the first method. The algorithms are illustrated with examples of conformal mappings constructed analytically from elementary mappings, such as the linear fractional map, the complex algorithm, etc. Applications of numerically generated maps between highly irregular regions and an example of the visual field mapping that motivates this work are also shown. >

A study of lump-type and interaction solutions to a (3+1)-dimensional Jimbo–Miwa-like equation

Abstract: Using generalized bilinear equations, we construct a (3+1)-dimensional Jimbo–Miwa-like equation which possesses the same bilinear type as the standard (3+1)-dimensional Jimbo–Miwa equation. Classes of lump-type solutions and interaction solutions between lump-type and kink solutions to the resulting Jimbo–Miwa-like equation are generated through Maple symbolic computation. We discuss the conditions guaranteeing analyticity and positiveness of the solutions. By taking special choices of the involved parameters, 3D plots are presented to illustrate the dynamical features of the solutions.

Alternating direction method of multipliers for a class of nonconvex bilinear optimization: convergence analysis and applications

Abstract: In this paper, we study a class of nonconvex nonsmooth optimization problems with bilinear constraints, which have wide applications in machine learning and signal processing. We propose an algorithm based on the alternating direction method of multipliers, and rigorously analyze its convergence properties (to the set of stationary solutions). To test the performance of the proposed method, we specialize it to the nonnegative matrix factorization problem and certain sparse principal component analysis problem. Extensive experiments on real and synthetic data sets have demonstrated the effectiveness and broad applicability of the proposed methods.

Bilinear interpolation method for quantum images based on quantum Fourier transform

Abstract: Image scaling is the basic operation that is widely used in classic image processing, including nearest-neighbor interpolation, bilinear interpolation, and bicubic interpolation. In quantum image p...