scispace - formally typeset
Search or ask a question

Showing papers on "Affine transformation published in 2020"


Proceedings Article
30 Apr 2020
TL;DR: This work presents a unifying view and proposes an open-set method to relax current generalization assumptions, and extends the applicability of transformation-based methods to non-image data using random affine transformations.
Abstract: Anomaly detection, finding patterns that substantially deviate from those seen previously, is one of the fundamental problems of artificial intelligence. Recently, classification-based methods were shown to achieve superior results on this task. In this work, we present a unifying view and propose an open-set method to relax current generalization assumptions. Furthermore, we extend the applicability of transformation-based methods to non-image data using random affine transformations. Our method is shown to obtain state-of-the-art accuracy and is applicable to broad data types. The strong performance of our method is extensively validated on multiple datasets from different domains.

208 citations


Posted Content
TL;DR: The core idea is to use feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage, and applies a learning-to-learn approach to search for the hyper-parameters of the feature- wise transformation layers.
Abstract: Few-shot classification aims to recognize novel categories with only few labeled images in each class. Existing metric-based few-shot classification algorithms predict categories by comparing the feature embeddings of query images with those from a few labeled images (support examples) using a learned metric function. While promising performance has been demonstrated, these methods often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains. In this work, we address the problem of few-shot classification under domain shifts for metric-based methods. Our core idea is to use feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage. To capture variations of the feature distributions under different domains, we further apply a learning-to-learn approach to search for the hyper-parameters of the feature-wise transformation layers. We conduct extensive experiments and ablation studies under the domain generalization setting using five few-shot classification datasets: mini-ImageNet, CUB, Cars, Places, and Plantae. Experimental results demonstrate that the proposed feature-wise transformation layer is applicable to various metric-based models, and provides consistent improvements on the few-shot classification performance under domain shift.

181 citations


Journal ArticleDOI
TL;DR: It is shown that artificial neural networks with rectifier units as activation functions can exactly represent the piecewise affine function that results from the formulation of model predictive control (MPC) of linear time-invariant systems.
Abstract: We show that artificial neural networks with rectifier units as activation functions can exactly represent the piecewise affine function that results from the formulation of model predictive control (MPC) of linear time-invariant systems. The choice of deep neural networks is particularly interesting as they can represent exponentially many more affine regions compared to networks with only one hidden layer. We provide theoretical bounds on the minimum number of hidden layers and neurons per layer that a neural network should have to exactly represent a given MPC law. The proposed approach has a strong potential as an approximation method of predictive control laws, leading to a better approximation quality and significantly smaller memory requirements than previous approaches, as we illustrate via simulation examples. We also suggest different alternatives to correct or quantify the approximation error. Since the online evaluation of neural networks is extremely simple, the approximated controllers can be deployed on low-power embedded devices with small storage capacity, enabling the implementation of advanced decision-making strategies for complex cyber-physical systems with limited computing capabilities.

164 citations


Proceedings ArticleDOI
14 Jun 2020
TL;DR: This work proposes to use a simple pair-wise ranking loss with a novel sampling strategy to improve the quality of depth map prediction and introduces a new relative depth dataset of about 21K diverse high-resolution web stereo photos to enhance the generalization ability of the model.
Abstract: Single image depth prediction is a challenging task due to its ill-posed nature and challenges with capturing ground truth for supervision. Large-scale disparity data generated from stereo photos and 3D videos is a promising source of supervision, however, such disparity data can only approximate the inverse ground truth depth up to an affine transformation. To more effectively learn from such pseudo-depth data, we propose to use a simple pair-wise ranking loss with a novel sampling strategy. Instead of randomly sampling point pairs, we guide the sampling to better characterize structure of important regions based on the low-level edge maps and high-level object instance masks. We show that the pair-wise ranking loss, combined with our structure-guided sampling strategies, can significantly improve the quality of depth map prediction. In addition, we introduce a new relative depth dataset of about 21K diverse high-resolution web stereo photos to enhance the generalization ability of our model. In experiments, we conduct cross-dataset evaluation on six benchmark datasets and show that our method consistently improves over the baselines, leading to superior quantitative and qualitative results.

123 citations


Journal ArticleDOI
TL;DR: A fast optimisation algorithm for approximately minimising convex quadratic functions over the intersection of affine and separable constraints (i.e. the Cartesian product of possibly nonconvex real sets) that is based on a variation of the alternating direction method of multipliers (ADMM).
Abstract: In this paper, we propose a fast optimisation algorithm for approximately minimising convex quadratic functions over the intersection of affine and separable constraints (i.e. the Cartesian product...

100 citations


Posted Content
TL;DR: The results highlight the under-appreciated role of the affine parameters in BatchNorm, but - in a broader sense - they characterize the expressive power of neural networks constructed simply by shifting and rescaling random features.
Abstract: A wide variety of deep learning techniques from style transfer to multitask learning rely on training affine transformations of features. Most prominent among these is the popular feature normalization technique BatchNorm, which normalizes activations and then subsequently applies a learned affine transform. In this paper, we aim to understand the role and expressive power of affine parameters used to transform features in this way. To isolate the contribution of these parameters from that of the learned features they transform, we investigate the performance achieved when training only these parameters in BatchNorm and freezing all weights at their random initializations. Doing so leads to surprisingly high performance considering the significant limitations that this style of training imposes. For example, sufficiently deep ResNets reach 82% (CIFAR-10) and 32% (ImageNet, top-5) accuracy in this configuration, far higher than when training an equivalent number of randomly chosen parameters elsewhere in the network. BatchNorm achieves this performance in part by naturally learning to disable around a third of the random features. Not only do these results highlight the expressive power of affine parameters in deep learning, but - in a broader sense - they characterize the expressive power of neural networks constructed simply by shifting and rescaling random features.

93 citations


Journal ArticleDOI
TL;DR: In this article, the authors investigate nonlinear composite models alternating proximity and affine operators defined on different spaces and establish conditions for the averagedness of the proposed composite constructs and investigate their asymptotic properties.
Abstract: Motivated by structures that appear in deep neural networks, we investigate nonlinear composite models alternating proximity and affine operators defined on different spaces. We first show that a wide range of activation operators used in neural networks are actually proximity operators. We then establish conditions for the averagedness of the proposed composite constructs and investigate their asymptotic properties. It is shown that the limit of the resulting process solves a variational inequality which, in general, does not derive from a minimization problem. The analysis relies on tools from monotone operator theory and sheds some light on a class of neural networks structures with so far elusive asymptotic properties.

86 citations


Proceedings Article
30 Apr 2020
TL;DR: In this paper, feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage are used to solve the problem of few-shot classification under domain shifts.
Abstract: Few-shot classification aims to recognize novel categories with only few labeled images in each class. Existing metric-based few-shot classification algorithms predict categories by comparing the feature embeddings of query images with those from a few labeled images (support examples) using a learned metric function. While promising performance has been demonstrated, these methods often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains. In this work, we address the problem of few-shot classification under domain shifts for metric-based methods. Our core idea is to use feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage. To capture variations of the feature distributions under different domains, we further apply a learning-to-learn approach to search for the hyper-parameters of the feature-wise transformation layers. We conduct extensive experiments and ablation study under the domain generalization setting using five few-shot classification datasets: mini-ImageNet, CUB, Cars, Places, and Plantae. Experimental results demonstrate that the proposed feature-wise transformation layer is applicable to various metric-based models, and provides consistent improvements on the few-shot classification performance under domain shift.

80 citations


Journal ArticleDOI
TL;DR: In this paper, the spin-projection operators for a metric affine theory of gravity with terms up to second order in curvature were constructed, and they were used to analyze the most general six-parameter class of theories that are projective invariant.
Abstract: We construct the spin-projection operators for a theory containing a symmetric two-index tensor and a general three-index tensor. We then use them to analyze, at linearized level, the most general action for a metric affine theory of gravity with terms up to second order in curvature, which depends on 28 parameters. In the metric case, we recover known results. In the torsion-free case, we are able to determine the most general six-parameter class of theories that are projective invariant, contain only one massless spin 2 and no spin 3, and are free of ghosts and tachyons.

80 citations


Proceedings Article
12 Dec 2020
TL;DR: This paper considers a structured affine distribution shift in users' data that captures the device-dependent data heterogeneity in federated settings and proposes a Federated Learning framework Robust to Affine distribution shifts (FLRA) that is provably robust against affine Wasserstein shifts to the distribution of observed samples.
Abstract: Federated learning is a distributed paradigm that aims at training models using samples distributed across multiple users in a network while keeping the samples on users' devices with the aim of efficiency and protecting users privacy. In such settings, the training data is often statistically heterogeneous and manifests various distribution shifts across users, which degrades the performance of the learnt model. The primary goal of this paper is to develop a robust federated learning algorithm that achieves satisfactory performance against distribution shifts in users' samples. To achieve this goal, we first consider a structured affine distribution shift in users' data that captures the device-dependent data heterogeneity in federated settings. This perturbation model is applicable to various federated learning problems such as image classification where the images undergo device-dependent imperfections, e.g. different intensity, contrast, and brightness. To address affine distribution shifts across users, we propose a Federated Learning framework Robust to Affine distribution shifts (FLRA) that is provably robust against affine Wasserstein shifts to the distribution of observed samples. To solve the FLRA's distributed minimax problem, we propose a fast and efficient optimization method and provide convergence guarantees via a gradient Descent Ascent (GDA) method. We further prove generalization error bounds for the learnt classifier to show proper generalization from empirical distribution of samples to the true underlying distribution. We perform several numerical experiments to empirically support FLRA. We show that an affine distribution shift indeed suffices to significantly decrease the performance of the learnt classifier in a new test user, and our proposed algorithm achieves a significant gain in comparison to standard federated learning and adversarial training methods.

79 citations


Journal ArticleDOI
TL;DR: The conclusion that the MM-QUATRE algorithm is superior to other intelligent algorithms is proved by the experimental results, which appear that this method has higher localization accuracy than other similar algorithms.
Abstract: QUasi-Affine TRansformation Evolutionary algorithm (QUATRE) is a new optimization algorithm based on population for complex multiple real parameter optimization problems in real world. In this paper, a novel multi-group multi-choice communication strategy algorithm for QUasi-Affine TRansformation Evolutionary (MM-QUATRE) algorithm is proposed to solve the disadvantage that the original QUATRE is always easily to fall into local optimization in the strategy of updating bad nodes with multiple groups and multiple choices. We compared it with other intelligent algorithms, the most advanced PSO variant, parallel PSO (P-PSO) variant, native QUATRE and parallel QUATRE (P-PSO) under CEC2013 large-scale optimization test suite. Thus, the performance of MM-QUATRE was verified. The conclusion that the MM-QUATRE algorithm is superior to other intelligent algorithms is proved by the experimental results. In addition, the application results of MM-QUATRE algorithm (MM-QUATRE-RSSI) based on RSSI in WSN node localization were analyzed and studied. The results appear that this method has higher localization accuracy than other similar algorithms.

Journal ArticleDOI
TL;DR: A computational method for sampling from a given target distribution based on first-order (overdamped) Langevin dynamics which satisfies the property of affine invariance is proposed, which enables application to high-dimensional sampling problems.
Abstract: We propose a computational method (with acronym ALDI) for sampling from a given target distribution based on first-order (overdamped) Langevin dynamics which satisfies the property of affine invari...

Journal ArticleDOI
Benoit Vicedo1
TL;DR: The notion of dihedral affine Gaudin models has been introduced in this article, where a broad family of classical integrable field theories can be recast as examples of such classical dihedral FGF models through (anti-)linear automorphisms.
Abstract: We introduce the notion of a classical dihedral affine Gaudin model, associated with an untwisted affine Kac–Moody algebra |$\widetilde{\mathfrak{g}}$| equipped with an action of the dihedral group |$D_{2T}$|⁠, |$T \geq 1$| through (anti-)linear automorphisms. We show that a very broad family of classical integrable field theories can be recast as examples of such classical dihedral affine Gaudin models. Among these are the principal chiral model on an arbitrary real Lie group |$G_0$| and the |$\mathbb{Z}_T$|-graded coset |$\sigma $|-model on any coset of |$G_0$| defined in terms of an order |$T$| automorphism of its complexification. Most of the multi-parameter integrable deformations of these |$\sigma $|-models recently constructed in the literature provide further examples. The common feature shared by all these integrable field theories, which makes it possible to reformulate them as classical dihedral affine Gaudin models, is the fact that they are non-ultralocal. In particular, we also obtain affine Toda field theory in its lesser-known non-ultralocal formulation as another example of this construction. We propose that the interpretation of a given classical non-ultralocal integrable field theory as a classical dihedral affine Gaudin model provides a natural setting within which to address its quantisation. At the same time, it may also furnish a general framework for understanding the massive ordinary differential equations (ODE)/integrals of motion (IM) correspondence since the known examples of integrable field theories for which such a correspondence has been formulated can all be viewed as dihedral affine Gaudin models.

Proceedings ArticleDOI
14 Jun 2020
TL;DR: An approach to predict future video frames given a sequence of continuous video frames in the past by decoupling the background scene and moving objects and shows that this model outperforms the state-of-the-art in terms of visual quality and accuracy.
Abstract: We present an approach to predict future video frames given a sequence of continuous video frames in the past. Instead of synthesizing images directly, our approach is designed to understand the complex scene dynamics by decoupling the background scene and moving objects. The appearance of the scene components in the future is predicted by non-rigid deformation of the background and affine transformation of moving objects. The anticipated appearances are combined to create a reasonable video in the future. With this procedure, our method exhibits much less tearing or distortion artifact compared to other approaches. Experimental results on the Cityscapes and KITTI datasets show that our model outperforms the state-of-the-art in terms of visual quality and accuracy.

Posted Content
TL;DR: Spatial Warping LSTM is proposed, a new 3D shape representation that supports explicit correspondence reasoning in deep implicit representations and can not only learn a common implicit tem-plate for a collection of shapes, but also establish dense correspondences across all the shapes simultaneously with-out any supervision.
Abstract: Deep implicit functions (DIFs), as a kind of 3D shape representation, are becoming more and more popular in the 3D vision community due to their compactness and strong representation power. However, unlike polygon mesh-based templates, it remains a challenge to reason dense correspondences or other semantic relationships across shapes represented by DIFs, which limits its applications in texture transfer, shape analysis and so on. To overcome this limitation and also make DIFs more interpretable, we propose Deep Implicit Templates, a new 3D shape representation that supports explicit correspondence reasoning in deep implicit representations. Our key idea is to formulate DIFs as conditional deformations of a template implicit function. To this end, we propose Spatial Warping LSTM, which decomposes the conditional spatial transformation into multiple affine transformations and guarantees generalization capability. Moreover, the training loss is carefully designed in order to achieve high reconstruction accuracy while learning a plausible template with accurate correspondences in an unsupervised manner. Experiments show that our method can not only learn a common implicit template for a collection of shapes, but also establish dense correspondences across all the shapes simultaneously without any supervision.

Journal ArticleDOI
TL;DR: This work proposes to accomplish the approximation of partial differential equations with a data-driven approach based on the reduced basis method and machine learning with a neural network embedding a reduced basis solver as exotic activation function in the last layer.

Journal ArticleDOI
TL;DR: In this paper, the authors proposed a cohort of dominant data set selection algorithms for electricity consumption TSD with a focus on discriminating the dominant data sets that is a small data set but capable of representing the kernel information carried by TSD.
Abstract: In the explosive growth of time-series data (TSD), the scale of TSD suggests that the scale and capability of many Internet of Things (IoT)-based applications has already been exceeded. Moreover, redundancy persists in TSD due to the correlation between information acquired via different sources. In this article, we propose a cohort of dominant data set selection algorithms for electricity consumption TSD with a focus on discriminating the dominant data set that is a small data set but capable of representing the kernel information carried by TSD with an arbitrarily small error rate less than $\varepsilon $ . Furthermore, we prove that the selection problem of the minimum dominant data set is an NP-complete problem. The affine transformation model is introduced to define the linear correlation relationship between TSD objects. Our proposed framework consists of the scanning selection algorithm with $O({n^{3}})$ time complexity and the greedy selection algorithm with $O({n^{4}})$ time complexity, which are, respectively, proposed to select the dominant data set based on the linear correlation distance between TSD objects. The proposed algorithms are evaluated on the real electricity consumption data of Harbin city in China. The experimental results show that the proposed algorithms not only reduce the size of the extracted kernel data set but also ensure the TSD integrity in terms of accuracy and efficiency.

Posted Content
Abstract: Image animation consists of generating a video sequence so that an object in a source image is animated according to the motion of a driving video. Our framework addresses this problem without using any annotation or prior information about the specific object to animate. Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. To achieve this, we decouple appearance and motion information using a self-supervised formulation. To support complex motions, we use a representation consisting of a set of learned keypoints along with their local affine transformations. A generator network models occlusions arising during target motions and combines the appearance extracted from the source image and the motion derived from the driving video. Our framework scores best on diverse benchmarks and on a variety of object categories. Our source code is publicly available.

Journal ArticleDOI
TL;DR: This note develops a distributed algorithm to solve a convex optimization problem with coupled constraints, where both coupled equality and inequality constraints are considered, and the algorithm focuses on smooth problems and uses a fixed stepsize to find the exact optimal solution.
Abstract: This note develops a distributed algorithm to solve a convex optimization problem with coupled constraints. Both coupled equality and inequality constraints are considered, where functions in the equality constraints are affine and functions in the inequality constraints are convex. Different from primal-dual subgradient methods with decreasing stepsizes for nonsmooth optimizations, our algorithm focuses on smooth problems and uses a fixed stepsize to find the exact optimal solution. Convergence analysis is derived with rigorous proofs. Our result is also illustrated by simulations.

Posted Content
TL;DR: In this article, the authors show that CF-INNs can be universal approximators for invertible functions if their layers contain affine coupling and linear functions as special cases.
Abstract: Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning However, their desirable characteristics such as analytic invertibility come at the cost of restricting the functional forms This poses a question on their representation power: are CF-INNs universal approximators for invertible functions? Without a universality, there could be a well-behaved invertible transformation that the CF-INN can never approximate, hence it would render the model class unreliable We answer this question by showing a convenient criterion: a CF-INN is universal if its layers contain affine coupling and invertible linear functions as special cases As its corollary, we can affirmatively resolve a previously unsolved problem: whether normalizing flow models based on affine coupling can be universal distributional approximators In the course of proving the universality, we prove a general theorem to show the equivalence of the universality for certain diffeomorphism classes, a theoretical insight that is of interest by itself

Journal ArticleDOI
TL;DR: Scalings in which the graph Laplacian approaches a differential operator in the large graph limit are used to develop understanding of a number of algorithms for semi-supervised learning; in particular the extension, to this graph setting, of the probit algorithm, level set and kriging methods are studied.

Journal ArticleDOI
TL;DR: This paper addresses the problem of assessing the stability of linear time-invariant (LTI) systems with time-varying delay with a new stability criterion specified as a negativity condition for a quadratic function parameterized by the delay.

Journal ArticleDOI
TL;DR: This paper tackles the problem of piecewise affine memory filtering design for the discrete-time norm-bounded uncertain Takagi–Sugeno fuzzy affine systems by designing an admissible filter using past output measurements of the system, guaranteeing the asymptotic stability of the filtering error system with a given inline-formula.
Abstract: This paper tackles the problem of piecewise affine memory filtering design for the discrete-time norm-bounded uncertain Takagi–Sugeno fuzzy affine systems. The objective is to design an admissible filter using past output measurements of the system, guaranteeing the asymptotic stability of the filtering error system with a given ${\mathscr H}_{\infty }$ performance index. Based on the piecewise fuzzy Lyapunov functions and the projection lemma, a new sufficient condition for $\mathscr H_{\infty }$ filtering performance analysis is first derived, and then the filter synthesis is carried out. It is shown that the filter gains can be obtained by solving a set of linear matrix inequalities. In addition, it is also shown that the filtering performance can be improved with the increasing number of past output measurements used in the filtering design. Finally, two examples are presented to show the advantages and effectiveness of the proposed approach.

Journal ArticleDOI
TL;DR: A local compact form dynamic linearization (local-CFDL) is developed at first to transform the original nonlinear nonaffine system into an affine structure consisting of both an unknown residual nonlinear time-varying term and a linearly parametric term affine to the control input.
Abstract: Linearization is often used for control design of nonlinear systems but what degree of a linearization is sufficient for the controller design is always a question. Furthermore, most of the existing linearization methods aim to develop a completely linear model without retaining any nonlinearity and thus the unmodeled dynamics unavoidably exists due to omitted higher order terms. In this article, a local compact form dynamic linearization (local-CFDL) is developed at first to transform the original nonlinear nonaffine system into an affine structure consisting of both an unknown residual nonlinear time-varying term and a linearly parametric term affine to the control input. A discrete-time extended state observer (DESO) is introduced to estimate the unknown residual nonlinear time-varying term as a new extended state. Then, a local-CFDL-based DESO-model-free adaptive control (MFAC) is proposed where the estimation of DESO is incorporated to compensate for the disturbances and uncertainties. Furthermore, a local partial-form dynamic linearization (local-PFDL) is also presented using multi-lag inputs and partial derivatives. And, a corresponding local-PFDL-based DESO-MFAC is proposed utilizing additional control information to improve control performance. The two proposed methods are both data-driven and do not require any explicit model information. Theoretical analysis shows the robust convergence of the proposed methods in the presence of disturbances. Simulations verify the effectiveness of the proposed method and show that the local-PFDL-based DESO-MFAC outperforms the local-CFDL-based one owing to the use of additional control information.

Journal ArticleDOI
TL;DR: An event-triggered sliding mode control approach for trajectory tracking problem of nonlinear input affine system with disturbance has been proposed and shows better performance in terms of reduced control updates, ensures system stability which further guarantees optimization of resource usage and cost.
Abstract: In this paper, an event-triggered sliding mode control approach for trajectory tracking problem of nonlinear input affine system with disturbance has been proposed. A second order robotic manipulator system has been modeled into a general nonlinear input affine system. Initially, the global asymptotic stability is ensured with conventional periodic sampling approach for reference trajectory tracking. Then the proposed approach of event-triggered sliding mode control is discussed which guarantees semi-global uniform ultimate boundedness. The proposed control approach guarantees non-accumulation of control updates ensuring lower bounds on inter-event triggering instants avoiding Zeno behavior in presence of the disturbance. The system shows better performance in terms of reduced control updates, ensures system stability which further guarantees optimization of resource usage and cost. The simulation results are provided for validation of proposed methodology for tracking problem by a robotic manipulator. The number of aperiodic control updates is found to be approximately 44% and 61% in the presence of constant and time-varying disturbances respectively.

Journal ArticleDOI
TL;DR: A novel deep learning network is introduced that fuses complementary information from spatial and wavelet domains to synthesize 7T T1-weighted images from their 3T counterparts, taking into account both low-frequency tissue contrast and high-frequency anatomical details.

Journal ArticleDOI
Ba Tuan Le1
TL;DR: This study applies deep learning to spectral analysis techniques and proposes a rapid analysis method for cereals that achieves good prediction results and is superior to other typical NIR analysis methods.

Journal ArticleDOI
TL;DR: The results show that methodology adapted to design proposed key-based dynamic S-boxes entails near-optimal cryptographic properties so that proposed S- boxes are as stronger as AES S-box.
Abstract: This work reports a novel chaos-based affine transformation generation method, which is based on rotational matrices to design strong key-based S-boxes. Chaotic logistic map’s nonlinear trajectories are used to generate rotational matrices under given design conditions. Thus, the inherent logic is to generate key-based S-boxes, as strong as AES S-box, in terms of cryptographic properties using chaos in affine transformation. The randomness of chaotic sequences is tested using the National Institute of Standard and Technology (NIST) Statistical Test Suit (STS) 800–22 that validates the generated sequences for S-box design. The results show that methodology adapted to design proposed key-based dynamic S-boxes entails near-optimal cryptographic properties so that proposed S-boxes are as stronger as AES S-box.

Posted Content
TL;DR: In this paper, the appearance of the scene components in the future is predicted by non-rigid deformation of the background and affine transformation of moving objects, and the anticipated appearances are combined to create a reasonable video.
Abstract: We present an approach to predict future video frames given a sequence of continuous video frames in the past. Instead of synthesizing images directly, our approach is designed to understand the complex scene dynamics by decoupling the background scene and moving objects. The appearance of the scene components in the future is predicted by non-rigid deformation of the background and affine transformation of moving objects. The anticipated appearances are combined to create a reasonable video in the future. With this procedure, our method exhibits much less tearing or distortion artifact compared to other approaches. Experimental results on the Cityscapes and KITTI datasets show that our model outperforms the state-of-the-art in terms of visual quality and accuracy.

Journal ArticleDOI
TL;DR: In this article, the authors consider stochastic (partial) differential equations appearing as Markovian lifts of affine Volterra processes with jumps from the point of view of the generalized Feller property.
Abstract: We consider stochastic (partial) differential equations appearing as Markovian lifts of affine Volterra processes with jumps from the point of view of the generalized Feller property which was introduced in, e.g., Dorsek and Teichmann (A semigroup point of view on splitting schemes for stochastic (partial) differential equations, 2010. arXiv:1011.2651 ). In particular, we provide new existence, uniqueness and approximation results for Markovian lifts of affine rough volatility models of general jump diffusion type. We demonstrate that in this Markovian light the theory of stochastic Volterra processes becomes almost classical.