Showing papers on "Filter (video) published in 2020"

Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration

Abstract: Filter pruning has been widely applied to neural network compression and acceleration. Existing methods usually utilize pre-defined pruning criteria, such as Lp-norm, to prune unimportant filters. There are two major limitations to these methods. First, existing methods fail to consider the variety of filter distribution across layers. To extract features of the coarse level to the fine level, the filters of different layers have various distributions. Therefore, it is not suitable to utilize the same pruning criteria to different functional layers. Second, prevailing layer-by-layer pruning methods process each layer independently and sequentially, failing to consider that all the layers in the network collaboratively make the final prediction. In this paper, we propose Learning Filter Pruning Criteria (LFPC) to solve the above problems. Specifically, we develop a differentiable pruning criteria sampler. This sampler is learnable and optimized by the validation loss of the pruned network obtained from the sampled criteria. In this way, we could adaptively select the appropriate pruning criteria for different functional layers. Besides, when evaluating the sampled criteria, LFPC comprehensively consider the contribution of all the layers at the same time. Experiments validate our approach on three image classification benchmarks. Notably, on ILSVRC-2012, our LFPC reduces more than 60% FLOPs on ResNet-50 with only 0.83% top-5 accuracy loss.

Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression

Abstract: In this paper, we analyze two popular network compression techniques, i.e. filter pruning and low-rank decomposition, in a unified sense. By simply changing the way the sparsity regularization is enforced, filter pruning and low-rank decomposition can be derived accordingly. This provides another flexible choice for network compression because the techniques complement each other. For example, in popular network architectures with shortcut connections (e.g. ResNet), filter pruning cannot deal with the last convolutional layer in a ResBlock while the low-rank decomposition methods can. In addition, we propose to compress the whole network jointly instead of in a layer-wise manner. Our approach proves its potential as it compares favorably to the state-of-the-art on several benchmarks. Code is available at https://github.com/ofsoundof/group_sparsity.

Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning

Abstract: The success of convolutional neural networks (CNNs) in computer vision applications has been accompanied by a significant increase of computation and memory costs, which prohibits their usage on resource-limited environments, such as mobile systems or embedded devices. To this end, the research of CNN compression has recently become emerging. In this paper, we propose a novel filter pruning scheme, termed structured sparsity regularization (SSR), to simultaneously speed up the computation and reduce the memory overhead of CNNs, which can be well supported by various off-the-shelf deep learning libraries. Concretely, the proposed scheme incorporates two different regularizers of structured sparsity into the original objective function of filter pruning, which fully coordinates the global output and local pruning operations to adaptively prune filters. We further propose an alternative updating with Lagrange multipliers (AULM) scheme to efficiently solve its optimization. AULM follows the principle of alternating direction method of multipliers (ADMM) and alternates between promoting the structured sparsity of CNNs and optimizing the recognition loss, which leads to a very efficient solver ( $2.5\times $ to the most recent work that directly solves the group sparsity-based regularization). Moreover, by imposing the structured sparsity, the online inference is extremely memory-light since the number of filters and the output feature maps are simultaneously reduced. The proposed scheme has been deployed to a variety of state-of-the-art CNN structures, including LeNet, AlexNet, VGGNet, ResNet, and GoogLeNet, over different data sets. Quantitative results demonstrate that the proposed scheme achieves superior performance over the state-of-the-art methods. We further demonstrate the proposed compression scheme for the task of transfer learning, including domain adaptation and object detection, which also show exciting performance gains over the state-of-the-art filter pruning methods.

Research on Remaining Useful Life Prediction of Rolling Element Bearings Based on Time-Varying Kalman Filter

Abstract: Rolling bearings are the key components of rotating machinery. Thus, the prediction of remaining useful life (RUL) is vital in condition-based maintenance (CBM). This paper proposes a new method for RUL prediction of bearings based on time-varying Kalman filter, which can automatically match different degradation stages of bearings and effectively realize the prediction of RUL. The evolution of monitoring data in normal and slow degradation stages is a linear trend, and the evolution in accelerated degradation stage is nonlinear. Therefore, Kalman filter models based on linear and quadratic functions are established. Meanwhile, a sliding window relative error is constructed to adaptively judge the bearing degradation stages. It can automatically switch filter models to process monitoring data at different stages. Then, the RUL can be predicted effectively. Two groups of bearing run-to-failure data sets are utilized to demonstrate the feasibility and validity of the proposed method.

Provable Filter Pruning for Efficient Neural Networks

Abstract: We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network. Our algorithm uses a small batch of input data points to assign a saliency score for each filter and constructs an importance sampling distribution where filters that highly affect the output are sampled with correspondingly high probability. Unlike weight pruning approaches that lead to irregular sparsity patterns -- requiring specialized libraries or hardware to enable computational speedups -- our approach compresses the original network to a slimmer subnetwork, which enables accelerated inference with any off-the-shelf deep learning library and hardware. Existing filter pruning methods are generally data-oblivious, rely on heuristics for evaluating the parameter importance, or require manual and tedious hyper-parameter tuning. In contrast, our method is data-informed, exhibits provable guarantees on the size and performance of the pruned network, and is widely applicable to varying network architectures and data sets. Our analytical bounds bridge the notions of compressibility and importance of network structures, which gives rise to a fully-automated procedure for identifying and preserving the filters in layers that are essential to the network's performance. Our experimental results across varying pruning scenarios show that our algorithm consistently generates sparser and more efficient models than those generated by existing filter pruning approaches.

5G mmWave Cooperative Positioning and Mapping Using Multi-Model PHD Filter and Map Fusion

Abstract: 5G millimeter wave (mmWave) signals can enable accurate positioning in vehicular networks when the base station and vehicles are equipped with large antenna arrays. However, radio-based positioning suffers from multipath signals generated by different types of objects in the physical environment. Multipath can be turned into a benefit, by building up a radio map (comprising the number of objects, object type, and object state) and using this map to exploit all available signal paths for positioning. We propose a new method for cooperative vehicle positioning and mapping of the radio environment, comprising a multiple-model probability hypothesis density filter and a map fusion routine, which is able to consider different types of objects and different fields of views. Simulation results demonstrate the performance of the proposed method.

BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Abstract: Video interpolation increases the temporal resolution of a video sequence by synthesizing intermediate frames between two consecutive frames. We propose a novel deep-learning-based video interpolation algorithm based on bilateral motion estimation. First, we develop the bilateral motion network with the bilateral cost volume to estimate bilateral motions accurately. Then, we approximate bi-directional motions to predict a different kind of bilateral motions. We then warp the two input frames using the estimated bilateral motions. Next, we develop the dynamic filter generation network to yield dynamic blending filters. Finally, we combine the warped frames using the dynamic blending filters to generate intermediate frames. Experimental results show that the proposed algorithm outperforms the state-of-the-art video interpolation algorithms on several benchmark datasets.

Dual-Band Half Mode Substrate Integrated Waveguide Filter With Independently Tunable Bands

Abstract: In this brief, a dual-mode dual-band filter based on half-mode substrate integrated waveguide (HMSIW) is presented. The proposed filter has the capability to tune the two pass-bands independently. The proposed filter consists of two HMSIW resonators coupled through a pair of ${E}$ -shaped coupling slots. Two resonating modes ( TE 101 and TE 102) are excited in each resonator. A varactor diode is used in the proposed filter for tuning purpose. The varactor diode is placed in the structure in such a way that both bands can be tuned independently. The lower passband can be tuned from 3.26 to 3.47 GHz with insertion loss of 0.2–2.9 dB. The higher passband can be tuned from 5.47 to 6.13 GHz with low insertion loss of 0.1–2.1 dB. The presented results show the flexibility of the filter to achieve desired responses and its suitability for integration with any tunable planar structure. A good agreement between the simulated and measured results is observed.

A Study of Virtual Resistor-Based Active Damping Alternatives for LCL Resonance in Grid-Connected Voltage Source Inverters

Abstract: LCL resonance complicates the design of a current control loop and can even threaten its stability. Extensive approaches have been proposed to deal with this resonance, among which active damping (AD) schemes based on the feedback of a single filter variable have been shown to be effective and cost-efficient. This paper presents a study of such AD techniques, where a generalized approach to developing an AD controller by relating a control diagram to an equivalent circuit is proposed. Based on this approach, AD controller forms with any one of four commonly used filter variables to realize virtual resistors (VRs) in six different connections to the LCL filter are derived. Comparisons are then made between these 24 AD controller alternatives by considering the implementation complexity of the AD controller, the number of measuring sensors, and the effect of the AD controller on the power stage. Consequently, grid-side current feedback AD to realize a VR in parallel with the filter capacitor is selected as a considered alternative. Next, two issues associated with the practical implementation of the selected grid-side current feedback AD, caused by the second-order differential expression and the digital time delay, are discussed and solved. Finally, the selected AD method is analyzed in the discrete z -domain, and its effectiveness is experimentally verified.

SCOP: Scientific Control for Reliable Neural Network Pruning

Abstract: This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the importance of filters to the network and then execute filter pruning accordingly. To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output. Acting as a control group, knockoff feature is generated to mimic the feature map produced by the network filter, but they are conditionally independent of the example label given the real feature map. We theoretically suggest that the knockoff condition can be approximately preserved given the information propagation of network layers. Besides the real feature map on an intermediate layer, the corresponding knockoff feature is brought in as another auxiliary input signal for the subsequent layers. Redundant filters can be discovered in the adversarial process of different features. Through experiments, we demonstrate the superiority of the proposed algorithm over state-of-the-art methods. For example, our method can reduce 57.8% parameters and 60.2% FLOPs of ResNet-101 with only 0.01% top-1 accuracy loss on ImageNet. The code is available at this https URL

Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attention

Abstract: This paper studies the problem of temporal moment localization in a long untrimmed video using natural language as the query. Given an untrimmed video and a query sentence, the goal is to determine the start and end of the relevant visual moment in the video that corresponds to the query sentence. While most previous works have tackled this by a propose-and-rank approach, we introduce a more efficient, end-to-end trainable, and proposal-free approach that is built upon three key components: a dynamic filter which adaptively transfers language information to visual domain attention map, a new loss function to guide the model to attend the most relevant part of the video, and soft labels to cope with annotation uncertainties. Our method is evaluated on three standard benchmark datasets, Charades-STA, TACoS and ActivityNet-Captions. Experimental results show our method outperforms state-of-the-art methods on these datasets, confirming the effectiveness of the method. We believe the proposed dynamic filter-based guided attention mechanism will prove valuable for other vision and language tasks as well.

Hierarchical Controller-Estimator for Coordination of Networked Euler–Lagrange Systems

Abstract: This paper proposes several hierarchical controller-estimator algorithms (HCEAs) to solve the coordination problem of networked Euler–Lagrange systems (NELSs) with sampled-data interactions and switching interaction topologies, where the cases with both discontinuous and continuous signals are successfully addressed in a unified framework. The HCEAs comprise two main layers (i.e., a control layer and an estimator layer) and one optional layer (i.e., a filter layer), in which the coordination problem is tackled in the main layers and the transient response can be optionally smoothed in the filter layer. For stabilizing the corresponding cascade closed-loop systems, several sufficient conditions on the upper bound of the aperiodic sampling intervals and the lower bound of the control parameters are established. In addition, the HCEAs are extended to address the task-space coordination problem of networked heterogeneous robotic systems, which shows the versatility of the HCEAs. Finally, comparison studies and simulation results are provided to demonstrate the effectiveness, significance, and advantages of the presented algorithms.

TLIO: Tight Learned Inertial Odometry

Abstract: In this letter we propose a tightly-coupled Extended Kalman Filter framework for IMU-only state estimation. Strap-down IMU measurements provide relative state estimates based on IMU kinematic motion model. However the integration of measurements is sensitive to sensor bias and noise, causing significant drift within seconds. Recent research by Yan et al. (RoNIN) and Chen et al. (IONet) showed the capability of using trained neural networks to obtain accurate 2D displacement estimates from segments of IMU data and obtained good position estimates from concatenating them. This letter demonstrates a network that regresses 3D displacement estimates and its uncertainty, giving us the ability to tightly fuse the relative state measurement into a stochastic cloning EKF to solve for pose, velocity and sensor biases. We show that our network, trained with pedestrian data from a headset, can produce statistically consistent measurement and uncertainty to be used as the update step in the filter, and the tightly-coupled system outperforms velocity integration approaches in position estimates, and AHRS attitude filter in orientation estimates. Video materials and code can be found on our project page:http://cathias.github.io/TLIO/.

Robust fault detection filter design for a class of discrete-time conic-type non-linear Markov jump systems with jump fault signals

Abstract: This study investigates the robust fault detection filter design problem for a class of discrete-time conic-type non-linear Markov jump systems with jump fault signals. The conic-type non-linearities satisfy a restrictive condition that lies in an n-dimensional hyper-sphere with an uncertain centre. A crucial idea is to formulate the robust fault detection filter design problem of non-linear Markov jump systems as H ∞ filtering problem. The authors aim to design a fault detection filter such that the augmented Markov jump systems with conic-type non-linearities are stochastically stable and satisfy the given H ∞ performance against the external disturbances. By means of the appropriate mode-dependent Lyapunov functional method, sufficient conditions for the existence of the designed fault detection filter are presented in terms of linear matrix inequalities. Finally, a practical circuit model example is employed to demonstrate the availability of the main results.

BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Abstract: Video interpolation increases the temporal resolution of a video sequence by synthesizing intermediate frames between two consecutive frames. We propose a novel deep-learning-based video interpolation algorithm based on bilateral motion estimation. First, we develop the bilateral motion network with the bilateral cost volume to estimate bilateral motions accurately. Then, we approximate bi-directional motions to predict a different kind of bilateral motions. We then warp the two input frames using the estimated bilateral motions. Next, we develop the dynamic filter generation network to yield dynamic blending filters. Finally, we combine the warped frames using the dynamic blending filters to generate intermediate frames. Experimental results show that the proposed algorithm outperforms the state-of-the-art video interpolation algorithms on several benchmark datasets. The source codes and pre-trained models are available at https://github.com/JunHeum/BMBC.

A novel hybrid wrapper–filter approach based on genetic algorithm, particle swarm optimization for feature subset selection

Abstract: The classification is one of the main technique of machine learning science. In many problems, the data sets have a high dimensionality that the existence of all features is not important to the purpose of the problem, and this will decrease the accuracy and performance of the algorithm. In this situation, the feature selection will play a significant role, and by eliminating unrelated features, the efficiency of the algorithm will be increased. A hybrid filter-wrapper method is proposed in the present study for feature subset selection established with integration of evolutionary based genetic algorithms (GA) and particle swarm optimization (PSO). The presented method mainly aims to reduce the complication of calculation and the search time expended to achieve an optimum solution to the high dimensional datasets feature selection problem. The proposed method, named smart HGP-FS, utilizes artificial neural network (ANN) in the fitness function. The filter and wrapper methods are integrated in order to take the benefit of filter technique acceleration and the wrapper technique vigor for selection of dataset efficacious characteristics. Some dataset characteristics are eliminated through the filter phase, which in turn reduces complex computations and search time in the wrapper phase. Comparisons have been made for the effectiveness of the proposed hybrid algorithm with the usability of three hybrid filter-wrapper methods, two pure wrapper algorithms, two pure filter procedures, and two traditional wrapper feature selection techniques. The findings obtained over real-world datasets show the efficiency of the presented algorithm. The outcomes of algorithm examination on five datasets reveal that the developed method is able to obtain a more accurate classification and to remove unsuitable and unessential characteristics more effectively relative to the other approaches.

Probabilistic Kalman filter for moving object tracking

Abstract: Kalman filter has been successfully applied to tracking moving objects in real-time situations. However, the filter cannot take into account the existing prior knowledge to improve its predictions. In the moving object tracking, the trajectories of multiple targets in the same environment could be available, which can be viewed as the prior knowledge for the tracking procedure. This paper presents the probabilistic Kalman filter (PKF) that is able to take into account the stored trajectories to improve tracking estimation. The PKF has an extra stage after two steps of the Kalman filter to refine the estimated position of the targets. The refinement is obtained by applying the Viterbi algorithm to a probabilistic graph, that is constructed based on the observed trajectories. The graph is built in the offline situation and could be adapted in the online tracking. The proposed tracker has higher accuracy compared to the standard Kalman filter and could handle widespread problems such as occlusion. Another significant achievement of the proposed tracker is to track an object with anomalous behaviors by drawing an inference based on the constructed probabilistic graph. The PKF was applied to several manually-built videos and several other video-bases containing severe occlusions, which demonstrates a significant performance in comparison with other state-of-the-art trackers.

Event-Triggered Neural Control of Nonlinear Systems With Rate-Dependent Hysteresis Input Based on a New Filter

Abstract: In controlling nonlinear uncertain systems, compensating for rate-dependent hysteresis nonlinearity is an important, yet challenging problem in adaptive control. In fact, it can be illustrated through simulation examples that instability is observed when existing control methods in canceling hysteresis nonlinearities are applied to the networked control systems (NCSs). One control difficulty that obstructs these methods is the design conflict between the quantized networked control signal and the rate-dependent hysteresis characteristics. So far, there is still no solution to this problem. In this paper, we consider the event-triggered control for NCSs subject to actuator rate-dependent hysteresis and failures. A new second-order filter is proposed to overcome the design conflict and used for control design. With the incorporation of the filter, a novel adaptive control strategy is developed from a neural network technique and a modified backstepping recursive design. It is proved that all the control signals are semiglobally uniformly ultimately bounded and the tracking error will converge to a tunable residual around zero.

A new high resolution filter for source edge detection of potential field data

Abstract: Determining the source edges is a frequently requested task in the analysis of potential fields. However, the edge detection methods have some drawbacks or shortcomings, for example, blurred respon...

Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images

Abstract: Histology images are inherently symmetric under rotation, where each orientation is equally as likely to appear. However, this rotational symmetry is not widely utilised as prior knowledge in modern Convolutional Neural Networks (CNNs), resulting in data hungry models that learn independent features at each orientation. Allowing CNNs to be rotation-equivariant removes the necessity to learn this set of transformations from the data and instead frees up model capacity, allowing more discriminative features to be learned. This reduction in the number of required parameters also reduces the risk of overfitting. In this paper, we propose Dense Steerable Filter CNNs (DSF-CNNs) that use group convolutions with multiple rotated copies of each filter in a densely connected framework. Each filter is defined as a linear combination of steerable basis filters, enabling exact rotation and decreasing the number of trainable parameters compared to standard filters. We also provide the first in-depth comparison of different rotation-equivariant CNNs for histology image analysis and demonstrate the advantage of encoding rotational symmetry into modern architectures. We show that DSF-CNNs achieve state-of-the-art performance, with significantly fewer parameters, when applied to three different tasks in the area of computational pathology: breast tumour classification, colon gland segmentation and multi-tissue nuclear segmentation.

Individual Privacy Accounting via a Renyi Filter

Abstract: We consider a sequential setting in which a single dataset of individuals is used to perform adaptively-chosen analyses, while ensuring that the differential privacy loss of each participant does not exceed a pre-specified privacy budget. The standard approach to this problem relies on bounding a worst-case estimate of the privacy loss over all individuals and all possible values of their data, for every single analysis. Yet, in many scenarios this approach is overly conservative, especially for "typical" data points which incur little privacy loss by participation in most of the analyses. In this work, we give a method for tighter privacy loss accounting based on the value of a personalized privacy loss estimate for each individual in each analysis. To implement the accounting method we design a filter for Renyi differential privacy. A filter is a tool that ensures that the privacy parameter of a composed sequence of algorithms with adaptively-chosen privacy parameters does not exceed a pre-specified budget. Our filter is simpler and tighter than the known filter for $(\epsilon,\delta)$-differential privacy by Rogers et al. We apply our results to the analysis of noisy gradient descent and show that personalized accounting can be practical, easy to implement, and can only make the privacy-utility tradeoff tighter.

Modeling and Simulation of a PI Controlled Shunt Active Power Filter for Power Quality Enhancement Based on P-Q Theory

Abstract: The design of reliable power filters that mitigate current and voltage harmonics to meet the power quality requirements of the utility grid is a major requirement of present-day power systems. In this paper, a detailed systematic approach to design a shunt active power filter (SAPF) for power quality enhancement is discussed. A proportional–integral (PI) controller is adopted to regulate the DC-link voltage. The instantaneous reactive power theory is employed for the reference current’s extraction. Hysteresis current control is used to obtain the gate pulses that control the voltage source inverter (VSI) switches. The detailed SAPF is developed and simulated for balanced nonlinear loads and unbalanced nonlinear loads using MATLAB/Simulink. The simulation results indicate that the proposed filter can minimize the harmonic distortion to a level below that deployed by the Institute of Electrical and Electronics Engineers (IEEE) standards.

FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding

Abstract: Large-scale cross-lingual language models (LM), such as mBERT, Unicoder and XLM, have achieved great success in cross-lingual representation learning. However, when applied to zero-shot cross-lingual transfer tasks, most existing methods use only single-language input for LM finetuning, without leveraging the intrinsic cross-lingual alignment between different languages that proves essential for multilingual tasks. In this paper, we propose FILTER, an enhanced fusion method that takes cross-lingual data as input for XLM finetuning. Specifically, FILTER first encodes text input in the source language and its translation in the target language independently in the shallow layers, then performs cross-language fusion to extract multilingual knowledge in the intermediate layers, and finally performs further language-specific encoding. During inference, the model makes predictions based on the text input in the target language and its translation in the source language. For simple tasks such as classification, translated text in the target language shares the same label as the source language. However, this shared label becomes less accurate or even unavailable for more complex tasks such as question answering, NER and POS tagging. To tackle this issue, we further propose an additional KL-divergence self-teaching loss for model training, based on auto-generated soft pseudo-labels for translated text in the target language. Extensive experiments demonstrate that FILTER achieves new state of the art on two challenging multilingual multi-task benchmarks, XTREME and XGLUE.