Showing papers on "Fast Fourier transform published in 2015"

Fast Algorithms for Convolutional Neural Networks

Abstract: Deep convolutional neural networks take GPU days of compute time to train on large data sets. Pedestrian detection for self driving cars requires very low latency. Image recognition for mobile phones is constrained by limited processing resources. The success of convolutional neural networks in these situations is limited by how fast we can compute them. Conventional FFT based convolution is fast for large filters, but state of the art convolutional neural networks use small, 3x3 filters. We introduce a new class of fast algorithms for convolutional neural networks using Winograd's minimal filtering algorithms. The algorithms compute minimal complexity convolution over small tiles, which makes them fast with small filters and small batch sizes. We benchmark a GPU implementation of our algorithm with the VGG network and show state of the art throughput at batch sizes from 1 to 64.

An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections

Abstract: We explore the redundancy of parameters in deep neural networks by replacing the conventional linear projection in fully-connected layers with the circulant projection. The circulant structure substantially reduces memory footprint and enables the use of the Fast Fourier Transform to speed up the computation. Considering a fully-connected neural network layer with d input nodes, and d output nodes, this method improves the time complexity from O(d2) to O(dlogd) and space complexity from O(d2) to O(d). The space savings are particularly important for modern deep convolutional neural network architectures, where fully-connected layers typically contain more than 90% of the network parameters. We further show that the gradient computation and optimization of the circulant projections can be performed very efficiently. Our experiments on three standard datasets show that the proposed approach achieves this significant gain in storage and efficiency with minimal increase in error rate compared to neural networks with unstructured projections.

Fast Convolutional Nets With fbfft: A GPU Performance Evaluation

Abstract: We examine the performance profile of Convolutional Neural Network training on the current generation of NVIDIA Graphics Processing Units. We introduce two new Fast Fourier Transform convolution implementations: one based on NVIDIA's cuFFT library, and another based on a Facebook authored FFT implementation, fbfft, that provides significant speedups over cuFFT (over 1.5x) for whole CNNs. Both of these convolution implementations are available in open source, and are faster than NVIDIA's cuDNN implementation for many common convolutional layers (up to 23.5x for some synthetic kernel configurations). We discuss different performance regimes of convolutions, comparing areas where straightforward time domain convolutions outperform Fourier frequency domain convolutions. Details on algorithmic applications of NVIDIA GPU hardware specifics in the implementation of fbfft are also provided.

Optimization Methods for Designing Sequences With Low Autocorrelation Sidelobes

Abstract: Unimodular sequences with low autocorrelation are desired in many applications, especially in radar systems and code-division multiple access (CDMA) communication systems. In this paper, we propose a new algorithm to design unimodular sequences with low autocorrelation via directly minimizing the integrated sidelobe level (ISL) of the autocorrelation. The algorithm is derived based on the general framework of majorization-minimization (MM) algorithms and thus shares the monotonic property of such methods, and two acceleration schemes have been considered to accelerate the overall convergence. In addition, the proposed algorithm can be implemented via fast Fourier transform (FFT) operations and thus is computationally efficient. Furthermore, after some modifications the algorithm can be adapted to incorporate spectral constraints, which makes the design more flexible. Numerical experiments show that the proposed algorithms outperform existing ones in terms of both the merit factors of designed sequences and the computational complexity.

Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method.

Abstract: Fast calculation and correct depth cue are crucial issues in the calculation of computer-generated hologram (CGH) for high quality three-dimensional (3-D) display. An angular-spectrum based algorithm for layer-oriented CGH is proposed. Angular spectra from each layer are synthesized as a layer-corresponded sub-hologram based on the fast Fourier transform without paraxial approximation. The proposed method can avoid the huge computational cost of the point-oriented method and yield accurate predictions of the whole diffracted field compared with other layer-oriented methods. CGHs of versatile formats of 3-D digital scenes, including computed tomography and 3-D digital models, are demonstrated with precise depth performance and advanced image quality.

Fast estimators for redshift-space clustering

Abstract: Redshift-space distortions in galaxy surveys happen along the radial direction, breaking statistical translation invariance. We construct estimators for radial distortions that, using only fast fourier transforms (FFTs) of the overdensity field multipoles for a given survey geometry, compute the power spectrum monopole, quadrupole and hexadecapole, and generalize such estimators to the bispectrum. Using realistic mock catalogs we compare the signal to noise of two estimators for the power spectrum hexadecapole that require different number of FFTs and measure the bispectrum monopole, quadrupole and hexadecapole. The resulting algorithm is very efficient, e.g. the BOSS survey requires about 3 min for $\ensuremath{\ell}=0,2,4$ power spectra for scales up to $k=0.3\text{ }h\text{ }{\mathrm{Mpc}}^{\ensuremath{-}1}$ and about 15 additional min for $\ensuremath{\ell}=0,2,4$ bispectra for all scales and triangle shapes up to $k=0.2\text{ }h\text{ }{\mathrm{Mpc}}^{\ensuremath{-}1}$ on a single core. The speed of these estimators is essential as it makes possible for one to compute covariance matrices from large number of realizations of mock catalogs with realistic survey characteristics, and paves the way for improved constraints of gravity on cosmological scales, inflation and galaxy bias.

Efficient convolutional sparse coding

Abstract: Computationally efficient algorithms may be applied for fast dictionary learning solving the convolutional sparse coding problem in the Fourier domain. More specifically, efficient convolutional sparse coding may be derived within an alternating direction method of multipliers (ADMM) framework that utilizes fast Fourier transforms (FFT) to solve the main linear system in the frequency domain. Such algorithms may enable a significant reduction in computational cost over conventional approaches by implementing a linear solver for the most critical and computationally expensive component of the conventional iterative algorithm. The theoretical computational cost of the algorithm may be reduced from O(M 3 N) to O(MN log N), where N is the dimensionality of the data and M is the number of elements in the dictionary. This significant improvement in efficiency may greatly increase the range of problems that can practically be addressed via convolutional sparse representations.

Exploring the Feasibility of Fully Homomorphic Encryption

Abstract: In 2010, Gentry and Halevi presented the first FHE implementation. FHE allows the evaluation of arbitrary functions directly on encrypted data on untrusted servers. However, even for the small setting with 2048 dimensions, the authors reported a performance of 1.8 s for a single bit encryption and 32 s for recryption on a high-end server. Much of the latency is due to computationally intensive multi-million-bit modular multiplications. In this paper, we introduce two optimizations coupled with a novel precomputation technique. In the first optimization called partial FFT, we adopt Strassen’s FFT-based multiplication algorithm along with Barret reduction to speedup modular multiplications. For the encrypt primitive, we employ a window-based evaluation technique along with a modest degree of precomputation. In the full FFT optimization, we delay modular reductions and change the window algorithm, which allows us to carry out the bulk of computations in the frequency domain. We manage to eliminate all FFT conversion except the final inverse transformation drastically reducing the computation latency for all FHE primitives. We implemented the GH FHE scheme on two GPUs to further speedup the operations. Our experimental results with small parameter setting show speedups of 174, 7.6, and 13.5 times for encryption, decryption, and recryption, respectively, when compared to the Gentry–Halevi implementation. The speedup is enhanced in the medium setting. However, in the large setting, memory becomes the bottleneck and the speedup is somewhat diminished.

Fast Numerical Nonlinear Fourier Transforms

Abstract: The nonlinear Fourier transform, which is also known as the forward scattering transform, decomposes a periodic signal into nonlinearly interacting waves. In contrast to the common Fourier transform, these waves no longer have to be sinusoidal. Physically relevant waveforms are often available for the analysis instead. The details of the transform depend on the waveforms underlying the analysis, which in turn are specified through the implicit assumption that the signal is governed by a certain evolution equation. For example, water waves generated by the Korteweg–de Vries equation can be expressed in terms of cnoidal waves. Light waves in optical fiber governed by the nonlinear Schrodinger equation (NSE) are another example. Nonlinear analogs of classic problems such as spectral analysis and filtering arise in many applications, with information transmission in optical fiber, as proposed by Yousefi and Kschischang, being a very recent one. The nonlinear Fourier transform is eminently suited to address them—at least from a theoretical point of view. Although numerical algorithms are available for computing the transform, a fast nonlinear Fourier transform that is similarly effective as the fast Fourier transform is for computing the common Fourier transform has not been available so far. The goal of this paper is to address this problem. Two fast numerical methods for computing the nonlinear Fourier transform with respect to the NSE are presented. The first method achieves a runtime of $O(D^{2})$ floating point operations, where $D$ is the number of sample points. The second method applies only to the case where the NSE is defocusing, but it achieves an $O(D\log ^{2}D)$ runtime. Extensions of the results to other evolution equations are discussed as well.

Harmonic Estimation Using Symmetrical Interpolation FFT Based on Triangular Self-Convolution Window

Abstract: Harmonic estimation is an important topic in power system signal processing. Windowed interpolation fast Fourier transformation (WIFFT) is an efficient algorithm for power system harmonic estimation, which can eliminate the errors caused by spectral leakage and picket fence effect. However, the fitting polynomial in the interpolation procedure contains both even and odd terms, and this increases the computational burden. This paper proposes a simple symmetrical interpolation FFT algorithm, where the even terms are removed from the fitting polynomial based on the triangular self-convolution windows (TSCW). The polynomials for frequency and amplitude computations are provided. Considerable leakage errors and harmonic interferences can be suppressed by the TSCW. Accurate estimations of harmonic parameters can be obtained via the fitting polynomial and the TSCW, both with adjustable order to fulfill different accuracy and speed requirements of practical power harmonic measurement. Simulation results and measurements have validated the proposed method.

Fast Explicit Integration Factor Methods for Semilinear Parabolic Equations

Abstract: In this paper, an explicit numerical method and its fast implementation are proposed and discussed for the solution of a wide class of semilinear parabolic equations including the Allen---Cahn equation as a special case. The method combines decompositions of compact spatial difference operators on a regular mesh with stable and accurate exponential time integrators and efficient discrete FFT-based algorithms. It can deal with stiff nonlinearity and both homogeneous and inhomogeneous boundary conditions of different types based on multistep approximations and analytic evaluations of time integrals. Numerical experiments demonstrate effectiveness of the new method for both linear and nonlinear model problems.

High-Speed Polynomial Multiplication Architecture for Ring-LWE and SHE Cryptosystems

Abstract: Polynomial multiplication is the basic and most computationally intensive operation in ring-learning with errors (ring-LWE) encryption and "somewhat" homomorphic encryption (SHE) cryptosystems. In this paper, the fast Fourier transform (FFT) with a linearithmic complexity of $O(n\log n)$ , is exploited in the design of a high-speed polynomial multiplier. A constant geometry FFT datapath is used in the computation to simplify the control of the architecture. The contribution of this work is three-fold. First, parameter sets which support both an efficient modular reduction design and the security requirements for ring-LWE encryption and SHE are provided. Second, a versatile pipelined architecture accompanied with an improved dataflow are proposed to obtain a high-speed polynomial multiplier. Third, the proposed architecture supports polynomial multiplications for different lengths $n$ and moduli $p$ . The experimental results on a Spartan-6 FPGA show that the proposed design results in a speedup of 3.5 times on average when compared with the state of the art. It performs a polynomial multiplication in the ring-LWE scheme $(n=256,p=1049089)$ and the SHE scheme $(n=1024,p=536903681)$ in only 6.3 $\mu{\rm s}$ and 33.1 $\mu{\rm s}$ , respectively.

An exploration of parameter redundancy in deep networks with circulant projections

Abstract: We explore the redundancy of parameters in deep neural networks by replacing the conventional linear projection in fully-connected layers with the circulant projection. The circulant structure substantially reduces memory footprint and enables the use of the Fast Fourier Transform to speed up the computation. Considering a fully-connected neural network layer with d input nodes, and d output nodes, this method improves the time complexity from O(d^2) to O(dlogd) and space complexity from O(d^2) to O(d). The space savings are particularly important for modern deep convolutional neural network architectures, where fully-connected layers typically contain more than 90% of the network parameters. We further show that the gradient computation and optimization of the circulant projections can be performed very efficiently. Our experiments on three standard datasets show that the proposed approach achieves this significant gain in storage and efficiency with minimal increase in error rate compared to neural networks with unstructured projections.

Fast Approximations of Shift-Variant Blur

Abstract: Image deblurring is essential in high resolution imaging, e.g., astronomy, microscopy or computational photography. Shift-invariant blur is fully characterized by a single point-spread-function (PSF). Blurring is then modeled by a convolution, leading to efficient algorithms for blur simulation and removal that rely on fast Fourier transforms. However, in many different contexts, blur cannot be considered constant throughout the field-of-view, and thus necessitates to model variations of the PSF with the location. These models must achieve a trade-off between the accuracy that can be reached with their flexibility, and their computational efficiency. Several fast approximations of blur have been proposed in the literature. We give a unified presentation of these methods in the light of matrix decompositions of the blurring operator. We establish the connection between different computational tricks that can be found in the literature and the physical sense of corresponding approximations in terms of equivalent PSFs, physically-based approximations being preferable. We derive an improved approximation that preserves the same desirable low complexity as other fast algorithms while reaching a minimal approximation error. Comparison of theoretical properties and empirical performances of each blur approximation suggests that the proposed general model is preferable for approximation and inversion of a known shift-variant blur.

Differentially Coherent Multichannel Detection of Acoustic OFDM Signals

Abstract: In this paper, we propose a class of methods for compensating for the Doppler distortions of the underwater acoustic channel for differentially coherent detection of orthogonal frequency-division multiplexing (OFDM) signals. These methods are based on multiple fast Fourier transform (FFT) demodulation, and are implemented as partial (P), shaped (S), fractional (F), and Taylor (T) series expansion FFT demodulation. They replace the conventional FFT demodulation with a few FFTs and a combiner. The input to each FFT is a specific transformation of the input signal, and the combiner performs weighted summation of the FFT outputs. The four methods differ in the choice of the pre-FFT transformation (P, S, F, T), while the rest of the receiver remains identical across these methods. We design an adaptive algorithm of stochastic gradient type to learn the combiner weights for differentially coherent detection. The algorithm is cast into the multichannel framework to take advantage of spatial diversity. The receiver is also equipped with an improved synchronization technique for estimating the dominant Doppler shift and resampling the signal before demodulation. An additional technique of carrier sliding is introduced to aid in the post-FFT combining process when residual Doppler shift is nonnegligible. Synthetic data, as well as experimental data from a recent mobile acoustic communication experiment (few kilometers in shallow water, 10.5–15.5-kHz band) are used to demonstrate the performance of the proposed methods, showing significant improvement over conventional detection techniques with or without intercarrier interference equalization (5–7 dB on average over multiple hours), as well as improved bandwidth efficiency [ability to support up to 2048 quadrature phase-shift keying (QPSK) modulated carriers].

New Pattern-Recognition Method for Fault Analysis in Transmission Line With UPFC

Abstract: In this paper, a new set of time-frequency features for fault-type identification, fault-loop status supervision, and fault-zone detection modules in a compensated transmission line with a unified power-flow controller is proposed. Some features are extracted from a one-cycle data window of one side of the compensated line, including 3/16 cycle of postfault data by the fast discrete orthonormal S-Transform (FDOST). The computation burden of the FDOST as a time–frequency decomposition is the same as the fast Fourier transform. The support vector machine is employed for classification of the ranked features by the Gram–Schmidt method. The graphical representations of extracted features and the obtained numerical results under different conditions confirm the efficacy of the proposed scheme.

ISAR Imaging of Nonuniformly Rotating Target Based on a Fast Parameter Estimation Algorithm of Cubic Phase Signal

Abstract: In inverse synthetic aperture radar (ISAR) imaging of nonuniformly rotating targets, such as highly maneuvering airplanes and ships fluctuating with oceanic waves, azimuth echoes have to be modeled as cubic phase signals (CPSs) after the range migration compensation and the translational-induced phase error correction. For the CPS model, the chirp rate and the quadratic chirp rate, which deteriorate the azimuth focusing quality due to the Doppler frequency shift, need to be estimated with a parameter estimation algorithm. In this paper, by employing the proposed generalized scaled Fourier transform (GSCFT) and the nonuniform fast Fourier transform (NUFFT), a fast parameter estimation algorithm is presented and utilized in the ISAR imaging of the nonuniformly rotating target. Compared to the scaled Fourier transform-based algorithm, advantages of the fast parameter estimation algorithm include the following: 1) the computational cost is lower due to the utilization of the NUFFT, and 2) the GSCFT has a wider applicability in ISAR imaging applications. The CPS model and the algorithm implementation are verified with the real radar data of a ship target. In addition, the cross-term, which plays an important role in correlation algorithms, is analyzed for the fast parameter estimation algorithm. Through simulations of the synthetic data and the real radar data, we verify the effectiveness of the fast parameter estimation algorithm and the corresponding ISAR imaging algorithm.

Fast Hankel tensor–vector product and its application to exponential data fitting

Abstract: Summary This paper is contributed to a fast algorithm for Hankel tensor–vector products. First, we explain the necessity of fast algorithms for Hankel and block Hankel tensor–vector products by sketching the algorithm for both one-dimensional and multi-dimensional exponential data fitting. For proposing the fast algorithm, we define and investigate a special class of Hankel tensors that can be diagonalized by the Fourier matrices, which is called anti-circulant tensors. Then, we obtain a fast algorithm for Hankel tensor–vector products by embedding a Hankel tensor into a larger anti-circulant tensor. The computational complexity is about O(m2nlogmn) for a square Hankel tensor of order m and dimension n, and the numerical examples also show the efficiency of this scheme. Moreover, the block version for multi-level block Hankel tensors is discussed. Copyright © 2015 John Wiley & Sons, Ltd.

Neural Network Based Simplified Clipping and Filtering Technique for PAPR Reduction of OFDM Signals

Abstract: Many iterative clipping and filtering (ICF) based techniques have been proposed that achieve similar peak-to-average power ratio (PAPR) reduction of orthogonal frequency division multiplexing (OFDM) signals as the original ICF, but with lower complexity, such as the simplified clipping and filtering (SCF) technique. However, these low complexity methods require numerous complex fast Fourier transform (FFT) operations and parameter calculations. In this letter, we introduce a novel ICF method that uses an optimized mapper based on artificial neural network and SCF techniques. Compared to the conventional ICF based methods, the proposed scheme offers desirable cubic metric (CM) and bit error rate (BER) simulation results with significantly reduced computational complexity.

FPGA-Based Voltage and Current Dual Drive System for High Frame Rate Electrical Impedance Tomography

Abstract: Electrical impedance tomography (EIT) is used to image the electrical property distribution of a tissue under test. An EIT system comprises complex hardware and software modules, which are typically designed for a specific application. Upgrading these modules is a time-consuming process, and requires rigorous testing to ensure proper functioning of new modules with the existing ones. To this end, we developed a modular and reconfigurable data acquisition (DAQ) system using National Instruments' (NI) hardware and software modules, which offer inherent compatibility over generations of hardware and software revisions. The system can be configured to use up to 32-channels. This EIT system can be used to interchangeably apply current or voltage signal, and measure the tissue response in a semi-parallel fashion. A novel signal averaging algorithm, and 512-point fast Fourier transform (FFT) computation block was implemented on the FPGA. FFT output bins were classified as signal or noise. Signal bins constitute a tissue's response to a pure or mixed tone signal. Signal bins' data can be used for traditional applications, as well as synchronous frequency-difference imaging. Noise bins were used to compute noise power on the FPGA. Noise power represents a metric of signal quality, and can be used to ensure proper tissue-electrode contact. Allocation of these computationally expensive tasks to the FPGA reduced the required bandwidth between PC, and the FPGA for high frame rate EIT. In 16-channel configuration, with a signal-averaging factor of 8, the DAQ frame rate at 100 kHz exceeded 110 frames ${\rm s} ^{-1}$ , and signal-to-noise ratio exceeded 90 dB across the spectrum. Reciprocity error was found to be $ for frequencies up to 1 MHz. Static imaging experiments were performed on a high-conductivity inclusion placed in a saline filled tank; the inclusion was clearly localized in the reconstructions obtained for both absolute current and voltage mode data.

Joint Scale-Spatial Correlation Tracking with Adaptive Rotation Estimation

Abstract: Boosted by large and standardized benchmark datasets, visual object tracking has made great progress in recent years and brought about many new trackers. Among these trackers, correlation filter based tracking schema exhibits impressive robustness and accuracy. In this work, we present a fully functional correlation filter based tracking algorithm which is able to simultaneously model target appearance changes from spatial displacements, scale variations, and rotation transformations. The proposed tracker first represents the exhaustive template searching in the joint scale and spatial space by a block-circulant matrix. Then, by transferring the target template from the Cartesian coordinate system to the Log-Polar coordinate system, the circulant structure is well preserved for the target even after whole orientation rotation. With these novel representation and transformation, object tracking is efficiently and effectively performed in the joint space with fast Fourier Transform. Experimental results on the VOT 2015 benchmark dataset demonstrate its superior performance over state-of-the-art tracking algorithms.

Efficient Option Pricing by Frame Duality with the Fast Fourier Transform

Abstract: We develop a method for efficiently inverting analytic characteristic functions using frame projection, as in the case of Heston's model and exponential Levy models. Utilizing the duality theory of Riesz bases, we derive analytical formulas for coefficients of the orthogonally projected density, which are computed numerically with exponential convergence by the fast Fourier transform (FFT). Convergence is demonstrated for geometric Asian options as well as the pricing of baskets of European options. The method is compared to state-of-the-art procedures to demonstrate its efficiency and robustness, without requiring any user-supplied “control parameters.” Even greater improvement is observed in the method's extension to arithmetic Asian option pricing, as well as to Bermudan options, barrier options, and credit default swaps, which will appear in follow-up papers that expand on the foundations developed in this work.

Audio watermarking based on Fibonacci numbers

Abstract: This paper presents a novel high-capacity audio watermarking system to embed data and extract them in a bit-exact manner by changing some of the magnitudes of the FFT spectrum. The key idea is to divide the FFT spectrum into short frames and change the magnitude of the selected FFT samples using Fibonacci numbers. Taking advantage of Fibonacci numbers, it is possible to change the frequency samples adaptively. In fact, the suggested technique guarantees and proves, mathematically, that the maximum change is less than 61% of the related FFT sample and the average error for each sample is 25%. Using the closest Fibonacci number to FFT magnitudes results in a robust and transparent technique. On top of very remarkable capacity, transparency and robustness, this scheme provides two parameters which facilitate the regulation of these properties. The experimental results show that the method has a high capacity (700 bps to 3 kbps), without significant perceptual distortion (ODG is about 1) and provides robustness against common audio signal processing such as echo, added noise, filtering, and MPEG compression (MP3). In addition to the experimental results, the fidelity of suggested system is proved mathematically.