Showing papers on "Quantization (image processing) published in 2016"

Going Deeper with Embedded FPGA Platform for Convolutional Neural Network

Abstract: In recent years, convolutional neural network (CNN) based methods have achieved great success in a large number of applications and have been among the most powerful and widely used techniques in computer vision. However, CNN-based methods are com-putational-intensive and resource-consuming, and thus are hard to be integrated into embedded systems such as smart phones, smart glasses, and robots. FPGA is one of the most promising platforms for accelerating CNN, but the limited bandwidth and on-chip memory size limit the performance of FPGA accelerator for CNN.In this paper, we go deeper with the embedded FPGA platform on accelerating CNNs and propose a CNN accelerator design on embedded FPGA for Image-Net large-scale image classification. We first present an in-depth analysis of state-of-the-art CNN models and show that Convolutional layers are computational-centric and Fully-Connected layers are memory-centric.Then the dynamic-precision data quantization method and a convolver design that is efficient for all layer types in CNN are proposed to improve the bandwidth and resource utilization. Results show that only 0.4% accuracy loss is introduced by our data quantization flow for the very deep VGG16 model when 8/4-bit quantization is used. A data arrangement method is proposed to further ensure a high utilization of the external memory bandwidth. Finally, a state-of-the-art CNN, VGG16-SVD, is implemented on an embedded FPGA platform as a case study. VGG16-SVD is the largest and most accurate network that has been implemented on FPGA end-to-end so far. The system on Xilinx Zynq ZC706 board achieves a frame rate at 4.45 fps with the top-5 accuracy of 86.66% using 16-bit quantization. The average performance of convolutional layers and the full CNN is 187.8 GOP/s and 137.0 GOP/s under 150MHz working frequency, which outperform previous approaches significantly.

FastText.zip: Compressing text classification models

Abstract: We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory. After considering different solutions inspired by the hashing literature, we propose a method built upon product quantization to store word embeddings. While the original technique leads to a loss in accuracy, we adapt this method to circumvent quantization artefacts. Our experiments carried out on several benchmarks show that our approach typically requires two orders of magnitude less memory than fastText while being only slightly inferior with respect to accuracy. As a result, it outperforms the state of the art by a good margin in terms of the compromise between memory usage and accuracy.

Discrete Collaborative Filtering

Abstract: We address the efficiency problem of Collaborative Filtering (CF) by hashing users and items as latent vectors in the form of binary codes, so that user-item affinity can be efficiently calculated in a Hamming space. However, existing hashing methods for CF employ binary code learning procedures that most suffer from the challenging discrete constraints. Hence, those methods generally adopt a two-stage learning scheme composed of relaxed optimization via discarding the discrete constraints, followed by binary quantization. We argue that such a scheme will result in a large quantization loss, which especially compromises the performance of large-scale CF that resorts to longer binary codes. In this paper, we propose a principled CF hashing framework called Discrete Collaborative Filtering (DCF), which directly tackles the challenging discrete optimization that should have been treated adequately in hashing. The formulation of DCF has two advantages: 1) the Hamming similarity induced loss that preserves the intrinsic user-item similarity, and 2) the balanced and uncorrelated code constraints that yield compact yet informative binary codes. We devise a computationally efficient algorithm with a rigorous convergence proof of DCF. Through extensive experiments on several real-world benchmarks, we show that DCF consistently outperforms state-of-the-art CF hashing techniques, e.g, though using only 8 bits, DCF is even significantly better than other methods using 128 bits.

A deep neural network for image quality assessment

Abstract: This paper presents a no reference image (NR) quality assessment (IQA) method based on a deep convolutional neural network (CNN). The CNN takes unpreprocessed image patches as an input and estimates the quality without employing any domain knowledge. By that, features and natural scene statistics are learnt purely data driven and combined with pooling and regression in one framework. We evaluate the network on the LIVE database and achieve a linear Pearson correlation superior to state-of-the-art NR IQA methods. We also apply the network to the image forensics task of decoder-sided quantization parameter estimation and also here achieve correlations of r = 0.989.

Double JPEG compression forensics based on a convolutional neural network

Abstract: Double JPEG compression detection has received considerable attention in blind image forensics. However, only few techniques can provide automatic localization. To address this challenge, this paper proposes a double JPEG compression detection algorithm based on a convolutional neural network (CNN). The CNN is designed to classify histograms of discrete cosine transform (DCT) coefficients, which differ between single-compressed areas (tampered areas) and double-compressed areas (untampered areas). The localization result is obtained according to the classification results. Experimental results show that the proposed algorithm performs well in double JPEG compression detection and forgery localization, especially when the first compression quality factor is higher than the second.

Towards the Limit of Network Quantization

Abstract: Network quantization is one of network compression techniques to reduce the redundancy of deep neural networks. It reduces the number of distinct network parameter values by quantization in order to save the storage for them. In this paper, we design network quantization schemes that minimize the performance loss due to quantization given a compression ratio constraint. We analyze the quantitative relation of quantization errors to the neural network loss function and identify that the Hessian-weighted distortion measure is locally the right objective function for the optimization of network quantization. As a result, Hessian-weighted k-means clustering is proposed for clustering network parameters to quantize. When optimal variable-length binary codes, e.g., Huffman codes, are employed for further compression, we derive that the network quantization problem can be related to the entropy-constrained scalar quantization (ECSQ) problem in information theory and consequently propose two solutions of ECSQ for network quantization, i.e., uniform quantization and an iterative solution similar to Lloyd's algorithm. Finally, using the simple uniform quantization followed by Huffman coding, we show from our experiments that the compression ratios of 51.25, 22.17 and 40.65 are achievable for LeNet, 32-layer ResNet and AlexNet, respectively.

A survey on compressed domain video analysis techniques

Abstract: Image and video analysis requires rich features that can characterize various aspects of visual information. These rich features are typically extracted from the pixel values of the images and videos, which require huge amount of computation and seldom useful for real-time analysis. On the contrary, the compressed domain analysis offers relevant information pertaining to the visual content in the form of transform coefficients, motion vectors, quantization steps, coded block patterns with minimal computational burden. The quantum of work done in compressed domain is relatively much less compared to pixel domain. This paper aims to survey various video analysis efforts published during the last decade across the spectrum of video compression standards. In this survey, we have included only the analysis part, excluding the processing aspect of compressed domain. This analysis spans through various computer vision applications such as moving object segmentation, human action recognition, indexing, retrieval, face detection, video classification and object tracking in compressed videos.

Enhancing Sketch-Based Image Retrieval by Re-Ranking and Relevance Feedback

Abstract: A sketch-based image retrieval often needs to optimize the tradeoff between efficiency and precision. Index structures are typically applied to large-scale databases to realize efficient retrievals. However, the performance can be affected by quantization errors. Moreover, the ambiguousness of user-provided examples may also degrade the performance, when compared with traditional image retrieval methods. Sketch-based image retrieval systems that preserve the index structure are challenging. In this paper, we propose an effective sketch-based image retrieval approach with re-ranking and relevance feedback schemes. Our approach makes full use of the semantics in query sketches and the top ranked images of the initial results. We also apply relevance feedback to find more relevant images for the input query sketch. The integration of the two schemes results in mutual benefits and improves the performance of the sketch-based image retrieval.

High Dynamic Range Video Compression Exploiting Luminance Masking

Abstract: The human visual system (HVS) exhibits nonlinear sensitivity to the distortions introduced by lossy image and video coding. This effect is due to the luminance masking, contrast masking, and spatial and temporal frequency masking characteristics of the HVS. This paper proposes a novel perception-based quantization to remove nonvisible information in high dynamic range (HDR) color pixels by exploiting luminance masking so that the performance of the High Efficiency Video Coding (HEVC) standard is improved for HDR content. A profile scaling based on a tone-mapping curve computed for each HDR frame is introduced. The quantization step is then perceptually tuned on a transform unit basis. The proposed method has been integrated into the HEVC reference model for the HEVC range extensions (HM-RExt), and its performance was assessed by measuring the bitrate reduction against the HM-RExt. The results indicate that the proposed method achieves significant bitrate savings, up to 42.2%, with an average of 12.8%, compared with HEVC at the same quality (based on HDR-visible difference predictor-2 and subjective evaluations).

Markov process-based retrieval for encrypted JPEG images

Abstract: This paper develops a retrieval scheme for encrypted JPEG images based on a Markov process. In our scheme, the stream cipher and permutation encryption are combined to encrypt discrete cosine transform (DCT) coefficients for protecting JPEG image content's confidentiality. And thus, it is easy for the content owner to achieve the encrypted JPEG images uploaded to a database server. In the image retrieval stage, although the server does not know the plaintext content of a given encrypted query image, he can still extract image feature calculated from the transition probability matrices related to DCT coefficients, which indicate the intra-block, inter-block, and inter-component dependencies among DCT coefficients. And these three types of dependencies are modeled by the Markov process. After that, with the multi-class support vector machine (SVM), the feature of the encrypted query image can be converted into a vector with low dimensionality determined by the number of image categories. The encrypted database images are conducted similarly. After low-dimensional vector representation, the similarity between the encrypted query image and database image may be evaluated by calculating the distance of their corresponding feature vectors. At the client side, the returned encrypted images similar to the query image can be decrypted to the plaintext images with the help of the encryption key.

Data-Driven Soft Decoding of Compressed Images in Dual Transform-Pixel Domain

Abstract: In the large body of research literature on image restoration, very few papers were concerned with compression-induced degradations, although in practice, the most common cause of image degradation is compression. This paper presents a novel approach to restoring JPEG-compressed images. The main innovation is in the approach of exploiting residual redundancies of JPEG code streams and sparsity properties of latent images. The restoration is a sparse coding process carried out jointly in the DCT and pixel domains. The prowess of the proposed approach is directly restoring DCT coefficients of the latent image to prevent the spreading of quantization errors into the pixel domain, and at the same time, using online machine-learned local spatial features to regulate the solution of the underlying inverse problem. Experimental results are encouraging and show the promise of the new approach in significantly improving the quality of DCT-coded images.

Improving digital image watermarking by means of optimal channel selection

Abstract: Novel digital image watermarking method using a wavelet-based quantization approachOptimal color channel selection scheme for the embeddingOtsu's classification-based adaptive threshold for the extraction processOutperformance of imperceptibility and robustness to state-of-the-art techniques Supporting safe and resilient authentication and integrity of digital images is of critical importance in a time of enormous creation and sharing of these contents This paper presents an improved digital image watermarking model based on a coefficient quantization technique that intelligently encodes the owner's information for each color channel to improve imperceptibility and robustness of the hidden information Concretely, a novel color channel selection mechanism automatically selects the optimal HL4 and LH4 wavelet coefficient blocks for embedding binary bits by adjusting block differences, calculated between LH and HL coefficients of the host image The channel selection aims to minimize the visual difference between the original image and the embedded image On the other hand, the strength of the watermark is controlled by a factor to achieve an acceptable tradeoff between robustness and imperceptibility The arrangement of the watermark pixels before shuffling and the channel into which each pixel is embedded is ciphered in an associated key This key is utterly required to recover the original watermark, which is extracted through an adaptive clustering thresholding mechanism based on the Otsu's algorithm Benchmark results prove the model to support imperceptible watermarking as well as high robustness against common attacks in image processing, including geometric, non-geometric transformations, and lossy JPEG compression The proposed method enhances more than 4źdB in the watermarked image quality and significantly reduces Bit Error Rate in the comparison of state-of-the-art approaches

Texture Classification Using Dense Micro-Block Difference

Abstract: This paper is devoted to the problem of texture classification. Motivated by recent advancements in the field of compressive sensing and keypoints descriptors, a set of novel features called dense micro-block difference (DMD) is proposed. These features provide highly descriptive representation of image patches by densely capturing the granularities at multiple scales and orientations. Unlike most of the earlier work on local features, the DMD does not involve any quantization, thus retaining the complete information. We demonstrate that the DMD have dimensionality much lower than Scale Invariant Feature Transform (SIFT) and can be computed using integral image much faster than SIFT. The proposed features are encoded using the Fisher vector method to obtain an image descriptor, which considers high-order statistics. The proposed image representation is combined with the linear support vector machine classifier. Extensive experiments are conducted on five texture data sets (KTH-TIPS, UMD, KTH-TIPS-2a, Brodatz, and Curet) using standard protocols. The results demonstrate that our approach outperforms the state-of-the-art in texture classification.

Large-scale JPEG steganalysis using hybrid deep-learning framework

Abstract: Adoption of deep learning in image steganalysis is still in its initial stage. In this paper we propose a generic hybrid deep-learning framework for JPEG steganalysis incorporating the domain knowledge behind rich steganalytic models. Our proposed framework involves two main stages. The first stage is hand-crafted, corresponding to the convolution phase and the quantization & truncation phase of the rich models. The second stage is a compound deep neural network containing multiple deep subnets in which the model parameters are learned in the training procedure. We provided experimental evidences and theoretical reflections to argue that the introduction of threshold quantizers, though disable the gradient-descent-based learning of the bottom convolution phase, is indeed cost-effective. We have conducted extensive experiments on a large-scale dataset extracted from ImageNet. The primary dataset used in our experiments contains 500,000 cover images, while our largest dataset contains five million cover images. Our experiments show that the integration of quantization and truncation into deep-learning steganalyzers do boost the detection performance by a clear margin. Furthermore, we demonstrate that our framework is insensitive to JPEG blocking artifact alterations, and the learned model can be easily transferred to a different attacking target and even a different dataset. These properties are of critical importance in practical applications.

A framework of adaptive steganography resisting JPEG compression and detection

Abstract: Current typical adaptive steganography algorithms take the detection resistant capability into account adequately but usually cannot extract the embedded secret messages correctly when stego images suffer from compression attack. In order to solve this problem, a framework of adaptive steganography resisting JPEG compression and detection is proposed. Utilizing the relationship between Discrete Cosine Transformation DCT coefficients, the domain of messages embedding is determined; for the maximum of the JPEG compression resistant ability, the modifying magnitude of different DCT coefficients caused by messages embedding can be determined; in order to ensure the completely correct extraction of embedded messages after JPEG compression, error correct codes are used to encode the messages to be embedded; on the basis of the current distortion functions, the distortion value of DCT coefficients corresponding to the modifying magnitude in the embedding domain can be calculated; to improve the detection resistant ability of the stego images and realize the minimum distortion embedding, syndrome-trellis codes are used to embed the encoded messages into the DCT coefficients that have a smaller distortion value. Based on the proposed framework, an adaptive steganography algorithm resisting JPEG compression and detection is designed, which utilizes the relationship between coefficients in a DCT block and the means of that in three adjacent DCT blocks. The experimental results that demonstrate the proposed algorithm not only has a good JPEG compression resistant ability but also has a strong detection resistant performance. Comparing with current J-UNIWARD steganography under quality factor 85 of JPEG compression, the extraction error rates without pre-compression decrease from about 50% to nearly 0, while the stego images remain a good detection resistant ability comparing with a typical robust watermarking algorithm, which shows the validity of the proposed framework. Copyright © 2016 John Wiley & Sons, Ltd.

Hybrid distortion function for JPEG steganography

Abstract: A hybrid distortion function for JPEG steganography exploiting block fluctuation and quantization steps is proposed. To resist multidomain steganalysis, both spatial domain and discrete cosine transformation (DCT) domain are involved in the proposed distortion function. In spatial domain, a distortion value is allotted for each 8×8 block according to block fluctuation. In DCT domain, quantization steps are employed to allot distortion values for DCT coefficients in a block. The two elements, block distortion and quantization steps, are combined together to measure the embedding risk. By employing the syndrome trellis coding to embed secret data, the embedding changes are constrained in complex regions, where modifications are hard to be detected. When compared to current state-of-the-art steganographic methods for JPEG images, the proposed method presents less detectable artifacts.

Robust blind image watermarking by modulating the mean of partly sign-altered DCT coefficients guided by human visual perception

Abstract: A novel discrete cosine transform (DCT)-based blind watermarking scheme is proposed to embed binary information into images with substantial improvement in robustness against commonly encountered attacks. This scheme modulates the partly sign-altered (PSA) mean of selected DCT coefficients derived from an 8 × 8 image block. To enhance the imperceptibility of the watermark, the variation margin of each coefficient is regulated either by referring to a JPEG quantization table or the just noticeable distortion (JND) of the human visual system. In particular, a companding function is employed to emulate the contrast masking effect of human eyes. Compared with other DCT-based methods executed at a peak signal-to-noise ratio of approximately 40 dB, the incorporation of the JPEG quantization table into the PSA mean modulation achieves significant improvement in terms of bit error rates for attacks like JPEG & JPEG 2000 compression and noise corruption. The exploitation of JND also offers advantages in cases of median filtering, Gaussian lowpass filtering, and scaling correction.

A perceptual visibility metric for banding artifacts

Abstract: Banding is a common video artifact caused by compressing low texture regions with coarse quantization. Relatively few previous attempts exist to address banding and none incorporate subjective testing for calibrating the measurement. In this paper, we propose a novel metric that incorporates both edge length and contrast across the edge to measure video banding. We further introduce both reference and non-reference metrics. Our results demonstrate that the new metrics have a very high correlation with subjective assessment and certainly outperforms PSNR, SSIM, and VQM.

Optimal design of JPEG hardware under the approximate computing paradigm

Abstract: JPEG compression based on the discrete cosine transform (DCT) is a key building block in low-power multimedia applications. We use approximate computing to exploit the error tolerance of JPEG and formulate a novel optimization problem that maximizes power savings under an error budget. We analyze the error propagation sensitivity in the DCT network and use this information to model the impact of introduced errors on the output quality. Simulations show up to 15% reduction in area and delay which corresponds to 40% power savings at iso-delay.

Quality Scalability Aware Watermarking for Visual Content

Abstract: Scalable coding-based content adaptation poses serious challenges to traditional watermarking algorithms, which do not consider the scalable coding structure and hence cannot guarantee correct watermark extraction in media consumption chain. In this paper, we propose a novel concept of scalable blind watermarking that ensures more robust watermark extraction at various compression ratios while not effecting the visual quality of host media. The proposed algorithm generates scalable and robust watermarked image code-stream that allows the user to constrain embedding distortion for target content adaptations. The watermarked image code-stream consists of hierarchically nested joint distortion-robustness coding atoms. The code-stream is generated by proposing a new wavelet domain blind watermarking algorithm guided by a quantization based binary tree. The code-stream can be truncated at any distortion-robustness atom to generate the watermarked image with the desired distortion-robustness requirements. A blind extractor is capable of extracting watermark data from the watermarked images. The algorithm is further extended to incorporate a bit-plane discarding-based quantization model used in scalable coding-based content adaptation, e.g., JPEG2000. This improves the robustness against quality scalability of JPEG2000 compression. The simulation results verify the feasibility of the proposed concept, its applications, and its improved robustness against quality scalable content adaptation. Our proposed algorithm also outperforms existing methods showing 35% improvement. In terms of robustness to quality scalable video content adaptation using Motion JPEG2000 and wavelet-based scalable video coding, the proposed method shows major improvement for video watermarking.

Analyzing the Effect of JPEG Compression on Local Variance of Image Intensity

Abstract: The local variance of image intensity is a typical measure of image smoothness. It has been extensively used, for example, to measure the visual saliency or to adjust the filtering strength in image processing and analysis. However, to the best of our knowledge, no analytical work has been reported about the effect of JPEG compression on image local variance. In this paper, a theoretical analysis on the variation of local variance caused by JPEG compression is presented. First, the expectation of intensity variance of $8\times 8$ non-overlapping blocks in a JPEG image is derived. The expectation is determined by the Laplacian parameters of the discrete cosine transform coefficient distributions of the original image and the quantization step sizes used in the JPEG compression. Second, some interesting properties that describe the behavior of the local variance under different degrees of JPEG compression are discussed. Finally, both the simulation and the experiments are performed to verify our derivation and discussion. The theoretical analysis presented in this paper provides some new insights into the behavior of local variance under JPEG compression. Moreover, it has the potential to be used in some areas of image processing and analysis, such as image enhancement, image quality assessment, and image filtering.

A population-based approach to point-sampling spatial color algorithms.

Abstract: Inspired by the behavior of the human visual system, spatial color algorithms perform image enhancement by correcting the pixel channel lightness based on the spatial distribution of the intensities in the surrounding area. The two visual contrast enhancement algorithms RSR and STRESS belong to this family of models: they rescale the input based on local reference values, which are determined by exploring the image by means of random point samples, called sprays. Due to the use of sampling, they may yield a noisy output. In this paper, we introduce a probabilistic formulation of the two models: our algorithms (RSR-P and STRESS-P) rely implicitly on the whole population of possible sprays. For processing larger images, we also provide two approximated algorithms that exploit a suitable target-dependent space quantization. Those spray population-based formulations outperform RSR and STRESS in terms of the processing time required for the production of noiseless outputs. We argue that this population-based approach, which can be extended to other members of the family, complements the sampling-based approach, in that it offers not only a better control in the design of approximated algorithms, but also additional insight into individual models and their relationships. We illustrate the latter point by providing a model of halo artifact formation.

Spatial Steganalysis Using Contrast of Residuals

Abstract: This letter proposes a novel scheme for spatial steganalysis based on contrast of residuals (CoR). After selecting complex blocks from an uncompressed image by a fluctuation function, the residuals are calculated from the selected blocks and the whole image after applying diverse filters. The CoR is represented as an angle and the norm of residuals is considered as the corresponding weight of angle, which is used as the new steganalysis feature. In the proposed scheme, no quantization and truncation is required and the effective information of long-range dependencies among pixels is kept properly. Also, the dimensionality of feature is linear with the number of residuals. The accuracy of proposed scheme is evaluated on HUGO and WOW algorithms, and the experimental results show that the proposed CoR feature has superior performance at low embedding rate with lower dimensionality.

Compression of interferometric radio-astronomical data

Abstract: Context. The volume of radio-astronomical data is a considerable burden in the processing and storing of radio observations that have high time and frequency resolutions and large bandwidths. For future telescopes such as the Square Kilometre Array (SKA), the data volume will be even larger.Aims. Lossy compression of interferometric radio-astronomical data is considered to reduce the volume of visibility data and to speed up processing.Methods. A new compression technique named “Dysco” is introduced that consists of two steps: a normalization step, in which grouped visibilities are normalized to have a similar distribution; and a quantization and encoding step, which rounds values to a given quantization scheme using a dithering scheme. Several non-linear quantization schemes are tested and combined with different methods for normalizing the data. Four data sets with observations from the LOFAR and MWA telescopes are processed with different processing strategies and different combinations of normalization and quantization. The effects of compression are measured in image plane.Results. The noise added by the lossy compression technique acts similarly to normal system noise. The accuracy of Dysco is depending on the signal-to-noise ratio (S/N) of the data: noisy data can be compressed with a smaller loss of image quality. Data with typical correlator time and frequency resolutions can be compressed by a factor of 6.4 for LOFAR and 5.3 for MWA observations with less than 1% added system noise. An implementation of the compression technique is released that provides a Casacore storage manager and allows transparent encoding and decoding. Encoding and decoding is faster than the read/write speed of typical disks.Conclusions. The technique can be used for LOFAR and MWA to reduce the archival space requirements for storing observed data. Data from SKA-low will likely be compressible by the same amount as LOFAR. The same technique can be used to compress data from other telescopes, but a different bit-rate might be required.

A Review of Image Compression Techniques

Abstract: An image compression method eradicates redundant and/or unrelated information, and resourcefully encodes leftovers. Practically, it is frequently essential to toss away both non redundant information and relevant information to attain the essential compression. In any case, the ploy is discovering methods that permit important information to be resourcefully extracted and represented. This paper copes with dissimilar compression methods for comprising the information in an image. The information can be compressed by means of Lossy techniques such as Quantization, Transform coding, Block Transform Coding or Lossless techniques such as Run Length Coding, Lossless Predictive Coding, Multi-resolution Coding. All these techniques have been discussed in this paper and the performance of any technique/method is analyzed on various parameters like MSE and PSNR.

Geometry-driven quantization for omnidirectional image coding

Abstract: In this paper we propose a method to adapt the quantization tables of typical block-based transform codecs when the input to the encoder is a panoramic image resulting from equirectangular projection of a spherical image. When the visual content is projected from the panorama to the viewport, a frequency shift is occurring. The quantization can be adapted accordingly: the quantization step sizes that would be optimal to quantize the transform coefficients of the viewport image block, can be used to quantize the coefficients of the panoramic block. As a proof of concept, the proposed quantization strategy has been used in JPEG compression. Results show that a rate reduction up to 2.99% can be achieved for the same perceptual quality of the spherical signal with respect to a standard quantization.