We present a new algorithm for manifold learning and nonlinear dimensionality reduction. Based on a set of unorganized da-ta points sampled with noise from a parameterized manifold, the local geometry of the manifold is learned by constructing an approxi-mation for the tangent space at each point, and those tangent spaces are then aligned to give the global coordinates of the data pointswith respect to the underlying manifold. We also present an error analysis of our algorithm showing that reconstruction errors can bequite small in some cases. We illustrate our algorithm using curves and surfaces both in 2D/3D Euclidean spaces and higher dimension-al Euclidean spaces. We also address several theoretical and algorithmic issues for further research and improvements.

Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment

Deep learning on image denoising: An overview.

Perceptual quality assessmentplays a vital role in the visual communication systems owing to theexistence of quality degradations introduced in various stages of visual signalacquisition, compression, transmission and display.Quality assessment for visual signals can be performed subjectively andobjectively, and objective quality assessment is usually preferred owing to itshigh efficiency and easy deployment. A large number of subjective andobjective visual quality assessment studies have been conducted during recent years.In this survey, we give an up-to-date and comprehensivereview of these studies.Specifically, the frequently used subjective image quality assessment databases are firstreviewed, as they serve as the validation set for the objective measures.Second, the objective image quality assessment measures are classified and reviewed according to the applications and the methodologies utilized in the quality measures.Third, the performances of the state-of-the-artquality measures for visual signals are compared with an introduction of theevaluation protocols.This survey provides a general overview of classical algorithms andrecent progresses in the field of perceptual image quality assessment.

Perceptual image quality assessment: a survey

Perceptual quality assessment is a challenging issue in 3D signal processing research. It is important to study 3D signal directly instead of studying simple extension of the 2D metrics directly to the 3D case as in some previous studies. In this paper, we propose a new perceptual full-reference quality assessment metric of stereoscopic images by considering the binocular visual characteristics. The major technical contribution of this paper is that the binocular perception and combination properties are considered in quality assessment. To be more specific, we first perform left-right consistency checks and compare matching error between the corresponding pixels in binocular disparity calculation, and classify the stereoscopic images into non-corresponding, binocular fusion, and binocular suppression regions. Also, local phase and local amplitude maps are extracted from the original and distorted stereoscopic images as features in quality assessment. Then, each region is evaluated independently by considering its binocular perception property, and all evaluation results are integrated into an overall score. Besides, a binocular just noticeable difference model is used to reflect the visual sensitivity for the binocular fusion and suppression regions. Experimental results show that compared with the relevant existing metrics, the proposed metric can achieve higher consistency with subjective assessment of stereoscopic images.

Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics

State-of-the-art algorithms for blind image quality assessment (BIQA) typically have two categories. The first category approaches extract natural scene statistics (NSS) as features based on the statistical regularity of natural images. The second category approaches extract features by feature encoding with respect to a learned codebook. However, several problems need to be addressed in existing codebook-based BIQA methods. First, the high-dimensional codebook-based features are memory-consuming and have the risk of over-fitting. Second, there is a semantic gap between the constructed codebook by unsupervised learning and image quality. To address these problems, we propose a novel codebook-based BIQA method by optimizing multistage discriminative dictionaries (MSDDs). To be specific, MSDDs are learned by performing the label consistent K-SVD (LC-KSVD) algorithm in a stage-by-stage manner. For each stage, a new quality consistency constraint called “quality-discriminative regularization” term is introduced and incorporated into the reconstruction error term to form a unified objective function, which can be effectively solved by LC-KSVD for discriminative dictionary learning. Then, the latter stage takes the reconstruction residual data in the former stage as input based on which LC-KSVD is repeatedly performed until the final stage is reached. Once the MSDDs are learned, multistage feature encoding is performed to extract feature codes. Finally, the feature codes are concatenated across all stages and aggregated over the entire image for quality prediction via regression. The proposed method has been evaluated on five databases and experimental results well confirm its superiority over existing relevant BIQA methods.

Optimizing Multistage Discriminative Dictionaries for Blind Image Quality Assessment

High Efficiency Video Coding (HEVC) improves the compression efficiency at the cost of high computational complexity by using the quad-tree coding unit (CU) structure and variable prediction unit (PU) modes. To minimize the HEVC encoding complexity while maintaining its compression efficiency, a binary and multi-class support vector machine (SVM)-based fast HEVC encoding algorithm is presented in this paper. First, the processes of recursive CU decision and PU selection in HEVC are modeled as hierarchical binary classification and multi-class classification structures. Second, according to the two classification structures, the CU decision and PU selection are optimized by binary and multi-class SVM, i.e., the CU and PU can be predicted directly via classifiers without intensive rate distortion (RD) cost calculation. In particular, to achieve better prediction performance, a learning method is proposed to combine the off-line machine learning (ML) mode and on-line ML mode for classifiers based on a multiple reviewers system. Additionally, the optimal parameters determination scheme is adopted for flexible complexity allocation under a given RD constraint. Experimental results show that the proposed method can achieve 68.3%, 67.3%, and 65.6% time saving on average while the values of Bjontegaard delta peak signal-to-noise ratio are −0.093dB, −0.091dB, and −0.094dB and the values of Bjontegaard delta bit rate are 4.191%, 3.842%, and 3.665% under low delay $P$ main, low delay main, and random access configurations, respectively, when compared with the HEVC test model version HM 16.5. Meanwhile, the proposed method outperforms the state-of-the-art fast coding algorithms in terms of complexity reduction and RD performance.

Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding

Subjective quality evaluation is the basis of quality evaluation of stereoscopic images. As the lack of a public and diverse testing database currently, in this paper, a symmetric stereoscopic images database is built. And then the subjective quality of stereoscopic images is analyzed from two aspects, one is the effects of JPEG, JPEG2000, H.264. The other is the comparisons between symmetric and asymmetric stereoscopic images from Gaussian blurring, white Gaussian noise, JPEG and JPEG2000, respectively. The results show three compressions are quite different in the subjective quality of symmetric stereoscopic images at different bitrates, and the comparisons between symmetric and asymmetric stereoscopic images investigate the properties of binocular fusion, binocular suppression, and binocular summation.

Subjective quality analyses of stereoscopic images in 3DTV system

To address issues of image authenticity verification and integrity protection, a fragile watermarking method for stereo image authentication is proposed with stereo matching technique. According to stereo matching of left and right images, non-overlapping blocks of left image are classified into non-matchable and matchable ones. Right image blocks and non-matchable blocks of left image are categorized into smooth, texture and complex blocks considering their roughness. Then, the corresponding alterable-length watermark is computed based on the results of block classification, and the destroyed blocks are recovered by using the corresponding alterable-length watermark. Besides, matchable tampered blocks of left image are recovered with the matched contents in right image. A detection technique with the part of alterable-length watermark is used to increase accuracy of tamper localization at the receiver. The disparity can be used to recover the tampered matchable block. Therefore, the quality of watermarked images is improved because of low embedding capacity. Additionally, the alterable-capacity watermark provides suitable information for smoother images with fewer bits and we obtain high quality restoration. Experimental results show that the proposed method can not only localize the tampered regions precisely but also recover the tampered regions with higher quality compared with the state-of-the-art methods.

New fragile watermarking method for stereo image authentication with localization and recovery

Three-dimensional (3-D) visual comfort assessment (VCA) is a particularly important and challenging topic, which involves automatically predicting the degree of visual comfort in line with human subjective judgment. State-of-the-art VCA models typically focus on minimizing the distance between predicted visual comfort scores and subjective mean opinion scores (MOSs) by training a regression model. However, obtaining precise MOSs is often expensive and time-consuming, which greatly constrains the extension of existing MOS-aware VCA models. This study is inspired by the fact that humans tend to conduct a preference judgment between two stereoscopic images in terms of visual comfort. We propose to train a robust VCA model on a set of preference labels instead of MOSs. The preference label, representing the relative visual comfort of preference stereoscopic image pairs (PSIPs), is generally precise and can be obtained at much lower cost compared with MOS. More specifically, some representative stereoscopic images are first selected to generate the PSIP training set. Then, we use a support vector machine to learn a preference classification model by taking a differential feature vector and the corresponding preference label of each PSIP as input. Finally, given a testing sample, by considering a full-round paired comparison with all the selected representative stereoscopic images, the visual comfort score can be estimated via a simple linear mapping strategy. Experimental results on our newly built 3-D image database demonstrate that the proposed method can achieve a better performance compared with the models trained on MOSs.

Three-dimensional visual comfort assessment via preference learning

With the emerging development of three-dimensional (3D) related technologies, 3D visual saliency modeling is becoming particularly important and challenging. This paper presents a new depth perception and visual comfort guided saliency computational model for stereoscopic 3D images. The prominent advantage of the proposed model is that we incorporate the influence of depth perception and visual comfort on 3D visual saliency computation. The proposed saliency model is composed of three components: 2D image saliency, depth saliency and visual comfort based saliency. In the model, color saliency, texture saliency and spatial compactness are computed respectively and fused to derive 2D image saliency. Global disparity contrast is considered to compute depth saliency. Particularly, we train a visual comfort prediction function to distinguish stereoscopic image pair as high comfortable stereo viewing (HCSV) or low comfortable stereo viewing (LCSV), and devise different computational rules to generate a visual comfort based saliency map. The final 3D saliency map is obtained by using a linear combination and enhanced by a "saliency-center bias" model. Experimental results show that the proposed 3D saliency model outperforms the state-of-the-art models on predicting human eye fixations and visual comfort assessment. Relationship between 3D visual discomfort and visual attention is analyzed.Different saliency computational rules for high/low comfort 3D images are derived.Low-level features, depth cues and visual comfort are considered for 3D saliency.

Zongju Peng

Papers

Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding

Subjective quality analyses of stereoscopic images in 3DTV system

New fragile watermarking method for stereo image authentication with localization and recovery

Three-dimensional visual comfort assessment via preference learning

A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency