We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

[신간의 별자리x] 우리/미술, 그리고 ‘슬픔의 박물관’

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

I have developed "tennis elbow" from lugging this book around the past four weeks, but it is worth the pain, the effort, and the aspirin. It is also worth the (relatively speaking) bargain price. Including appendixes, this book contains 894 pages of text. The entire panorama of the neural sciences is surveyed and examined, and it is comprehensive in its scope, from genomes to social behaviors. The editors explicitly state that the book is designed as "an introductory text for students of biology, behavior, and medicine," but it is hard to imagine any audience, interested in any fragment of neuroscience at any level of sophistication, that would not enjoy this book. The editors have done a masterful job of weaving together the biologic, the behavioral, and the clinical sciences into a single tapestry in which everyone from the molecular biologist to the practicing psychiatrist can find and appreciate his or

Principles of Neural Science

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large upscaling factors? The behavior of optimization-based super-resolution methods is principally driven by the choice of the objective function. Recent work has largely focused on minimizing the mean squared reconstruction error. The resulting estimates have high peak signal-to-noise ratios, but they are often lacking high-frequency details and are perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. In this paper, we present SRGAN, a generative adversarial network (GAN) for image super-resolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4x upscaling factors. To achieve this, we propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space. Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive mean-opinion-score (MOS) test shows hugely significant gains in perceptual quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of the original high-resolution images than to those obtained with any state-of-the-art method.

/pdf/photo-realistic-single-image-super-resolution-using-a-2y0438rlmd.pdf

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

In compressive sensing, wavelet space is widely used to generate sparse signal (image signal in particular) representations. In this paper, we propose a novel approach of statistical context modeling to increase the level of sparsity of wavelet image representations. It is shown, contrary to a widely held assumption, that high-frequency wavelet coefficients have nonzero mean distributions if conditioned on local image structures. Removing this bias can make wavelet image representations sparser, i.e., having a greater number of zero and close-to-zero coefficients. The resulting unbiased probability models can significantly improve the performance of existing wavelet-based compressive image reconstruction methods in both PSNR and visual quality. An efficient algorithm is presented to solve the compressive image recovery (CIR) problem using the refined models. Experimental results on both simulated compressive sensing (CS) image data and real CS image data show that the new CIR method significantly outperforms existing CIR methods in both PSNR and visual quality.

Sparsity Fine Tuning in Wavelet Domain With Application to Compressive Image Reconstruction

The challenge of image impulse noise removal is to restore spatial details from damaged pixels using remaining ones in random locations. Most existing methods use all uncontaminated pixels within a local window to estimate the centred noisy one via a statistic way. These kinds of methods have two defects. First, all noisy pixels are treated as independent individuals and estimated by their neighbours one by one, with the correlation between their true values ignored. Second, the image structure as a natural feature is usually ignored. This study proposes a new denoising framework, in which all noisy pixels are jointly restored via non-uniform sampling and supervised piecewise autoregressive modelling based super-resolution. In this method, the noisy pixels are jointly estimated in groups through solving a well-designed optimisation problem, in which image structure feature is considered as an important constraint. Another contribution is that piecewise autoregressive model is not simply adopted but carefully designed so that all noise-free pixels can be used to supervise the model training and optimisation problem solving for higher accuracy. The experimental results demonstrate that the proposed method exhibits good denoising performance in a large noise density range (10–90%).

https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/iet-ipr.2015.0216

High quality impulse noise removal via non-uniform sampling and autoregressive modelling based super-resolution

This article introduces a new contrast enhancement algorithm of tone-preserving entropy maximization Its design objective is to present the maximal amount of information content in the enhanced image, or being optimal in an information theoretical sense, while preventing the loss of tone continuity The resulting optimization problem can be graph-theoretically modeled as the construction of K-edge maximum-weight path, and it can be solved efficiently by dynamic programming Moreover, the proposed algorithm is made more effective by being combined with a preprocess of image restoration that aims to correct quantization errors caused by the analog-to-digital conversion of image signals Empirical evidences are provided to demonstrate the superior visual quality obtained by the new image enhancement algorithm

Image enhancement by entropy maximization and quantization resolution upconversion

Mobile data offloading allows alternative network systems such as Wi-Fi hotspot access points (APs) to offload the traffic that is originally targeted for the cellular network operators (CNOs). It can alleviate the cellular network congestions and enhance users’ quality-of-service. One of the main challenges of the mobile data offloading is to develop simple and distributed traffic allocation strategies that can optimally allocate the cellular data traffic from multiple CNOs to the APs. In this work, we propose a distributed optimization framework with decomposition-coordination to solve this problem. In our framework, the utility maximization problem of the cellular network system with mobile data offloading is formulated as a nonsmooth convex optimization problem. We then divide this problem into a set of subproblems, and each of them is solved by a CNO or an AP using its local information. All the subproblems are coordinated with each other by a virtual data offloading coordinator (VDOC), which can collect the intermediate calculation values from CNOs and APs. The VDOC will then feedback the coordination results calculated from the collected values to the corresponding CNOs and APs. We propose two distributed algorithms that can achieve the global optimization solution under different scenarios. The first one is the multiblock proximal Jacobi alternating direction method of multipliers (ProxJ-ADMM). In this algorithm, the communication between the VDOC and CNOs or APs is assumed to be fully synchronized, and the VDOC will only feedback the coordination result after successfully receiving all the values from APs and CNOs. We relax this assumption by introducing the second algorithm referred to as the distributed asynchronized ADMM (Async-ADMM) algorithm. In this algorithm, the coordination between the VDOC and APs or CNOs does not need to be perfectly synchronized and the VDOC will feedback a coordination result whenever it receives a value from at least one AP or CNO. We prove that both algorithm can achieve the global optimal solution. We present the numerical results to verify the performance of our proposed approaches under various network settings and conditions.

A Distributed ADMM Approach With Decomposition-Coordination for Mobile Data Offloading

The wavelet transform (WT) is widely used in ultrasonic flaw detection (UFD) systems because of its property of multiresolution time-frequency analysis. Those traditional WT-based methods for UFD use a wavelet basis with limited types to match various echo signals (called wavelet matching signals), so it is difficult for those methods to achieve the optimal match between echo signal and wavelet basis. This results in limited detection ability in high background noise for those WT-based methods. In this paper, we propose a signal matching wavelet (SMW) method for UFD to solve this problem. Unlike traditional UFD systems, in the proposed SMW, the transmitted signal is designed to be a wavelet function for matching a wavelet basis. This makes it possible to obtain the optimal match between the echo signal and the wavelet basis. To achieve the optimal match from the aspect of energy, we derive three rules for designing the transmitted signal and selecting the wavelet basis. Further, the parameter selection in applying the proposed SMW method to a practical UFD system is analyzed. In addition, a low-rate discrete WT structure is designed to decrease the hardware cost, which facilitates the practical application of the proposed SMW. The simulation results show that the proposed SMW can efficiently detect flaws in high background noise even with SNR lower than -20 dB, outperforming the existing methods by 5 dB.

Guangming Shi

Papers

Sparsity Fine Tuning in Wavelet Domain With Application to Compressive Image Reconstruction

High quality impulse noise removal via non-uniform sampling and autoregressive modelling based super-resolution

Image enhancement by entropy maximization and quantization resolution upconversion

A Distributed ADMM Approach With Decomposition-Coordination for Mobile Data Offloading

Signal matching wavelet for ultrasonic flaw detection in high background noise