IEEE Transactions on Pattern Analysis and Machine Intelligence

Generative adversarial network in medical imaging: A review.

This paper concerns the estimation of facial attributes—namely, age and gender—from images of faces acquired in challenging, in the wild conditions This problem has received far less attention than the related problem of face recognition, and in particular, has not enjoyed the same dramatic improvement in capabilities demonstrated by contemporary face recognition systems Here, we address this problem by making the following contributions First, in answer to one of the key problems of age estimation research—absence of data—we offer a unique data set of face images, labeled for age and gender, acquired by smart-phones and other mobile devices, and uploaded without manual filtering to online image repositories We show the images in our collection to be more challenging than those offered by other face-photo benchmarks Second, we describe the dropout-support vector machine approach used by our system for face attribute estimation, in order to avoid over-fitting This method, inspired by the dropout learning techniques now popular with deep belief networks, is applied here for training support vector machines, to the best of our knowledge, for the first time Finally, we present a robust face alignment technique, which explicitly considers the uncertainties of facial feature detectors We report extensive tests analyzing both the difficulty levels of contemporary benchmarks as well as the capabilities of our own system These show our method to outperform state-of-the-art by a wide margin

/pdf/age-and-gender-estimation-of-unfiltered-faces-2ujy89s1qr.pdf

Age and Gender Estimation of Unfiltered Faces

We study on image super-resolution (SR), which aims to recover realistic textures from a low-resolution (LR) image. Recent progress has been made by taking high-resolution images as references (Ref), so that relevant textures can be transferred to LR images. However, existing SR approaches neglect to use attention mechanisms to transfer high-resolution (HR) textures from Ref images, which limits these approaches in challenging cases. In this paper, we propose a novel Texture Transformer Network for Image Super-Resolution (TTSR), in which the LR and Ref images are formulated as queries and keys in a transformer, respectively. TTSR consists of four closely-related modules optimized for image generation tasks, including a learnable texture extractor by DNN, a relevance embedding module, a hard-attention module for texture transfer, and a soft-attention module for texture synthesis. Such a design encourages joint feature learning across LR and Ref images, in which deep feature correspondences can be discovered by attention, and thus accurate texture features can be transferred. The proposed texture transformer can be further stacked in a cross-scale way, which enables texture recovery from different levels (e.g., from 1x to 4x magnification). Extensive experiments show that TTSR achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations.

/pdf/learning-texture-transformer-network-for-image-super-yk0ycq2wze.pdf

Learning Texture Transformer Network for Image Super-Resolution

A transition between energy levels at an avoided crossing is known as a Landau-Zener transition. When a two-level system (TLS) is subject to periodic driving with sufficiently large amplitude, a sequence of transitions occurs. The phase accumulated between transitions (commonly known as the Stuckelberg phase) may result in constructive or destructive interference. Accordingly, the physical observables of the system exhibit periodic dependence on the various system parameters. This phenomenon is often referred to as Landau-Zener-Stuckelberg (LZS) interferometry. Phenomena related to LZS interferometry occur in a variety of physical systems. In particular, recent experiments on LZS interferometry in superconducting TLSs (qubits) have demonstrated the potential for using this kind of interferometry as an effective tool for obtaining the parameters characterizing the TLS as well as its interaction with the control fields and with the environment. Furthermore, strong driving could allow for fast and reliable control of the quantum system. Here we review recent experimental results on LZS interferometry, and we present related theory.

Landau-Zener-Stuckelberg interferometry

With the recent availability of commercial light field cameras, we can foresee a future in which light field signals will be as common place as images. Hence, there is an imminent need to address the problem of light field processing. We provide a common framework for addressing many of the light field processing tasks, such as denoising, angular and spatial superresolution, etc. (in essence, all processing tasks whose observation models are linear). We propose a patch based approach, where we model the light field patches using a Gaussian mixture model (GMM). We use the ”disparity pattern” of the light field data to design the patch prior. We show that the light field patches with the same disparity value (i.e., at the same depth from the focal plane) lie on a low-dimensional subspace and that the dimensionality of such subspaces varies quadratically with the disparity value. We then model the patches as Gaussian random variables conditioned on its disparity value, thus, effectively leading to a GMM model. During inference, we first find the disparity value of a patch by a fast subspace projection technique and then reconstruct it using the LMMSE algorithm. With this prior and inference algorithm, we show that we can perform many different processing tasks under a common framework.

https://www.ece.rice.edu/~av21/Documents/2012/Light%20field%20denoising,%20light%20field%20superresolution%20and%20stereo%20camera%20based%20refocussing%20using%20a%20GMM%20light%20field%20patch%20prior.pdf

Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior

Current light field (LF) cameras provide low spatial resolution and limited depth-of-field (DOF) control when compared to traditional digital SLR (DSLR) cameras. We show that a hybrid imaging system consisting of a standard LF camera and a high-resolution standard camera enables (a) achieve high-resolution digital refocusing, (b) better DOF control than LF cameras, and (c) render graceful high-resolution viewpoint variations, all of which were previously unachievable. We propose a simple patch-based algorithm to super-resolve the low-resolution views of the light field using the high-resolution patches captured using a high-resolution SLR camera. The algorithm does not require the LF camera and the DSLR to be co-located or for any calibration information regarding the two imaging systems. We build an example prototype using a Lytro camera (380×380 pixel spatial resolution) and a 18 megapixel (MP) Canon DSLR camera to generate a light field with 11 MP resolution (9× super-resolution) and about 1 over 9 th of the DOF of the Lytro camera. We show several experimental results on challenging scenes containing occlusions, specularities and complex non-lambertian materials, demonstrating the effectiveness of our approach.

Improving resolution and depth-of-field of light field cameras using a hybrid imaging system

Glaucoma is a highly threatening and widespread ocular disease which may lead to permanent loss in vision. One of the important parameters used for Glaucoma screening in the cup-to-disc ratio (CDR), which requires accurate segmentation of optic cup and disc. We explore fully convolutional networks (FCNs) for the task of joint segmentation of optic cup and disc. We propose a novel improved architecture building upon FCNs by using the concept of residual learning. Additionally, we also explore if adversarial training helps in improving the segmentation results. The method does not require any complicated preprocessing techniques for feature enhancement. We learn a mapping between the retinal images and the corresponding segmentation map using fully convolutional and adversarial networks. We perform extensive experiments of various models on a set of 159 images from RIM-ONE database and also do extensive comparison. The proposed method outperforms the state of the art methods on various evaluation metrics for both disc and cup segmentation.

Joint Optic Disc and Cup Segmentation Using Fully Convolutional and Adversarial Networks

We consider the problem of automatic age estimation from face images. Age estimation is usually formulated as a regression problem relating the facial features and the age variable, and a single regression model is learnt for all ages. We propose a hierarchical approach, where we first divide the face images into various age groups and then learn a separate regression model for each group. Given a test image, we first classify the image into one of the age groups and then use the regression model for that particular group. To improve our classification result, we use many different classifiers and fuse them using the majority rule. Experiments show that our approach outperforms many state of the art regression methods for age estimation.

/pdf/a-hierarchical-approach-for-human-age-estimation-4dbfd4r2av.pdf

A hierarchical approach for human age estimation

We address the problem of unconstrained face recognition from remotely acquired images. The main factors that make this problem challenging are image degradation due to blur, and appearance variations due to illumination and pose. In this paper, we address the problems of blur and illumination. We show that the set of all images obtained by blurring a given image forms a convex set. Based on this set-theoretic characterization, we propose a blur-robust algorithm whose main step involves solving simple convex optimization problems. We do not assume any parametric form for the blur kernels, however, if this information is available it can be easily incorporated into our algorithm. Furthermore, using the low-dimensional model for illumination variations, we show that the set of all images obtained from a face image by blurring it and by changing the illumination conditions forms a bi-convex set. Based on this characterization, we propose a blur and illumination-robust algorithm. Our experiments on a challenging real dataset obtained in uncontrolled settings illustrate the importance of jointly modeling blur and illumination.

/pdf/blur-and-illumination-robust-face-recognition-via-set-2ir3ap9e56.pdf

Kaushik Mitra

Papers

Light field denoising, light field superresolution and stereo camera based refocussing using a GMM light field patch prior

Improving resolution and depth-of-field of light field cameras using a hybrid imaging system

Joint Optic Disc and Cup Segmentation Using Fully Convolutional and Adversarial Networks

A hierarchical approach for human age estimation

Blur and Illumination Robust Face Recognition via Set-Theoretic Characterization