In the last few years, the deep learning (DL) computing paradigm has been deemed the Gold Standard in the machine learning (ML) community. Moreover, it has gradually become the most widely used computational approach in the field of ML, thus achieving outstanding results on several complex cognitive tasks, matching or even beating those provided by human performance. One of the benefits of DL is the ability to learn massive amounts of data. The DL field has grown fast in the last few years and it has been extensively used to successfully address a wide range of traditional applications. More importantly, DL has outperformed well-known ML techniques in many domains, e.g., cybersecurity, natural language processing, bioinformatics, robotics and control, and medical information processing, among many others. Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it. Therefore, in this contribution, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of DL. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of DL and including those enhancements recently added to the field. In particular, this paper outlines the importance of DL, presents the types of DL techniques and networks. It then presents convolutional neural networks (CNNs) which the most utilized DL network type and describes the development of CNNs architectures together with their main features, e.g., starting with the AlexNet network and closing with the High-Resolution network (HR.Net). Finally, we further present the challenges and suggested solutions to help researchers understand the existing research gaps. It is followed by a list of the major DL applications. Computational tools including FPGA, GPU, and CPU are summarized along with a description of their influence on DL. The paper ends with the evolution matrix, benchmark datasets, and summary and conclusion.

/pdf/review-of-deep-learning-concepts-cnn-architectures-hpbk62qzc2.pdf

Review of deep learning: concepts, CNN architectures, challenges, applications, future directions

In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance comparison and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded.

Knowledge Distillation: A Survey

The integration of spatial context in the classification of hyperspectral images is known to be an effective way in improving classification accuracy. In this paper, a novel spectral-spatial classification framework based on edge-preserving filtering is proposed. The proposed framework consists of the following three steps. First, the hyperspectral image is classified using a pixelwise classifier, e.g., the support vector machine classifier. Then, the resulting classification map is represented as multiple probability maps, and edge-preserving filtering is conducted on each probability map, with the first principal component or the first three principal components of the hyperspectral image serving as the gray or color guidance image. Finally, according to the filtered probability maps, the class of each pixel is selected based on the maximum probability. Experimental results demonstrate that the proposed edge-preserving filtering based classification method can improve the classification accuracy significantly in a very short time. Thus, it can be easily applied in real applications.

Spectral–Spatial Hyperspectral Image Classification With Edge-Preserving Filtering

Facial expression recognition with Convolutional Neural Networks

Inferring a high dynamic range (HDR) image from a single low dynamic range (LDR) input is an ill-posed problem where we must compensate lost data caused by under-/over-exposure and color quantization. To tackle this, we propose the first deep-learning-based approach for fully automatic inference using convolutional neural networks. Because a naive way of directly inferring a 32-bit HDR image from an 8-bit LDR image is intractable due to the difficulty of training, we take an indirect approach; the key idea of our method is to synthesize LDR images taken with different exposures (i.e., bracketed images) based on supervised learning, and then reconstruct an HDR image by merging them. By learning the relative changes of pixel values due to increased/decreased exposures using 3D deconvolutional networks, our method can reproduce not only natural tones without introducing visible noise but also the colors of saturated pixels. We demonstrate the effectiveness of our method by comparing our results not only with those of conventional methods but also with ground-truth HDR images.

Deep reverse tone mapping

In this paper, we propose a switching bilateral filter (SBF) with a texture and noise detector for universal noise removal. Operation was carried out in two stages: detection followed by filtering. For detection, we propose the sorted quadrant median vector (SQMV) scheme, which includes important features such as edge or texture information. This information is utilized to allocate a reference median from SQMV, which is in turn compared with a current pixel to classify it as impulse noise, Gaussian noise, or noise-free. The SBF removes both Gaussian and impulse noise without adding another weighting function. The range filter inside the bilateral filter switches between the Gaussian and impulse modes depending upon the noise classification result. Simulation results show that our noise detector has a high noise detection rate as well as a high classification rate for salt-and-pepper, uniform impulse noise and mixed impulse noise. Unlike most other impulse noise filters, the proposed SBF achieves high peak signal-to-noise ratio and great image quality by efficiently removing both types of mixed noise, salt-and-pepper with uniform noise and salt-and-pepper with Gaussian noise. In addition, the computational complexity of SBF is significantly less than that of other mixed noise filters.

Switching Bilateral Filter With a Texture/Noise Detector for Universal Noise Removal

Unified efficient computations of the discrete cosine transform (DCT), discrete sine transform (DST), discrete Hartley transform (DHT), and their inverse transforms employing the time-recursive approach are considered. Unified parallel lattice structures that can dually generate the DCT and DST simultaneously as well the DHT are developed. These structures can obtain the transformed data for sequential input time recursively with a throughput rate of one per clock cycle. The total number of multipliers required is a linear function of the transform size N, with no constraint on N. The resulting architectures are regular, modular, and without global communication so that they are very suitable for VLSI implementation for high-speed applications. It is shown that the DCT, DST, DHT and their inverse transforms share an almost identical lattice structure. The tradeoff between time and area for the block data processing is considered. >

/pdf/unified-parallel-lattice-structures-for-time-recursive-2aiv6nfh1r.pdf

Unified parallel lattice structures for time-recursive discrete cosine/sine/Hartley transforms

New generations of display technologies provide a significantly improved dynamic range compared to conventional display devices. Inverse tone mapping methods have been proposed to convert low dynamic range (LDR) images to HDR ones, and several of them require multiple exposure LDR images of the same scene as inputs. However, the vast majority of LDR images and videos available have only one single exposure. In this paper, we propose a region-based enhancement of the pseudo-exposures to generate an HDR image. First, we present an exposure dependent $S$ curve to convert one LDR image to the pseudo-multiple-exposures. Only certain regions of the pseudo-exposures contain noticeable detail information. We propose a region-based enhancement on the pseudo-exposures to boost details in the most distinct region. Thereby the region-enhanced pseudo-exposures are fused into an HDR image. The fused image thus enhances details in the bright region of the dark image and the dark region of the bright image. Compared with other inverse tone mapped methods, our method generates lower total contrast error measured under the dynamic range independent image quality assessment method in .

Pseudo-Multiple-Exposure-Based Tone Fusion With Local Region Adjustment

The aggressive advent in VLSI manufacturing technology has made dramatic impacts on the dependability of devices and interconnects. In the modern manycore system, mesh based Networks-on-Chip (NoC) is widely adopted as on chip communication infrastructure. It is critical to provide an effective fault tolerance scheme on mesh based NoC. A faulty router or broken link isolates a well functional processing element (PE). Also, a set of faulty routers form faulty regions which may break down the whole design. To address these issues, we propose an innovative router-level fault tolerance scheme with spare routers which is different from the traditional microarchitecture-level approach. The spare routers not only provide redundancies but also diversify connection paths between adjacent routers. To exploit these valuable resources on fault tolerant capabilities, two configuration algorithms are demonstrated. One is shift-and-replace-allocation (SARA) and the other is defect-awareness-path-allocation (DAPA) that takes advantage of path diversity in our architecture. The proposed design is transparent to any routing algorithm since the output topology is consistent to the original mesh. Experimental results show that our scheme has remarkable improvements on fault tolerant metrics including reliability, mean time to failure (MTTF), and yield. In addition, the performance of spare router increases with the growth of NoC size but the relative connection cost decreases at the same time. This rare and valuable characteristic makes our solution suitable for large scale NoC design.

On the design and analysis of fault tolerant NoC architecture using spare routers

An optimal unified architecture that can efficiently compute the discrete cosine, sine, Hartley, Fourier, lapped orthogonal, and complex lapped transforms for a continuous stream of input data that arise in signal/image communications is proposed. This structure uses only half as many multipliers as the previous best known scheme (Liu and Chiu, 1993). The proposed architecture is regular, modular, and has only local interconnections in both data and control paths. There is no limitation on the transform size N and only 2N-2 multipliers are needed for the DCT. The throughput of this scheme is one input sample per clock cycle. The authors provide a theoretical justification by showing that any discrete transform whose basis functions satisfy the fundamental recurrence formula has a second-order autoregressive structure in its filter realization. They also demonstrate that dual generation transform pairs share the same autoregressive structure. They extend these time-recursive concepts to multi-dimensional transforms. The resulting d-dimensional structures are fully-pipelined and consist of only d 1D transform arrays and shift registers. >

/pdf/optimal-unified-architectures-for-the-real-time-computation-4sjssh4km2.pdf

Ching-Te Chiu

Papers

Switching Bilateral Filter With a Texture/Noise Detector for Universal Noise Removal

Unified parallel lattice structures for time-recursive discrete cosine/sine/Hartley transforms

Pseudo-Multiple-Exposure-Based Tone Fusion With Local Region Adjustment

On the design and analysis of fault tolerant NoC architecture using spare routers

Optimal unified architectures for the real-time computation of time-recursive discrete sinusoidal transforms