Deep Convolutional Neural Network (CNN) is a special type of Neural Networks, which has shown exemplary performance on several competitions related to Computer Vision and Image Processing. Some of the exciting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, and Speech Recognition. The powerful learning ability of deep CNN is primarily due to the use of multiple feature extraction stages that can automatically learn representations from the data. The availability of a large amount of data and improvement in the hardware technology has accelerated the research in CNNs, and recently interesting deep CNN architectures have been reported. Several inspiring ideas to bring advancements in CNNs have been explored, such as the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. However, the significant improvement in the representational capacity of the deep CNN is achieved through architectural innovations. Notably, the ideas of exploiting spatial and channel information, depth and width of architecture, and multi-path information processing have gained substantial attention. Similarly, the idea of using a block of layers as a structural unit is also gaining popularity. This survey thus focuses on the intrinsic taxonomy present in the recently reported deep CNN architectures and, consequently, classifies the recent innovations in CNN architectures into seven different categories. These seven categories are based on spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. Additionally, the elementary understanding of CNN components, current challenges, and applications of CNN are also provided.

https://link.springer.com/content/pdf/10.1007/s10462-020-09825-6.pdf

A survey of the recent architectures of deep convolutional neural networks

In the last few years, the deep learning (DL) computing paradigm has been deemed the Gold Standard in the machine learning (ML) community. Moreover, it has gradually become the most widely used computational approach in the field of ML, thus achieving outstanding results on several complex cognitive tasks, matching or even beating those provided by human performance. One of the benefits of DL is the ability to learn massive amounts of data. The DL field has grown fast in the last few years and it has been extensively used to successfully address a wide range of traditional applications. More importantly, DL has outperformed well-known ML techniques in many domains, e.g., cybersecurity, natural language processing, bioinformatics, robotics and control, and medical information processing, among many others. Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it. Therefore, in this contribution, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of DL. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of DL and including those enhancements recently added to the field. In particular, this paper outlines the importance of DL, presents the types of DL techniques and networks. It then presents convolutional neural networks (CNNs) which the most utilized DL network type and describes the development of CNNs architectures together with their main features, e.g., starting with the AlexNet network and closing with the High-Resolution network (HR.Net). Finally, we further present the challenges and suggested solutions to help researchers understand the existing research gaps. It is followed by a list of the major DL applications. Computational tools including FPGA, GPU, and CPU are summarized along with a description of their influence on DL. The paper ends with the evolution matrix, benchmark datasets, and summary and conclusion.

/pdf/review-of-deep-learning-concepts-cnn-architectures-hpbk62qzc2.pdf

Review of deep learning: concepts, CNN architectures, challenges, applications, future directions

Although using convolutional neural networks (CNNs) as backbones achieves great successes in computer vision, this work investigates a simple backbone network useful for many dense prediction tasks without convolutions. Unlike the recently-proposed Transformer model (e.g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer~(PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks. PVT has several merits compared to prior arts. (1) Different from ViT that typically has low-resolution outputs and high computational and memory cost, PVT can be not only trained on dense partitions of the image to achieve high output resolution, which is important for dense predictions but also using a progressive shrinking pyramid to reduce computations of large feature maps. (2) PVT inherits the advantages from both CNN and Transformer, making it a unified backbone in various vision tasks without convolutions by simply replacing CNN backbones. (3) We validate PVT by conducting extensive experiments, showing that it boosts the performance of many downstream tasks, e.g., object detection, semantic, and instance segmentation. For example, with a comparable number of parameters, RetinaNet+PVT achieves 40.4 AP on the COCO dataset, surpassing RetinNet+ResNet50 (36.3 AP) by 4.1 absolute AP. We hope PVT could serve as an alternative and useful backbone for pixel-level predictions and facilitate future researches. Code is available at this https URL.

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

Object detection is one of the most important and challenging branches of computer vision, which has been widely applied in people's life, such as monitoring security, autonomous driving and so on, with the purpose of locating instances of semantic objects of a certain class. With the rapid development of deep learning algorithms for detection tasks, the performance of object detectors has been greatly improved. In order to understand the main development status of object detection pipeline thoroughly and deeply, in this survey, we analyze the methods of existing typical detection models and describe the benchmark datasets at first. Afterwards and primarily, we provide a comprehensive overview of a variety of object detection methods in a systematic manner, covering the one-stage and two-stage detectors. Moreover, we list the traditional and new applications. Some representative branches of object detection are analyzed as well. Finally, we discuss the architecture of exploiting these object detection methods to build an effective and efficient system and point out a set of development trends to better follow the state-of-the-art algorithms and further research.

A Survey of Deep Learning-Based Object Detection

The contents of this review reflect some of the shifts of emphasis that are occurring among the fields of astrophysics, nuclear physics, and elementary particle physics. Particle physics has made great advances in the unification of the fundamental forces of nature. Discussions and planning for a next big step in accelerator-colliders are presented. The technology of superconducting magnet systems as well as the fundamental physical principles of particle accelerators are discussed. Also presented are: high-resolution electronic particle detectors; nuclear physics changes such as pion interactions within nuclei; discussion of future relativistic heavy-ion colliders; the role of rotational degrees of freedom in heavy-ion collisions at low and moderate energies; hyperon beta decays; and the analysis of materials via nuclear reaction techniques. Neutrinos, their interactions and possible masses, have an important bearing on cosmology and the matter density of the universe in addition to their inherent interest in the microscopic world and this is also examined.

Annual Review of Nuclear and Particle Science

Wind power prediction using deep neural network based meta regression and transfer learning

A Survey of the Recent Architectures of Deep Convolutional Neural Networks

Display Omitted An intelligent reversible watermarking technique for the protection of medical images.The concept of companding function is exploited for reducing embedding distortion.Integer Wavelet Transform is used as an embedding domain for achieving reversibility.Histogram processing is employed to avoid underflow/overflow.GP is used to evolve models that not only make an optimal tradeoff between imperceptibility and capacity of watermark. This work presents an intelligent technique based on reversible watermarking for protecting patient and medical related information. In the proposed technique IRW-Med, the concept of companding function is exploited for reducing embedding distortion, while Integer Wavelet Transform (IWT) is used as an embedding domain for achieving reversibility. Histogram processing is employed to avoid underflow/overflow. In addition, the learning capabilities of Genetic Programming (GP) are exploited for intelligent wavelet coefficient selection. In this context, GP is used to evolve models that not only make an optimal tradeoff between imperceptibility and capacity of the watermark, but also exploit the wavelet coefficient hidden dependencies and information related to the type of sub band. The novelty of the proposed IRW-Med technique lies in its ability to generate a model that can find optimal wavelet coefficients for embedding, and also acts as a companding factor for watermark embedding. The proposed IRW-Med is thus able to embed watermark with low distortion, take out the hidden information, and also recovers the original image. The proposed IRW-Med technique is effective with respect to capacity and imperceptibility and effectiveness is demonstrated through experimental comparisons with existing techniques using standard images as well as a publically available medical image dataset.

/pdf/protection-of-medical-images-and-patient-related-information-83u527amlv.pdf

Protection of medical images and patient related information in healthcare

With the enormous increase in the use of the Internet, secure transfer of data across networks has become a challenging task. Attackers are in continuous search of getting information from network traffic, and this is the main reason that efficient intrusion detection techniques are required to identify different kinds of network attacks. In past, various supervised and semi-supervised methods have been developed for intrusion detection. Most of these methods require a large amount of data to develop an efficient intrusion detection system. In the proposed deep neural network and adaptive self-taught-based transfer learning technique, we have exploited the concept of self-taught learning to train deep neural networks for reliable network intrusion detection. In the proposed method, a pre-trained network on regression-related task is used to extract features from NSL-KDD dataset. Original features along with extracted features from the pre-trained network are then provided as an input to the sparse auto-encoder. Self-taught learning-based extracted features, when concatenated with the original features of NSL-KDD dataset, enhance the performance of the sparse auto-encoder. Performance of self-taught learning-based approach is compared against several techniques using ten independent runs in terms of accuracy, false alarm and detection rate, area under the ROC, and PR curve. It is experimentally observed that the auto-encoder trained on the combined original and extracted features is stable and offers good generalization in comparison with the sparse auto-encoder trained on original features alone.

Aqsa Saeed Qureshi

Papers

A survey of the recent architectures of deep convolutional neural networks

Wind power prediction using deep neural network based meta regression and transfer learning

A Survey of the Recent Architectures of Deep Convolutional Neural Networks

Protection of medical images and patient related information in healthcare

Intrusion detection using deep sparse auto-encoder and self-taught learning