In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance comparison and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded.

Knowledge Distillation: A Survey

REFUGE Challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs

As an emerging biomedical image processing technology, medical image segmentation has made great contributions to sustainable medical care. Now it has become an important research direction in the field of computer vision. With the rapid development of deep learning, medical image processing based on deep convolutional neural networks has become a research hotspot. This paper focuses on the research of medical image segmentation based on deep learning. First, the basic ideas and characteristics of medical image segmentation based on deep learning are introduced. By explaining its research status and summarizing the three main methods of medical image segmentation and their own limitations, the future development direction is expanded. Based on the discussion of different pathological tissues and organs, the specificity between them and their classic segmentation algorithms are summarized. Despite the great achievements of medical image segmentation in recent years, medical image segmentation based on deep learning has still encountered difficulties in research. For example, the segmentation accuracy is not high, the number of medical images in the data set is small and the resolution is low. The inaccurate segmentation results are unable to meet the actual clinical requirements. Aiming at the above problems, a comprehensive review of current medical image segmentation methods based on deep learning is provided to help researchers solve existing problems.

https://www.mdpi.com/2071-1050/13/3/1224/pdf

A Review of Deep-Learning-Based Medical Image Segmentation Methods

Glaucoma is a leading cause of irreversible blindness. Accurate segmentation of the optic disc (OD) and optic cup (OC) from fundus images is beneficial to glaucoma screening and diagnosis. Recently, convolutional neural networks demonstrate promising progress in the joint OD and OC segmentation. However, affected by the domain shift among different datasets, deep networks are severely hindered in generalizing across different scanners and institutions. In this paper, we present a novel patch-based output space adversarial learning framework ( ${p}$ OSAL) to jointly and robustly segment the OD and OC from different fundus image datasets. We first devise a lightweight and efficient segmentation network as a backbone. Considering the specific morphology of OD and OC, a novel morphology-aware segmentation loss is proposed to guide the network to generate accurate and smooth segmentation. Our ${p}$ OSAL framework then exploits unsupervised domain adaptation to address the domain shift challenge by encouraging the segmentation in the target domain to be similar to the source ones. Since the whole-segmentation-based adversarial loss is insufficient to drive the network to capture segmentation details, we further design the ${p}$ OSAL in a patch-based fashion to enable fine-grained discrimination on local segmentation details. We extensively evaluate our ${p}$ OSAL framework and demonstrate its effectiveness in improving the segmentation performance on three public retinal fundus image datasets, i.e., Drishti-GS, RIM-ONE-r3, and REFUGE. Furthermore, our ${p}$ OSAL framework achieved the first place in the OD and OC segmentation tasks in the MICCAI 2018 Retinal Fundus Glaucoma Challenge .

Patch-Based Output Space Adversarial Learning for Joint Optic Disc and Cup Segmentation

Applications of deep learning in fundus images: A review.

Automatic segmentation of optic disk (OD) and cup regions in fundus images is essential in deriving clinical parameters, such as, cup-to-disk ratio (CDR), to assist glaucoma diagnosis This paper presents a deep learning system using fully convolutional neural networks (FCN) to perform such segmentation, discusses various strategies on how to leverage multiple doctor annotations and prioritize pixels belonging to different regions while training the neural network Experimental evaluations on Drishti-GS dataset demonstrate that the presented method achieves comparable and superior F-score to prior work on optic disk and cup segmentation, respectively

Automatic Optic Disk and Cup Segmentation of Fundus Images Using Deep Learning

We present DeepInversion for Object Detection (DIODE) to enable data-free knowledge distillation for neural networks trained on the object detection task. From a data-free perspective, DIODE synthesizes images given only an off-the-shelf pre-trained detection network and without any prior domain knowledge, generator network, or pre-computed activations. DIODE relies on two key components—first, an extensive set of differentiable augmentations to improve image fidelity and distillation effectiveness. Second, a novel automated bounding box and category sampling scheme for image synthesis enabling generating a large number of images with a diverse set of spatial and category objects. The resulting images enable data-free knowledge distillation from a teacher to a student detector, initialized from scratch.In an extensive set of experiments, we demonstrate that DIODE’s ability to match the original training distribution consistently enables more effective knowledge distillation than out-of-distribution proxy datasets, which unavoidably occur in a data-free setup given the absence of the original domain knowledge.

/pdf/data-free-knowledge-distillation-for-object-detection-2k7rsynahp.pdf

Data-free Knowledge Distillation for Object Detection

An overview of SnO2 based Z scheme heterojuctions: Fabrication, mechanism and advanced photocatalytic applications

Bi-rich BixOyBrz-based photocatalysts for energy conversion and environmental remediation: A review

Free-space optical communication is utilized for transmitting data between the source and destination using light traveling through free space. In this paper, the modulation technique applied to the propagating light is linear polarization shift keying. An FSO system is simulated in Optisystem software using the aforementioned modulation technique under different weather conditions. The performance analysis of the system is carried out by examining minimum BER values and the corresponding transmission distance achieved with different specific attenuations.

Akshay Chawla

Papers

Automatic Optic Disk and Cup Segmentation of Fundus Images Using Deep Learning

Data-free Knowledge Distillation for Object Detection

An overview of SnO2 based Z scheme heterojuctions: Fabrication, mechanism and advanced photocatalytic applications

Bi-rich BixOyBrz-based photocatalysts for energy conversion and environmental remediation: A review

Simulation research of free-space optical communication based on linear polarization shift keying modulation