In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Very Deep Convolutional Networks for Large-Scale Image Recognition

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.

/pdf/batch-normalization-accelerating-deep-network-training-by-3dkqye6twt.pdf

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Measuring vital signs plays a key role in both patient care and wellness, but can be challenging outside of medical settings due to the lack of specialized equipment. In this study, we prospectively evaluated smartphone camera-based techniques for measuring heart rate (HR) and respiratory rate (RR) for consumer wellness use. HR was measured by placing the finger over the rear-facing camera, while RR was measured via a video of the participants sitting still in front of the front-facing camera. In the HR study of 95 participants (with a protocol that included both measurements at rest and post exercise), the mean absolute percent error (MAPE) {+/-} standard deviation of the measurement was 1.6% {+/-} 4.3%, which was significantly lower than the pre-specified goal of 5%. No significant differences in the MAPE were present across colorimeter-measured skin-tone subgroups: 1.8% {+/-} 4.5% for very light to intermediate, 1.3% {+/-} 3.3% for tan and brown, and 1.8% {+/-} 4.9% for dark. In the RR study of 50 participants, the mean absolute error (MAE) was 0.78 {+/-} 0.61 breaths/min, which was significantly lower than the pre-specified goal of 3 breath/min. The MAE was low in both healthy participants (0.70 {+/-} 0.67 breaths/min), and participants with chronic respiratory conditions (0.80 {+/-} 0.60 breaths/min). Our results validate that smartphone camera-based techniques can accurately measure HR and RR across a range of pre-defined subgroups.

https://www.medrxiv.org/content/medrxiv/early/2021/03/12/2021.03.08.21252408.full.pdf

Prospective Validation of Smartphone-based Heart Rate and Respiratory Rate Measurement Algorithms

시스템은 하나 이상의 컴퓨터 및 상기 컴퓨터에 의해 실행될 때 상기 컴퓨터로 하여금 예측 출력을 생성하기 위해 입력을 처리하기 위한 복합(combined) 기계 학습 모델을 구현하게 하는 명령들을 저장하는 하나 이상의 저장 디바이스를 포함한다. 상기 결합 모델은, 심층 모델 출력을 생성하기 위해 특징들을 처리하는 심층 기계 학습 모델; 광역 모델 출력을 생성하기 위해 상기 특징들을 처리하는 광역 기계 학습 모델; 및 예측 출력을 생성하기 위해 상기 심층 기계 학습 모델에 의해 생성된 심층 모델 출력과 상기 광역 기계 학습 모델에 의해 생성된 광역 모델 출력을 처리하는 결합 계층을 포함한다. 상기 심층 기계 학습 모델 및 상기 광역 기계 학습 모델을 모두 포함함으로써, 상기 결합 기계 학습 도델은 암기 및 일반화의 이점을 모두 얻을 수 있고 그에 따라 입력 특징들로부터 출력을 예측할 때 더 잘 수행할 수 있다.

광역 및 심층 기계 학습 모델들

20. Dissociating the functional properties of the medial and lateral anterior prefrontal cortex

Key Points Question Can a prognostic machine learning–derived histopathologic feature be learned and validated by pathologists? Findings In this prognostic study, 2 pathologists were able to learn a machine learning–derived histopathologic feature and validate its prognostic value for survival among patients with colon cancer. Meaning These findings suggest that computationally identified histopathologic features can provide prognostic value for colon cancer, with the potential for integration into pathology practice.

Greg S. Corrado

Papers

Prospective Validation of Smartphone-based Heart Rate and Respiratory Rate Measurement Algorithms

광역 및 심층 기계 학습 모델들

20. Dissociating the functional properties of the medial and lateral anterior prefrontal cortex

Pathologist Validation of a Machine Learning–Derived Feature for Colon Cancer Risk Stratification

Choosing the greater of two goods: neural correlates of value based decision making