Audio event recognition, the human-like ability to identify and relate sounds from audio, is a nascent problem in machine perception. Comparable problems such as object detection in images have reaped enormous benefits from comprehensive datasets - principally ImageNet. This paper describes the creation of Audio Set, a large-scale dataset of manually-annotated audio events that endeavors to bridge the gap in data availability between image and audio research. Using a carefully structured hierarchical ontology of 632 audio classes guided by the literature and manual curation, we collect data from human labelers to probe the presence of specific audio classes in 10 second segments of YouTube videos. Segments are proposed for labeling using searches based on metadata, context (e.g., links), and content analysis. The result is a dataset of unprecedented breadth and size that will, we hope, substantially stimulate the development of high-performance audio event recognizers.

Audio Set: An ontology and human-labeled dataset for audio events

1. Interaction. The interactions learners have with each other build interpersonal skills, such as listening, politely interrupting, expressing ideas, raising questions, disagreeing, paraphrasing, negotiating, and asking for help. 2. Interdependence. Learners must depend on one another to accomplish a common objective. Each group member has specific tasks to complete, and successful completion of each member’s tasks results in attaining the overall group objective.

将“Cooperative Learning”融入课堂——浅谈英语素质教育

Visual inspection of histopathology slides is one of the main methods used by pathologists to assess the stage, type and subtype of lung tumors. Adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC) are the most prevalent subtypes of lung cancer, and their distinction requires visual inspection by an experienced pathologist. In this study, we trained a deep convolutional neural network (inception v3) on whole-slide images obtained from The Cancer Genome Atlas to accurately and automatically classify them into LUAD, LUSC or normal lung tissue. The performance of our method is comparable to that of pathologists, with an average area under the curve (AUC) of 0.97. Our model was validated on independent datasets of frozen tissues, formalin-fixed paraffin-embedded tissues and biopsies. Furthermore, we trained the network to predict the ten most commonly mutated genes in LUAD. We found that six of them—STK11, EGFR, FAT1, SETBP1, KRAS and TP53—can be predicted from pathology images, with AUCs from 0.733 to 0.856 as measured on a held-out population. These findings suggest that deep-learning models can assist pathologists in the detection of cancer subtype or gene mutations. Our approach can be applied to any cancer type, and the code is available at https://github.com/ncoudray/DeepPATH
 .

/pdf/classification-and-mutation-prediction-from-non-small-cell-9iqvajuj7y.pdf

Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning.

Recent work has shown that data augmentation has the potential to significantly improve the generalization of deep learning models. Recently, automated augmentation strategies have led to state-of-the-art results in image classification and object detection. While these strategies were optimized for improving validation accuracy, they also led to state-of-the-art results in semi-supervised learning and improved robustness to common corruptions of images. An obstacle to a large-scale adoption of these methods is a separate search phase which increases the training complexity and may substantially increase the computational cost. Additionally, due to the separate search phase, these approaches are unable to adjust the regularization strength based on model or dataset size. Automated augmentation policies are often found by training small models on small datasets and subsequently applied to train larger models. In this work, we remove both of these obstacles. RandAugment has a significantly reduced search space which allows it to be trained on the target task with no need for a separate proxy task. Furthermore, due to the parameterization, the regularization strength may be tailored to different model and dataset sizes. RandAugment can be used uniformly across different tasks and datasets and works out of the box, matching or surpassing all previous automated augmentation approaches on CIFAR-10/100, SVHN, and ImageNet. On the ImageNet dataset we achieve 85.0% accuracy, a 0.6% increase over the previous state-of-the-art and 1.0% increase over baseline augmentation. On object detection, RandAugment leads to 1.0-1.3% improvement over baseline augmentation, and is within 0.3% mAP of AutoAugment on COCO. Finally, due to its interpretable hyperparameter, RandAugment may be used to investigate the role of data augmentation with varying model and dataset size. Code is available online.

/pdf/randaugment-practical-automated-data-augmentation-with-a-3vjdmt07lf.pdf

RandAugment: Practical Automated Data Augmentation with a Reduced Search Space

Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations. However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations. We propose to leverage periodic activation functions for implicit neural representations and demonstrate that these networks, dubbed sinusoidal representation networks or Sirens, are ideally suited for representing complex natural signals and their derivatives. We analyze Siren activation statistics to propose a principled initialization scheme and demonstrate the representation of images, wavefields, video, sound, and their derivatives. Further, we show how Sirens can be leveraged to solve challenging boundary value problems, such as particular Eikonal equations (yielding signed distance functions), the Poisson equation, and the Helmholtz and wave equations. Lastly, we combine Sirens with hypernetworks to learn priors over the space of Siren functions.

Implicit Neural Representations with Periodic Activation Functions

A content system accessing an audio stream. The content system inputs segments of the audio stream into a speech classifier for classification, the speech classifier generating, for the segments of the audio stream, raw scores representing likelihoods that the respective segment of the audio stream includes an occurrence of a speech sound. The content system generates binary scores for the audio stream based on the set of raw scores, each binary score generated based on an aggregation of raw scores from consecutive series of the segments of the audio stream. The content system generates one or more timing windows for the speech sounds in the audio stream based on the binary scores, each timing window indicating an estimate of a beginning and ending timestamps of one or more speech sounds in the audio stream.

Automatic determination of timing windows for speech captions in an audio stream

This poster describes a controlled study that examines how individual motivation orientations affect computer-supported collaborative learning (CSCL) activities with an adaptive thermodynamics dialogue tutor. Our results show that motivation orientation has an effect on perceptions of one's learning partner, and possibly one's self-confidence.

Motivation and Collaboration On-Line

A content server accessing an audio stream, and inputs portions of the audio stream into one or more non-speech classifiers for classification, the non-speech classifiers generating, for portions of the audio stream, a set of raw scores representing likelihoods that the respective portion of the audio stream includes an occurrence of a particular class of non-speech sounds associated with each of the non-speech classifiers. The content server generates binary scores for the sets of raw scores, the binary scores generated based on a smoothing of a respective set of raw scores. The content server applies a set of non-speech captions to portions of the audio stream in time, each of the sets of non-speech captions based on a different one of the set binary scores of the corresponding portion of the audio stream.

Automatic smoothed captioning of non-speech sounds from audio

Automatic deduction of semantic labels for audiovisual data requires awareness of context, which in turn requires processing sequences of audiovisual scenes or events. The representation of such sequences is important for semantic analysis tasks. Whereas, conventionally, sequences of specific short-duration event labels, often hand-annotated for learning detectors or classifiers, have been used, we propose a new technique for audiovisual event categorization in this paper, wherein units of audio and image scenes are discovered automatically from data in a likelihood-maximization process. We show how these units for audio and video, respectively called AUDs and VIDs, can be used to learn the salient characteristics of broad-category semantic labels without requiring explicit error recovery measures. Experiments with the MED-11 dataset show that AUDs and VIDs are better able to retrieve semantic categories from mixed-content data as compared to vector quantization-based systems and systems that use library-based descriptors. Index: multimedia analysis, semantic labels, unsupervised lexicon learning, audiovisual data retrieval

Exploiting Temporal Sequence Structure for Semantic Analysis of Multimedia.

In this type-II demo, we introduce SIDE (the Summarization Integrated Development Environment), an infrastructure that facilitates construction of summaries tailored to the needs of the user. It aims to address the issue that there is no such thing as the perfect summary for all purposes. Rather, the quality of a summary is subjective, task dependent, and possibly specific to a user. The SIDE framework allows users flexibility in determining what they find more useful in a summary, both in terms of structure and content. As an educational tool, it has been successfully user tested by a class of 21 students in a graduate course on Summarization and Personal Information Management.

/pdf/side-the-summarization-integrated-development-environment-cjknfgee3s.pdf

Sourish Chaudhuri

Papers

Automatic determination of timing windows for speech captions in an audio stream

Motivation and Collaboration On-Line

Automatic smoothed captioning of non-speech sounds from audio

Exploiting Temporal Sequence Structure for Semantic Analysis of Multimedia.

SIDE: The Summarization Integrated Development Environment