Acoustic data provide scientific and engineering insights in fields ranging from biology and communications to ocean and Earth science. We survey the recent advances and transformative potential of machine learning (ML), including deep learning, in the field of acoustics. ML is a broad family of techniques, which are often based in statistics, for automatically detecting and utilizing patterns in data. Relative to conventional acoustics and signal processing, ML is data-driven. Given sufficient training data, ML can discover complex relationships between features and desired labels or actions, or between features themselves. With large volumes of training data, ML can discover models describing complex acoustic phenomena such as human speech and reverberation. ML in acoustics is rapidly developing with compelling results and significant future promise. We first introduce ML, then highlight ML developments in four acoustics research areas: source localization in speech processing, source localization in ocean acoustics, bioacoustics, and environmental sounds in everyday scenes.

Machine learning in acoustics: theory and applications

Deep neural networks have advanced the field of detection and classification and allowed for effective identification of signals in challenging data sets. Numerous time-critical conservation needs may benefit from these methods. We developed and empirically studied a variety of deep neural networks to detect the vocalizations of endangered North Atlantic right whales (Eubalaena glacialis). We compared the performance of these deep architectures to that of traditional detection algorithms for the primary vocalization produced by this species, the upcall. We show that deep-learning architectures are capable of producing false-positive rates that are orders of magnitude lower than alternative algorithms while substantially increasing the ability to detect calls. We demonstrate that a deep neural network trained with recordings from a single geographic region recorded over a span of days is capable of generalizing well to data from multiple years and across the species' range, and that the low false positives make the output of the algorithm amenable to quality control for verification. The deep neural networks we developed are relatively easy to implement with existing software, and may provide new insights applicable to the conservation of endangered species.

/pdf/deep-neural-networks-for-automated-detection-of-marine-56h6rkwrbd.pdf

Deep neural networks for automated detection of marine mammal species

Passive acoustics provides a powerful tool for monitoring the endangered North Atlantic right whale (Eubalaena glacialis), but robust detection algorithms are needed to handle diverse and variable acoustic conditions and differences in recording techniques and equipment. This paper investigates the potential of deep neural networks (DNNs) for addressing this need. ResNet, an architecture commonly used for image recognition, was trained to recognize the time-frequency representation of the characteristic North Atlantic right whale upcall. The network was trained on several thousand examples recorded at various locations in the Gulf of St. Lawrence in 2018 and 2019, using different equipment and deployment techniques. Used as a detection algorithm on fifty 30-min recordings from the years 2015-2017 containing over one thousand upcalls, the network achieved recalls up to 80% while maintaining a precision of 90%. Importantly, the performance of the network improved as more variance was introduced into the training dataset, whereas the opposite trend was observed using a conventional linear discriminant analysis approach. This study demonstrates that DNNs can be trained to identify North Atlantic right whale upcalls under diverse and variable conditions with a performance that compares favorably to that of existing algorithms.

Performance of a deep neural network at detecting North Atlantic right whale upcalls.

Cetaceans have elicited the attention of researchers in recent decades due to their importance to the ecosystem and their economic values. They use sound for communication, echolocation and other social activities. Their sounds are highly non-stationary, transitory and range from short to long sounds. Passive acoustic monitoring (PAM) is a popular method used for monitoring cetaceans in their ecosystems. The volumes of data accumulated using PAM are usually big, so they are difficult to analyze using manual inspection. Therefore different techniques with mixed outcomes have been developed for the automatic detection and classification of signals of different cetacean species. So far, no single technique developed is perfect to detect and classify the vocalizations of over 82 known species due to variability in time-frequency, difference in the amplitude among species and within species' vocal repertoire, physical environment, among others. The accuracy of any detector or classifier depends on the technique adopted as well as the nature of the signal to be analyzed. In this article, we review the existing techniques for the automatic detection and classification of cetacean vocalizations. We categorize the surveyed techniques, while emphasizing the advantages and disadvantages of these techniques. The article suggests possible research directions that can improve existing detection and classification techniques. In addition, the article recommends other suitable techniques that can be used to analyze non-linear and non-stationary signals such as the cetaceans' signals. Several research have been dedicated to this topic, however, there is no review of these past results that gives a quick overview in the area of cetacean detection and classification. This review will help researchers and practitioners in the field to make insightful decisions based on their requirements.

/pdf/review-of-automatic-detection-and-classification-techniques-3o7cugfdlz.pdf

Review of Automatic Detection and Classification Techniques for Cetacean Vocalization

Detection and classification of marine mammal sounds using AlexNet with transfer learning

Research into automated systems for detecting and classifying marine mammals in acoustic recordings is expanding internationally due to the necessity to analyze large collections of data for conservation purposes. In this work, we present a Convolutional Neural Network that is capable of classifying the vocalizations of three species of whales, non-biological sources of noise, and a fifth class pertaining to ambient noise. In this way, the classifier is capable of detecting the presence and absence of whale vocalizations in an acoustic recording. Through transfer learning, we show that the classifier is capable of learning high-level representations and can generalize to additional species. We also propose a novel representation of acoustic signals that builds upon the commonly used spectrogram representation by way of interpolating and stacking multiple spectrograms produced using different Short-time Fourier Transform (STFT) parameters. The proposed representation is particularly effective for the task of marine mammal species classification where the acoustic events we are attempting to classify are sensitive to the parameters of the STFT.

Marine Mammal Species Classification Using Convolutional Neural Networks and a Novel Acoustic Representation

For an expensive to evaluate computer
simulator, even the estimate of the overall surface can be a challenging
problem. In this paper, we focus on the estimation of the inverse solution,
i.e., to find the set(s) of input combinations of the simulator that generates
a pre-determined simulator output. Ranjan et al. [1] proposed an expected
improvement criterion under a sequential design framework for the inverse
problem with a scalar valued simulator. In this paper, we focus on the inverse
problem for a time-series valued simulator. We have used a few simulated and
two real examples for performance comparison.

/pdf/inverse-problem-for-a-time-series-valued-computer-simulator-5cejobtc52.pdf

Inverse Problem for a Time-Series Valued Computer Simulator via Scalarization

Marine Mammal Species Classification using Convolutional Neural Networks and a Novel Acoustic Representation

We propose a straightforward and cost-effective method to perform diffuse soundfield measurements for calibrating the magnitude response of a microphone array. Typically, such calibration is performed in a diffuse soundfield created in reverberation chambers, an expensive and time-consuming process. A method is proposed for obtaining diffuse field measurements in untreated environments. First, a closed-form expression for the spatial correlation of a wideband signal in a diffuse field is derived. Next, we describe a practical procedure for obtaining the diffuse field response of a microphone array in the presence of a non-diffuse soundfield by the introduction of random perturbations in the microphone location. Experimental spatial correlation data obtained is compared with the theoretical model, confirming that it is possible to obtain diffuse field measurements in untreated environments with relatively few loudspeakers. A 30 second test signal played from 4–8 loudspeakers is shown to be sufficient in obtaining a diffuse field measurement using the proposed method. An Eigenmike® is then successfully calibrated at two different geographical locations.

A Novel Method for Obtaining Diffuse Field Measurements for Microphone Calibration

Research into automated systems for detecting marine mammal vocalizations within acoustic recordings is expanding internationally due to the necessity to analyze large collections of data collected for passive acoustic monitoring. Recent work towards the development of such systems using Convolutional Neural Networks (CNNs) shows great promise and these systems are capable of generalizing to additional species without having to re-train the entire network [1]. However, to the best of our knowledge, the current deep learning implementations do not perform what we refer to as true detection. Instead these systems are simply capable of determining the presence or absence of a vocalization within a spectrogram. In this work we present a CNN trained on spectrograms containing labelled bounding boxes around low-frequency vocalizations produced by several species of marine mammals. In this way, the CNN can precisely detect vocalizations in terms of both time and frequency, while maintaining the advantage of being generalizable to additional species. [1] M. Thomas, B. Martin, K. Kowarski, B. Gaudet, and S. Stan, "Marine mammal species classification using convolutional neural networks and a novel acoustic representation," in ECML PKDD 2019 (Springer, Cham, 2019).Research into automated systems for detecting marine mammal vocalizations within acoustic recordings is expanding internationally due to the necessity to analyze large collections of data collected for passive acoustic monitoring. Recent work towards the development of such systems using Convolutional Neural Networks (CNNs) shows great promise and these systems are capable of generalizing to additional species without having to re-train the entire network [1]. However, to the best of our knowledge, the current deep learning implementations do not perform what we refer to as true detection. Instead these systems are simply capable of determining the presence or absence of a vocalization within a spectrogram. In this work we present a CNN trained on spectrograms containing labelled bounding boxes around low-frequency vocalizations produced by several species of marine mammals. In this way, the CNN can precisely detect vocalizations in terms of both time and frequency, while maintaining the advantage of bein...

Mark Thomas

Papers

Marine Mammal Species Classification Using Convolutional Neural Networks and a Novel Acoustic Representation

Inverse Problem for a Time-Series Valued Computer Simulator via Scalarization

Marine Mammal Species Classification using Convolutional Neural Networks and a Novel Acoustic Representation

A Novel Method for Obtaining Diffuse Field Measurements for Microphone Calibration

An end-to-end approach for true detection of low frequency marine mammal vocalizations