We introduce the openSMILE feature extraction toolkit, which unites feature extraction algorithms from the speech processing and the Music Information Retrieval communities. Audio low-level descriptors such as CHROMA and CENS features, loudness, Mel-frequency cepstral coefficients, perceptual linear predictive cepstral coefficients, linear predictive coefficients, line spectral frequencies, fundamental frequency, and formant frequencies are supported. Delta regression and various statistical functionals can be applied to the low-level descriptors. openSMILE is implemented in C++ with no third-party dependencies for the core functionality. It is fast, runs on Unix and Windows platforms, and has a modular, component based architecture which makes extensions via plug-ins easy. It supports on-line incremental processing for all implemented features as well as off-line and batch processing. Numeric compatibility with future versions is ensured by means of unit tests. openSMILE can be downloaded from http://opensmile.sourceforge.net/.

http://geniiz.com/wp-content/uploads/2012/01/25-TUM-Tools-openSMILE.pdf

Opensmile: the munich versatile and fast open-source audio feature extractor

Digital processing of speech signals

Filtering and smoothing methods are used to produce an accurate estimate of the state of a time-varying system based on multiple observational inputs (data). Interest in these methods has exploded in recent years, with numerous applications emerging in fields such as navigation, aerospace engineering, telecommunications, and medicine. This compact, informal introduction for graduate students and advanced undergraduates presents the current state-of-the-art filtering and smoothing methods in a unified Bayesian framework. Readers learn what non-linear Kalman filters and particle filters are, how they are related, and their relative advantages and disadvantages. They also discover how state-of-the-art Bayesian parameter estimation methods can be combined with state-of-the-art filtering and smoothing algorithms. The book’s practical and algorithmic approach assumes only modest mathematical prerequisites. Examples include MATLAB computations, and the numerous end-of-chapter exercises include computational assignments. MATLAB/GNU Octave source code is available for download at www.cambridge.org/sarkka, promoting hands-on work with the methods.

贝叶斯滤波与平滑 (Bayesian filtering and smoothing)

We present the MIRtoolbox, an integrated set of functions written in Matlab, dedicated to the extraction of musical features from audio files The design is based on a modular framework: the different algorithms are decomposed into stages, formalized using a minimal set of elementary mechanisms, and integrating different variants proposed by alternative approaches – including new strategies we have developed –, that users can select and parametrize This paper offers an overview of the set of features, related, among others, to timbre, tonality, rhythm or form, that can be extracted with the MIRtoolbox One particular analysis is provided as an example The toolbox also includes functions for statistical analysis, segmentation and clustering Particular attention has been paid to the design of a syntax that offers both simplicity of use and transparent adaptiveness to a multiplicity of possible input types Each feature extraction method can accept as argument an audio file, or any preliminary result from intermediary stages of the chain of operations Also the same syntax can be used for analyses of single audio files, batches of files, series of audio segments, multi-channel signals, etc For that purpose, the data and methods of the toolbox are organised in an object-oriented architecture 1 MOTIVATION AND APPROACH MIRtoolbox is a Matlab toolbox dedicated to the extraction of musically-related features from audio recordings It has been designed in particular with the objective of enabling the computation of a large range of features from databases of audio files, that can be subjected to statistical analyses Few softwares have been proposed in this area One particularity of our own approach relies in the use of the Matlab computing environment, which offers good visualisation capabilities and gives access to a large variety of other toolboxes In particular, the MIRtoolbox makes use of functions available in public-domain toolboxes such as the Auditory Toolbox [6], NetLab [5] and SOMtoolbox [10] Other toolboxes, such as the Statistics toolbox or the Neural Network toolbox from MathWorks, can be directly used for further analyses of the features extracted c © 2007 Austrian Computer Society (OCG) by MIRtoolbox without having to export the data from one software to another Such computational framework, because of its general objectives, could be useful to the research community in Music Information Retrieval (MIR), but also for educational purposes For that reason, particular attention has been paid concerning the ease of use of the toolbox In particular, complex analytic processes can be designed using a very simple syntax, whose expressive power comes from the use of an object-oriented paradigm The different musical features extracted from the audio files are highly interdependent: in particular, as can be seen in figure 1, some features are based on the same initial computations In order to improve the computational efficiency, it is important to avoid redundant computations of these common components Each of these intermediary components, and the final musical features, are therefore considered as building blocks that can been freely articulated one with each other Besides, in keeping with the objective of optimal ease of use of the toolbox, each building block has been conceived in a way that it can adapt to the type of input data For instance, the computation of the MFCCs can be based on the waveform of the initial audio signal, or on the intermediary representations such as spectrum, or mel-scale spectrum (see Fig 1) Similarly, autocorrelation is computed for different range of delays depending on the type of input data (audio waveform, envelope, spectrum) This decomposition of all feature extraction algorithms into a common set of building blocks has the advantage of offering a synthetic overview of the different approaches studied in this domain of research 2 FEATURE EXTRACTION 21 Feature overview Figure 1 shows an overview of the main features implemented in the toolbox All the different processes start from the audio signal (on the left) and form a chain of operations proceeding to right Each musical feature is related to one of the musical dimensions traditionally defined in music theory Boldface characters highlight features related to pitch and tonality Bold italics indicate features related to rhythm Simple italics highlight a large set of features that can be associated to timbre and dynamics Among them, all the operators in grey italics can be Audio signal waveform Zero-crossing rate RMS energy Envelope Low Energy Rate Attack Slope Attack Time Envelope Autocorrelation Tempo Onsets

/pdf/mir-in-matlab-ii-a-toolbox-for-musical-feature-extraction-1b88seukjd.pdf

MIR in Matlab (II): A Toolbox for Musical Feature Extraction from Audio.

Everyday, people flexibly perform different categorizations of common faces, objects and scenes. Intuition and scattered evidence suggest that these categorizations require the use of different visual information from the input. However, there is no unifying method, based on the categorization performance of subjects, that can isolate the information used. To this end, we developed Bubbles, a general technique that can assign the credit of human categorization performance to specific visual information. To illustrate the technique, we applied Bubbles on three categorization tasks (gender, expressive or not and identity) on the same set of faces, with human and ideal observers to compare the features they used.

Bubbles: A Technique to Reveal the Use of Information in Recognition Tasks EOA Report

This paper provides a review of gestural control of sound synthesis in the context of the design and evaluation of digital musical instruments. It discusses research in various areas related to this field and equally focuses on four main topics: analysis of music performers' gestures, gestural capture technologies, real-time sound synthesis methods, and strategies for mapping gesture variables to sound synthesis input parameters. Finally, this approach is illustrated by presenting an application of this research to the control of digital audio effects.

http://www.dei.unipd.it/~musica/IM/P1_Wandeley_04.pdf

Gestural control of sound synthesis

jAudio is a new framework for feature extraction designed to eliminate the duplication of effort in calculating features from an audio signal. This system meets the needs of MIR researchers by providing a library of analysis algorithms that are suitable for a wide array of MIR tasks. In order to provide these features with a minimal learning curve, the system implements a GUI that makes the process of selecting desired features straight forward. A command-line interface is also provided to manipulate jAudio via scripting. Furthermore, jAudio provides a unique method of handling multidimensional features and a new mechanism for dependency handling to prevent duplicate calculations. The system takes a sequence of audio files as input. In the GUI, users select the features that they wish to have extracted—letting jAudio take care of all dependency problems—and either execute directly from the GUI or save the settings for batch processing. The output is either an ACE XML file or an ARFF file depending on the user’s preference.

/pdf/jaudio-an-feature-extraction-library-2lwvgqxfn3.pdf

jAudio: An Feature Extraction Library.

This paper discusses methods for the adaptive reconstruction of the modes of multicomponent AM-FM signals by their time-frequency (TF) representation derived from their short-time Fourier transform (STFT). The STFT of an AM-FM component or mode spreads the information relative to that mode in the TF plane around curves commonly called ridges. An alternative view is to consider a mode as a particular TF domain termed a basin of attraction. Here we discuss two new approaches to mode reconstruction. The first determines the ridge associated with a mode by considering the location where the direction of the reassignment vector sharply changes, the technique used to determine the basin of attraction being directly derived from that used for ridge extraction. A second uses the fact that the STFT of a signal is fully characterized by its zeros (and then the particular distribution of these zeros for Gaussian noise) to deduce an algorithm to compute the mode domains. For both techniques, mode reconstruction is then carried out by simply integrating the information inside these basins of attraction or domains.

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2015.0205

Adaptive multimode signal reconstruction from time-frequency representations.

We promote a clearer definition of vibrato(Seashore, 1932),based on a review of various vibrato features.We also propose a generalised vibrato effect generator that includes spectral envelope modulation,and a frequency-dependent hysteresis behaviour.We then investigate the influence of spectral envelope modulation on perceived quality with a double-blind randomized ABcomparison task.Eight participants listened to12pairs of sounds with vibrato matched for loudness.Each pair included one sound with constant average spectral envelope (identical amplitude modulation over all frequencies) and one with modulated spectral envelope (frequency dependent amplitude modulation).Participants were asked to choose which version sounded the most natural.The statistical analysis revealed a significant preference for sounds with modulated spectral envelope (p<0.001).Our results highlight the need to consider spectral envelope modulation for vibrato modelling.

Perceptual Evaluation of Vibrato Models

In this paper, we describe a multi-level approach for the extraction of instrumental gesture parameters taken from the characteristics of the signal captured by a microphone and based on the knowledge of physical mechanisms taking place on the instrument. We also explore the relationships between some features of timbre and gesture parameters, taking as a starting point for the exploration the timbre descriptors commonly used by professional musicians when they verbally describe the sounds they produce with their instrument. Finally, we present how this multi-level approach can be applied to the study of the timbre space of the classical guitar.

/pdf/indirect-acquisition-of-instrumental-gesture-based-on-signal-456jgvgorg.pdf

Philippe Depalle

Papers

Gestural control of sound synthesis

jAudio: An Feature Extraction Library.

Adaptive multimode signal reconstruction from time-frequency representations.

Perceptual Evaluation of Vibrato Models

Indirect acquisition of instrumental gesture based on signal, physical and perceptual information