The paper considers the task of recognizing environmental sounds for the understanding of a scene or context surrounding an audio sensor. A variety of features have been proposed for audio recognition, including the popular Mel-frequency cepstral coefficients (MFCCs) which describe the audio spectral shape. Environmental sounds, such as chirpings of insects and sounds of rain which are typically noise-like with a broad flat spectrum, may include strong temporal domain signatures. However, only few temporal-domain features have been developed to characterize such diverse audio signals previously. Here, we perform an empirical feature analysis for audio environment characterization and propose to use the matching pursuit (MP) algorithm to obtain effective time-frequency features. The MP-based method utilizes a dictionary of atoms for feature selection, resulting in a flexible, intuitive and physically interpretable set of features. The MP-based feature is adopted to supplement the MFCC features to yield higher recognition accuracy for environmental sounds. Extensive experiments are conducted to demonstrate the effectiveness of these joint features for unstructured environmental sound classification, including listening tests to study human recognition capabilities. Our recognition system has shown to produce comparable performance as human listeners.

/pdf/environmental-sound-recognition-with-time-frequency-audio-5f5opmb7a5.pdf

Environmental Sound Recognition With Time–Frequency Audio Features

Music information retrieval (MIR) is an emerging research area that receives growing attention from both the research community and music industry. It addresses the problem of querying and retrieving certain types of music from large music data set. Classification is a fundamental problem in MIR. Many tasks in MIR can be naturally cast in a classification setting, such as genre classification, mood classification, artist recognition, instrument recognition, etc. Music annotation, a new research area in MIR that has attracted much attention in recent years, is also a classification problem in the general sense. Due to the importance of music classification in MIR research, rapid development of new methods, and lack of review papers on recent progress of the field, we provide a comprehensive review on audio-based classification in this paper and systematically summarize the state-of-the-art techniques for music classification. Specifically, we have stressed the difference in the features and the types of classifiers used for different classification tasks. This survey emphasizes on recent development of the techniques and discusses several open issues for future research.

A Survey of Audio-Based Music Classification and Annotation

We present an exhaustive review of research on automatic classification of sounds from musical instruments. Two different but complementary approaches are examined, the perceptual approach and the taxonomic approach. The former is targeted to derive perceptual similarity functions in order to use them for timbre clustering and for searching and retrieving sounds by timbral similarity. The latter is targeted to derive indexes for labeling sounds after culture- or user-biased taxonomies. We review the relevant features that have been used in the two areas and then we present and discuss different techniques for similarity-based clustering of sounds and for classification into pre-defined instrumental categories.

/pdf/automatic-classification-of-musical-instrument-sounds-3vn5i2ho3f.pdf

Automatic Classification of Musical Instrument Sounds

Today, a large number of audio features exists in audio retrieval for different purposes, such as automatic speech recognition, music information retrieval, audio segmentation, and environmental sound retrieval. The goal of this chapter is to review latest research in the context of audio feature extraction and to give an application-independent overview of the most important existing techniques. We survey state-of-the-art features from various domains and propose a novel taxonomy for the organization of audio features. Additionally, we identify the building blocks of audio features and propose a scheme that allows for the description of arbitrary features. We present an extensive literature survey and provide more than 200 references to relevant high-quality publications.

Features for Content-Based Audio Retrieval

With the proliferation of digital audio distribution over digital media, audio content analysis is fast becoming a requirement for designers of intelligent signal-adaptive audio processing systems. Written by a well-known expert in the field, this book provides quick access to different analysis algorithms and allows comparison between different approaches to the same task, making it useful for newcomers to audio signal processing and industry experts alike. A review of relevant fundamentals in audio signal processing, psychoacoustics, and music theory, as well as downloadable MATLAB files are also included. Please visit the companion website: www.AudioContentAnalysis.org

An Introduction to Audio Content Analysis: Applications in Signal Processing and Music Informatics

A set of features is evaluated for recognition of musical instruments out of monophonic musical signals. Aiming to achieve a compact representation, the adopted features regard only spectral characteristics of sound and are limited in number. On top of these descriptors, various classification methods are implemented and tested. Over a dataset of 1007 tones from 27 musical instruments, support vectormachines and quadratic discriminant analysis show comparable results with success rates close to 70% of successful classifications. Canonical discriminant analysis never had momentous results, while nearest neighbours performed on average among the employed classifiers. Strings have been the most misclassified instrument family, while very satisfactory results have been obtained with brass and woodwinds. The most relevant features are demonstrated to be the inharmonicity, the spectral centroid, and the energy contained in the first partial.

/pdf/musical-instrument-timbres-classification-with-spectral-xgvth70gbo.pdf

Musical instrument timbres classification with spectral features

A set of features is evaluated for musical instrument recognition out of monophonic musical signals. Aiming to achieve a compact representation, the adopted features regard only spectral characteristics of sound and are limited in number. On top of these descriptors, various classification methods are implemented and tested. Over a dataset of 1007 tones from 27 musical instruments and without employing any hierarchical structure, quadratic discriminant analysis shows the lowest error rate (7.19% for the individual instrument and 3.13% for instrument families), outperforming all the other classification methods (canonical discriminant analysis, support vector machines, nearest neighbours). The most relevant features are demonstrated to be the inharmonicity, the spectral centroid and the energy contained in the first partial.

/pdf/musical-instrument-timbres-classification-with-spectral-3t9xau09h9.pdf

70 Computer Music Journal There are six aspects into which music information can be divided: general, structural, music logic, notation, performance, and audio. We call these aspects “layers,” because each represents a different level of abstraction of the music information. However, these layers can be viewed as a single symbolic music information (SMI) entity. The purpose of SMI is to relate all existing representations in the notation, performance, and audio layers using the music logic and structural layers. In this article, we present an Extensible Markup Language (XML) instance intended for the integration of the different aspects of music representation. It tries to bring together ideas and concepts developed in the past. Because this format is still under development by the IEEE-SA Working Group on Music Application of XML (IEEE-SA MAX WG; see www.lim.dico.unimi.it/IEEE/XML.html), and because in an article it is impossible to give a detailed description of a whole format, we present only the main concepts that most probably will be affected only by minor changes by future developments of the format itself. These concepts are Layered Symbolic Music Information (Layered SMI) and the Spine structure. Owing to these two concepts, we consider the contribution of our format to be a more complete integration of previous concepts and formats in a framework usable by diverse music applications, especially those that are based on different concurrent music layers (for example, the automatic synchronization of audio, MIDI, and score). Layered SMI is important owing to the manifold nature of music representation. Downie (2003) explains this concept in a very meaningful way. According to Downie, music is composed of seven facets: pitch, temporal, harmonic, timbral, editorial, textual, and bibliographic. Moreover, each facet can interact with the others, increasing the complexity of the representational challenge. In addition, music representation also poses multi-representational, multi-cultural, multi-experiential, and multidisciplinary challenges. Among the several existing formats for music representation, there are only a small number of them that can be said to be a de facto standard. If we compare these few music formats, we can observe that each of them is designed to represent mainly a particular aspect or only a limited number of aspects of music information. We can subdivide these formats in four big clusters: audio, sub-symbolic, notational, and compositional, similar to the domains of Standard Music Description Language (SMDL; Sloan 1993; see also ftp.ornl.gov/pub/sgml/WG8/SMDL/ 10743.ps). Audio formats encode signal information, that is, only the purely aural aspect (the “Gestural” domain of SMDL). Sub-symbolic formats like MIDI (Musical Instrument Digital Interface; MIDI Manufacturers Association 2001) or Csound (Boulanger 1999) encode information about how to produce or reproduce music electronically (the “Logical” domain of SMDL). Many music notation file formats have been developed by the different music editing software producers. Some of them are rich enough, like NIFF (Notation Interchange File Format; see www.musique.umontreal.ca/personnel/Belkin/NIFF .doc.html) or Enigma, to generate a MIDI rendering of notational content (the “Visual” domain of SMDL). However, new software composition tools (e.g., Haus and Sametti 1991; Assayag et al. 1999) need to formalize and exchange information and structures that are not represented in these formats (the “Analytical” domain of SMDL). Moreover, the emerging possibility of wide dissemination of music via the Internet increases the urgency of dealing with the problems of cataloging and protecting intellectual rights of these items. In the past, many researchers have addressed the problem of representing different aspects of music. We think that the most conceptually meaningful of these attempts was SMDL (Sloan 1993; see also ftp.ornl.gov/pub/sgml/WG8/SMDL/10743.ps). Although it is not explicit in many existing representations, there is an intrinsic space–time relationship A Multi-Layered, TimeBased Music Description Approach Based on XML

A Multi-Layered, TimeBased Music Description Approach Based on XML

As the dimension and number of digital music archives grow, the problem of storing and accessing multimedia data is no longer confined to the database area. Specific approaches for music information retrieval are necessary to establish a connection between textual and content-based metadata. This article addresses such issues with the intent of surveying our perspective on music information retrieval. In particular, we stress the use of symbolic information as a central element in a complex musical environment. Musical themes, harmonies, and styles are automatically extracted from electronic music scores and employed as access keys to data. The data-base schema is extended to handle audio recordings. A score/audio matching module provides a temporal relationship between a music performance and the score played. Besides standard free-text search capabilities, three levels of retrieval strategies are employed. Moreover, the introduction of a hierarchy of input modalities assures meeting the needs and matching the expertise of a wide group of users. Singing, playing, and notating melodic excerpts is combined with more advanced musicological queries, such as querying by a sequence of chords. Finally, we present some experimental results and our future research directions.

/pdf/a-score-driven-approach-to-music-information-retrieval-d46n97ody9.pdf

A score-driven approach to music information retrieval

The representation of Symbolic Music Information is the fundamental element of Music Information Processing Among the several approaches developed in history the markupped one seems to be the most promising Starting from SMDL, some XML proposals are briefly analyzed We present our approach of layered representation of Symbolic Music Information based on the space-time organization of events Our model splits music information in layers of representation and takes the symbolic one as the gravity center Moreover, the symbolic layer contains a space-time structure by means of which all other layers are brought together Then we discuss other XML standard that can be usefully related to the XML representation of music information, such as security and vector graphics standards Lastly, we deal with some open problems in the field of music processing that might be of interest for our view of symbolic representation

https://xml.coverpages.org/MLongariMAX2002.pdf

Maurizio Longari

Papers

Musical instrument timbres classification with spectral features

Musical instrument timbres classification with spectral features

A Multi-Layered, TimeBased Music Description Approach Based on XML

A score-driven approach to music information retrieval

Towards a Symbolic/Time-Based Music language based on XML