Showing papers presented at "ACM international conference on Digital libraries in 2018"

Extended playing techniques: the next milestone in musical instrument recognition

Abstract: The expressive variability in producing a musical note conveys information essential to the modeling of orchestration and style. As such, it plays a crucial role in computer-assisted browsing of massive digital music corpora. Yet, although the automatic recognition of a musical instrument from the recording of a single "ordinary" note is considered a solved problem, automatic identification of instrumental playing technique (IPT) remains largely underdeveloped. We benchmark machine listening systems for query-by-example browsing among 143 extended IPTs for 16 instruments, amounting to 469 triplets of instrument, mute, and technique. We identify and discuss three necessary conditions for significantly outperforming the traditional mel-frequency cepstral coefficient (MFCC) baseline: the addition of second-order scattering coefficients to account for amplitude modulation, the incorporation of long-range temporal dependencies, and metric learning using large-margin nearest neighbors (LMNN) to reduce intra-class variability. Evaluating on the Studio On Line (SOL) dataset, we obtain a precision at rank 5 of 99.7% for instrument recognition (baseline at 89.0%) and of 61.0% for IPT recognition (baseline at 44.5%). We interpret this gain through a qualitative assessment of practical usability and visualization using nonlinear dimensionality reduction.

On large-scale genre classification in symbolically encoded music by automatic identification of repeating patterns

Abstract: The importance of repetitions in music is well-known. In this paper, we study music repetitions in the context of effective and efficient automatic genre classification in large-scale music-databases. We aim at enhancing the access and organization of pieces of music in Digital Libraries by allowing automatic categorization of entire collections by considering only their musical content. We handover to the public a set of genre-specific patterns to support research in musicology. The patterns can be used, for instance, to explore and analyze the relations between musical genres.There are many existing algorithms that could be used to identify and extract repeating patterns in symbolically encoded music. In our case, the extracted patterns are used as representations of the pieces of music on the underlying corpus and, consecutively, to train and evaluate a classifier to automatically identify genres. In this paper, we apply two very fast algorithms enabling us to experiment on large and diverse corpora. Thus, we are able to find patterns with strong discrimination power that can be used in various applications. We carried out experiments on a corpus containing over 40,000 MIDI files annotated with at least one genre. The experiments suggest that our approach is scalable and capable of dealing with real-world-size music collections.

MuRET: a music recognition, encoding, and transcription tool

Abstract: The transcription process from early and modern notation manuscripts to a structured digital encoding has been traditionally performed following a fully manual workflow. At most it has received some technological support in particular stages, like optical music recognition (OMR) of the source images, or transcription to modern notation with music edition applications. Currently, there is no mature and stable enough solution for the OMR problem, and the most used music editors do not support early notations, such as the mensural one. In this work, a new tool called MUsic Recognition, Encoding, and Transcription (MuRET) is introduced, which covers all transcription phases, from the manuscript source to the encoded digital content. MuRET is designed as a technology-focused research tool, allowing different processing approaches to be used, and producing both the expected transcribed contents in standard encodings and data for the study of the transcription process itself.

No longer 'somewhat arbitrary': calculating salience in GTTM-style reduction

Abstract: Following earlier work on the formalisation of Lerdahl and Jack-endoff's Generative Theory of Tonal Music (GTTM), we present a measure of the salience of events in a reduction tree, based on calculations relating the duration of time-spans to the structure of the tree. This allows for the proper graphical rendition of a tree on the basis of its time-spans and topology alone. It also has the potential to contribute to the development of sophisticated digital library systems able to operate on music in a musically intelligent manner. We present results of an empirical study of branch heights in the figures in GTTM which shows that salience calculated according to our proposals correlates better with branch height than alternatives. We also discuss the possible musical significance of this measure of salience. Finally we compare some results using salience in the calculation of melodic similarity on the basis of reduction trees to earlier results using time-span. While the correlation between these measures and human ratings of the similarity of the melodies is poor, using salience shows a definite improvement. Overall, the results suggest that the proposed definition of salience gives a potentially useful measure of an event's importance in a musical structure.

How current optical music recognition systems are becoming useful for digital libraries

Abstract: Optical Music Recognition (OMR) promises to make large collections of sheet music searchable by their musical content. It would open up novel ways of accessing the vast amount of written music that has never been recorded before. For a long time, OMR was not living up to that promise, as its performance was simply not good enough, especially on handwritten music or under non-ideal image conditions. However, OMR has recently seen a number of improvements, mainly due to the advances in machine learning. In this work, we take an OMR system based on the traditional pipeline and an end-to-end system, which represent the current state of the art, and illustrate in proof-of-concept experiments their applicability in retrieval settings. We also provide an example of a musicological study that can be replicated with OMR outputs at much lower costs. Taken together, this indicates that in some settings, current OMR can be used as a general tool for enriching digital libraries.

Scores of scores: an openscore project to encode and share sheet music

Abstract: This article reports on 'OpenScore', an initiative for encoding and sharing sheet music, and its first substantial sub-project, the 'Scores of Scores' Lieder Encoding Project in association with the social enterprise 'Four Score and More'. The project incorporated a wide range of approaches---including the re-licensing of existing encodings, automated file format conversion, and crowdsourcing encodings to new specifications---which may be taken as a summary of the state of play in this rapidly changing field. This paper sets out our experiences and focuses on the issues which may be of most interest and use to others considering similar projects, particularly those managing submissions from diverse sources.

Publishing musicology using multimedia digital libraries: creating interactive articles through a framework for linked data and MEI

Abstract: Modern web publishing enables sophisticated presentation of academic arguments, deploying evidence in different formats, using multiple media types, and through interactive user experiences. However, these developments have had little significant effect on the communication of musicology, which largely continues to be published in static and linear forms, and rarely with user interfaces that connect the different forms of supporting material.Our Music Encoding and Linked Data (MELD) framework uses RDF to associate music-related materials via structures afforded by the Music Encoding Initiative (MEI). Here we describe the use of MELD for publishing multimedia musicology articles as web applications in which musically-meaningful relationships are mapped to the interactions a user experiences when moving between digital resources such as text, audio, video, and notation.We motivate our work with an enhanced interactive article studying the performance of works by Frederick Delius, and demonstrate our framework's suitability for this situation by implementing a MELD application integrating TEI text, IIIF-served images, MEI notation and recordings of audio and video. We describe the semantic annotations which underpin this realisation, and how they relate the user experience of moving between this content to the musicological argument being marshalled. Through this example we illustrate how connecting diverse media types using musically-meaningful semantics can support a richer form of publication, beyond the current state of the art.

An open corpus for the computational research of arab-andalusian music

Abstract: In the medieval Islamic territories of the Iberian Peninsula known as Al-Andalus a unique style of music was formed combining local practices with Arab sensibilities. After the fall of the last Andalusian kingdom, this classical repertoire has been preserved to the present in North African countries. The idiosyncrasies of this repertoire, which combines musical traits from Western and Eastern Mediterranean traditions in orchestral and choral settings, as well as instrumental and vocal solos, deserves an in depth musicological study, that can benefit from computational tools for corpus-driven research. On the other hand, the characteristics of this music poses interesting challenges to MIR methods and therefore offer new research opportunities to this field. To address these topics, we present here the first complete release of the corpus for the research of the Moroccan tradition of Arab-Andalusian music built in the framework of the CompMusic project. The corpus comprises three data collections, namely audio recordings, music scores and lyrics, as well as related annotations and metadata. We also present a series of Jupyter Notebooks for browsing and retrieving data from the corpus. Both the corpus and notebooks are completely open to the research community.

Differentiae in the cantus manuscript database: standardization and musicological application

Abstract: When approaching the study of medieval plainchant, one is inevitably confronted with its immensity, spanning several centuries and a wide geographic area. Even at a specific time and place, its notation and liturgy varies depending on the type of institution: local churches were influenced by the cathedral tradition, whilst monastic houses were influenced by local dioceses and their order, each of which regulated the liturgy to varying degrees. As such, each extant manuscript contains elements that could be regionally and/or globally standardized and/or not standardized, making large-scale data collection and analysis a daunting task. One method of tracing the relationships between manuscripts of different provenances is to examine a shared, stable feature, like differentiae in antiphoners.Differentiae, melodic formulas that set the final two words of the doxology, are always included at the end of psalm recitations in antiphonal psalmody and appear in conjunction with every antiphon in an antiphoner, regardless of the manuscript's provenance. This paper describes an ongoing project to standardize the differentiae field of the Cantus Manuscript Database so as to enable cross-manuscript comparisons. With over 1,400 unique differentiae across 144 manuscripts (900s-1500s) processed to date, this project will enable scholars to explore hitherto unanswerable questions about not only the function of differentiae, but also, more broadly, chant transmission.To demonstrate the musicological potential of the differentiae standardization project, this paper includes a case study that interrogates the most commonly used definition of these melodic formulas: that they provided melodic transitions from psalm recitations to antiphon openings. The existence of this melodic connection is contested amongst scholars and its exact nature has never been clearly defined, due to the lack of available and standardized data. This paper demonstrates and defines the melodic relationship between differentiae and antiphon openings for the first of eight modes, whilst considering the ramifications of this relationship on the use of differentiae as mnemonic devices for the recollection of antiphon melodies.

Encoding matters

Abstract: In this paper, we discuss how different encodings in symbolic music files can have consequences for music analysis, where a truthful representation, not only of the musical score, but of the semantics of the music, can change the results of music analysis tools. We introduce a series of examples in which different encodings effectively modify the content of two---apparently equivalent---symbolic music files. These examples have been obtained from comparing three different encodings of a string quartet movement by Ludwig van Beethoven.We present two scenarios in which encoding discrepancies may be introduced. In the first scenario, they have been introduced during the encoding of the symbolic music file by either the music notation software or the human encoder. The discrepancies introduced in this scenario are typically difficult to notice because they are visually identical to an accurate encoding. In the second scenario, the discrepancies have been introduced during the translation of the original file into other symbolic formats. In this scenario, the discrepancies may be related to propagating errors in the original encoding or to an erroneous translation of certain attributes of the musical content. Finally, we discuss the possibility of using the examples provided here for the mitigation of some of these discrepancies in the future.

Music in newspapers: interdisciplinary opportunities and data-related challenges

Abstract: Over the past millennia, music has actively been performed and listened to by mankind, thus also playing an important role in establishing sociocultural identities that have evolved over time. In parallel, for many centuries, newspapers played an important role in informing society on a regular and frequent basis on topics noteworthy at that time. Therefore, in retrospect, these newspapers offer windows into historic topics of sociocultural significance, including cultural and musical life. Thanks to ongoing digitization efforts, large-scale newspaper corpora now have become broadly available and accessible. Taking the digitized historical newspaper collection of the National Library of The Netherlands as an example, in this paper, we discuss how considering music-related mentionings in newspapers can enable potential new research directions and questions. We discuss open syntactic and semantic data-related technical challenges when analyzing music-related mentionings in digitized historical newspaper collections. Finally, we discuss how successful detection of music-related mentionings can also benefit engagement of non-scholarly end users, concluding with an invitation to the interdisciplinary research community to actively contribute to the given use case.

Jazzcats: navigating an RDF triplestore of integrated performance metadata

Abstract: Applying Linked Data techniques to musical metadata can facilitate new paths of musicological inquiry. JazzCats: Jazz Collection of Aggregated Triples is a prototype project interlinking four discrete jazz performance datasets and external sources as references. Tabular, relational, and graph legacy datasets have necessitated different RDF production and ingestion workflows to support scholarly study of performance traditions. This paper highlights critical processes of data curation for digital libraries, including quality assessment of the ingested datasets. In addition, we describe research questions enabled by JazzCats, raise musicological implications, and offer suggestions to overcome current limitations.

Evaluating symbolic representations in melodic similarity

Abstract: Symbolic melodic similarity measures have been the subject of considerable investigation for their role in content-based querying, digital musicological analysis, and other data driven applications of Music Information Retrieval (MIR). Despite these efforts, there has been little focus on the representations or encodings employed by symbolic similarity measures, and how each of these representations affects the analysis that follows it. Understanding how these similarity measures behave can improve the way we index and retrieve digital musical content, and offer insights into the underlying musical patterns.This work explores how five melodic encodings, with varying information types and loss, behave using common string matching melodic similarity measures for exact and inexact matching, both globally and locally. The differences in the various symbolic melodic encodings are summarized to provide understanding and context as to when and in what applications these encodings could be applied.

mytune: webvisualization technologies for irish traditional music archives

Abstract: Online digital music libraries have become important repositories for music for all enthusiasts of Irish traditional music. These libraries have certainly facilitated the discovery of new material and enabled extensive exploration of musical variations on well-known favorites. Current web interfaces to these repositories, offer a combination of text and notation-based information as well as providing audio and midi listening formats. However, these interfaces could take advantage of more recently available technologies offering graph-based, interactive, visual displays and more useful data analytic content. Drawing on the state-of the-art data visualization web libraries, the objective of this work is to present a prototype of a new digital library for Irish traditional music. The design and architecture is detailed using standard Unified Modelling Language (UML) notation. Evaluation is carried out by a usability survey to determine the effectiveness of the enhancements.