Showing papers by "J. Stephen Downie published in 2017"

A framework for evaluating multimodal music mood classification

Abstract: This research proposes a framework for music mood classification that uses multiple and complementary information sources, namely, music audio, lyric text, and social tags associated with music pieces. This article presents the framework and a thorough evaluation of each of its components. Experimental results on a large data set of 18 mood categories show that combining lyrics and audio significantly outperformed systems using audio-only features. Automatic feature selection techniques were further proved to have reduced feature space. In addition, the examination of learning curves shows that the hybrid systems using lyrics and audio needed fewer training samples and shorter audio clips to achieve the same or better classification accuracies than systems using lyrics or audio singularly. Last but not least, performance comparisons reveal the relative importance of audio and lyric features across mood categories.

The MIREX grand challenge: A framework of holistic user-experience evaluation in music information retrieval

Abstract: Music Information Retrieval MIR evaluation has traditionally focused on system-centered approaches where components of MIR systems are evaluated against predefined data sets and golden answers i.e., ground truth. There are two major limitations of such system-centered evaluation approaches: a The evaluation focuses on subtasks in music information retrieval, but not on entire systems and b users and their interactions with MIR systems are largely excluded. This article describes the first implementation of a holistic user-experience evaluation in MIR, the MIREX Grand Challenge, where complete MIR systems are evaluated, with user experience being the single overarching goal. It is the first time that complete MIR systems have been evaluated with end users in a realistic scenario. We present the design of the evaluation task, the evaluation criteria and a novel evaluation interface, and the data-collection platform. This is followed by an analysis of the results, reflection on the experience and lessons learned, and plans for future directions.

Access to Billions of Pages for Large-Scale Text Analysis

Abstract: Consortial collections have led to unprecedented scales of digitized corpora, but the insights that they enable are hampered by the complexities of access, particularly to in-copyright or orphan works. Pursuing a principle of non-consumptive access, we developed the Extracted Features (EF) dataset, a dataset of quantitative counts for every page of nearly 5 million scanned books. The EF includes unigram counts, part of speech tagging, header and footer extraction, counts of characters at both sides of the page, and more. Distributing book data with features already extracted saves resource costs associated with large-scale text use, improves the reproducibility of research done on the dataset, and opens the door to datasets on copyrighted books. We describe the coverage of the dataset and demonstrate its useful application through duplicate book alignment and identification of their cleanest scans, topic modeling, word list expansion, and multifaceted visualization.

Information-seeking in large-scale digital libraries: strategies for scholarly workset creation

Abstract: Large-scale digital libraries such as the HathiTrust contain massive quantities of content combined from heterogeneous collections, with consequential challenges in providing mechanisms for discovery, unified access, and analysis. The HathiTrust Research Center has proposed 'worksets' as a solution for users to conduct their research into the 15 million volumes of HathiTrust content; however existing models of users' information-seeking behaviour, which might otherwise inform workset development, were established before digital library resources existed at such a scale. We examine whether these information-seeking models can sufficiently articulate the emergent user activities of scholarly investigation as perceived during the creation of worksets. We demonstrate that a combination of established models by Bates, Ellis, and Wilson can accommodate many aspects of information seeking in large-scale digital libraries at a broad, conceptual, level. We go on to identify the supplemental information-seeking strategies necessary to specifically describe several workset creation exemplars. Finally, we propose complementary additions to the existing models: we classify strategies as instances of querying, browsing, and contribution. Similarly we introduce a notion of scope according to the interaction of a strategy with content, content-derived metadata, or contextual metadata. Considering the scope and modality of new and existing strategies within the composite model allows us to better express--and so aid our understanding of--information-seeking behaviour within large-scale digital libraries.

Towards publishing secure capsule-based analysis

Abstract: Computational engagement with the HathiTrust Digital Library (HTDL) is confounded by the in- copyright status and licensing restrictions on the majority of the content. Because of these limitations, computational analysis on the HTDL must either be carried out in a secure environment or on derivative datasets. The HathiTrust Research Center (HTRC) Data Capsule service provides researchers with a secure environment through which they invoke tools that create, analyze, and export non-consumptive datasets. These derivative datasets, so long as they do not reproduce the full-text of the original work, are a transformative work protected by Fair Use provisions of United States Copyright Law, and can be published for reuse by other researchers, as the HTRC Extracted Features Dataset has been. Secure environments and derivative datasets enable researchers to engage with restricted data from focused studies of a few dozen volumes to large- scale experiments on millions of volumes. This paper describes advances in the Capsule service through a case study of how the HTRC Data Capsule service has advanced our activities on provenance, workflows, worksets, and non-consumptive exports through a topic modeling example. We also discuss the potential applications of this Capsule-based model to other digital libraries wrestling with research access and copyright restrictions.

Writers of the Lost Paper: A Case Study on Barriers to (Re-) Finding Publications

Abstract: We document the surprising hurdles that we encountered when attempting a known-item search to locate copies of four of our own published research papers, known to be archived in the ACM Digital Library and Google Scholar. The discoveries made in this exercise in ‘search engine archaeology’ are noteworthy as they are equally relevant to other users engaging with these and other digital libraries, to whom the pitfalls are much less readily apparent. We present details of our investigation together with a description of MEDDLE (a ModifiED Digital Library Environment), a proof-of-concept system that illustrates a technique to address some of these search issues for a target digital library. We conclude with suggestions on how scholarly digital libraries may avoid these issues in the future.

Building Worksets for Scholarship by Linking Complementary Corpora.

Abstract: The HathiTrust Digital Library (HTDL) comprises digitized representations of 15.1 million volumes: approximately 7.47 million book titles, 418,216 serial titles, and 5.3 billion pages, across 460 languages. HTDL is best described as “a partnership of major research institutions and libraries working to ensure that the cultural record is preserved and accessible long into the future”. The HathiTrust Research Center (HTRC) develops software models, tools, and infrastructure to help digital humanities (DH) scholars conduct new computational analyses of works in the HTDL. For many scholars the size of the HTDL corpus is both attractive and daunting: many existing DH tools are designed for smaller collections, and many research inquiries are facilitated by more focused, homogeneous collections of texts (Gibbs and Owens, 2012).

Creating a Policy Framework for Analytic Access to In-Copyright Works for Non-Consumptive Research.

Abstract: We report on the work of a recent HathiTrust Research Center (HTRC) task force charged to draft an actionable, definitional Non-Consumptive Use Research Policy. As the research division of HathiTrust, the HTRC facilitates computational text analysis of materials in the HathiTrust Digital Library (HTDL) by adhering to a non-consumptive research paradigm. As the HTRC has integrated the text of the full HTDL corpus into its datastore, it has become increasingly important to clarify and codify the Center’s policy for non-consumptive research. The task force, which consisted of copyright and scholarly communications librarians and representatives from HathiTrust operations and the HTRC, recommended a policy that clarifies acceptable researcher behavior and allowable exports from the HTRC Data Capsule (Plale, et al., 2015). This poster describes the task force’s work to establish a Non-Consumptive Use Research Policy for the HTRC that aims to achieve the same goals as copyright itself: to promote progress in the discovery and spread of knowledge, without harming the commercial interests of authors, publishers, and other stakeholders.

Exploring the Music Library Association Mailing List: A Text Mining Approach

Abstract: Music librarians and people pursuing music librarianship have exchanged emails via the Music Library Association Mailing List (MLA-L) for decades. The list archive is an invaluable resource to discover new insights on music information retrieval from the perspective of the music librarian community. This study analyzes a corpus of 53,648 emails posted on MLA-L from 2000 to 2016 by using text mining and quantitative analysis methods. In addition to descriptive analysis, main topics of discussions and their trends over the years are identified through topic modeling. We also compare messages that stimulated discussions to those that did not. Inspection of semantic topics reveals insights complementary to previous topic analyses of other Music Information Retrieval (MIR) related resources.

All for one and one for all: reconciling research and production values at the HathiTrust through user-scripting

Abstract: This article details a practical technique that safely reconciles the production stability and integrity of the HathiTrust Digital Library (HTDL) with the riskier and potentially disruptive experimental functionalities created by the HathiTrust Research Center. Web systems produced by HTRC are necessarily more speculative and, understandably, operate on equipment outside of the HTDL production environment. The key to our approach that brings these two parts closer together is to exploit user-scripting: a web browser add-in technique that allows users to introduce bespoke Javascript code that alters the behavior of specific website(s). We demonstrate how it can be used to provide a mashup of three web sites: HTDL and two web-based offerings operated independently by HTRC. The end result is that the user interacts with the HTDL as usual, and at strategic locations in the interface additionally functionality drawn from the research systems---which takes account of the user's current context---is seamlessly blended in.

Big-Data Oriented Text Analysis For The Humanities: Pedagogical Use Of The HathiTrust+Bookworm Tool.

Abstract: ion of distant reading and the discovery of specific texts that they can then investigate further through close reading. Our poster includes instances of the kinds of exploration HT+Bookworm made possible for students. An example follows. The concept of “fidelity” (an important word to explore in connection with translation studies, a topic of the classes) shows different characteristics when explored in English (in which the concept maps onto the two words “fidelity” and “faithfulness”) and in Spanish (where the concept maps onto the single word “fidelidad”). Investigating the occurrence of the word by LoC category allows students to explore hypotheses such as whether the greater strength, historically speaking, of religious tradition in the Spanish-speaking world in comparison with the Anglophone world affects the relative prevalence of this word in different domains of use. Another example is a stacked area chart for a word of a kind for which HT+Bookworm helps provide an understanding of the word’s differentiated meanings in different use categories (for example, in the case of the word “depression”, in the use category of psychology and medicine versus that of economics). HT+Bookworm accomplishes this by abstracting separately across different LoC classes, while aiding in the indication of the points in time at which, for each class, the word entered widespread usage.

Exploiting graph‐based data to realize new functionalities for scholar‐built worksets

Abstract: In this poster, we describe how the HathiTrust Research Center (HTRC) is developing a graph‐based approach to representing scholar‐built worksets in the HTRC's research workflows. The use of named graphs to express relationships among worksets, workset items (volumes and pages) and relevant external resources and to manage ownership, version control and access to worksets aligns well with HTRC's non‐consumptive research architecture. In addition to managing scholars' research collections, HTRC workset graphs also provide a means to link individual workset items to descriptive metadata and allows scholars to annotate their worksets and the items in them.