Showing papers presented at "International Conference on Auditory Display in 2009"

PhysioSonic - evaluated movement sonification as auditory feedback in physiotherapy

Abstract: We detect human body movement interactively via a tracking system. This data is used to synthesize sound and transform sound files (music or text). A subject triggers and controls sound parameters with his or her movement within a pre-set range of motion. The resulting acoustic feedback enhances new modalities of perception and the awareness of the body movements. It is ideal for application in physiotherapy and other training contexts. The sounds we use depend on the context and aesthetic preferences of the subject. On the one hand, metaphorical sounds are used to indicate the leaving of the range of motion or to make unintended movements aware. On the other hand, sound material like music or speech is played as intuitive means and motivating feedback to address humans. The sound material is transformed in order to indicate deviations from the target movement. PhysioSonic has been evaluated with a small study on 12 patients with limited shoulder mobility. The results show a clear benefit for most patients, who also report on PhysioSonic being an enrichment of their therapeutic offer.

Imagine the sounds: an intuitive control of an impact sound synthesizer

Abstract: In this paper we present a synthesizer developed for musical and Virtual Reality purposes that offers an intuitive control of impact sounds. A three layer control strategy is proposed for this purpose, where the top layer gives access to a control of the sound source through verbal descriptions, the middle layer to a control of perceptually relevant sound descriptors, while the bottom layer is directly linked to the parameters of the additive synthesis model. The mapping strategies between the parameters of the different layers are described. The synthesizer has been implemented using Max/MSP, offering the possibility to manipulate intrinsic characteristics of sounds in real-time through the control of few parameters.

Exploring Function and Aesthetics in Sonifications for Elite sports

Abstract: The potential for real-time sonifications to enhance attention for specific movement sections in athletics has led to a growing interest from the elite sports community. Early examples of sonifications for swimming and (ergometer) rowing have demonstrated the functionality of this idea. These pioneering sport sonifications use simple sine-tones to achieve real-time responsiveness. However further improvements in technology will enable more complex sounds to be synthesised, opening up the design possibilities. During the COST-SID workshop on sonification, different design possibilities were explored using the rowing movement and data as a case study. The results identify the potential to include the motivational and physiological aspects of music and the richness of natural sounds. In further related work the initial sine-wave sonification was trialed in action with rowers on the water with improvements in speed demonstrating the effectiveness. Responses to a questionnaire about the sound indicate that functionality is a primary dimension in the aesthetics of a sonification in elite sports. The results from the workshop combined with the questionnaire identify many issues and raise many questions about function and aesthetics in sonifications designed specifically for elite sports applications.

Auditory representations of a graphical user interface for a better human-computer interaction

Abstract: As part of a project to improve human computer interaction mostly for blind users, a survey with 50 blind and 100 sighted users included a questionnaire about their user habits during everyday use of personal computers. Based on their answers, the most important functions and applications were selected and results of the two groups were compared. Special user habits and needs of blind users are described. The second part of the investigation included collecting of auditory representations (auditory icons, spearcons etc.), mapping with visual information and evaluation with the target groups. Furthermore, a new design method for auditory events and class was introduced, called “auditory emoticons”. These use non-verbal human voice samples to represent additional emotional content. Blind and sighted users evaluated different auditory representations for the selected events, including spearcons for different languages. Auditory icons using environmental, familiar sounds as well emoticons are received very well, whilst spearcons seem to be redundant except menu navigation for blind users.

Investigating narrative and performative sound design strategies for interactive commodities

Abstract: Computing technologies turn everyday artefacts into narrative, procedural objects. This observation suggests that the narrative sound design strategies used in films and many video games could also be applied for the design of interactive commodities. However, it is unknown whether these strategies from immersive media can be applied in physical artefacts of everyday use. In this paper we describe methodological considerations and outline a structure of a revisable, design oriented, participatory research process, which allows to explore narrative sound designs and their possible application in interactive commodities in a systematic yet explorative way. The process, which focused on interpretational aspects, has been applied in two workshops and their results are reported and discussed. The experience of the prototyping and evaluation method, which made use of theatrical strategies, raised important questions about the role of performativity in the emergence of meaning and the possible limitations of a strictly hermeneutic aesthetics, when dealing with sonically enhanced interactive commodities.

A review of methods and frameworks for sonic interaction design: exploring existing approaches

Abstract: This article presents a review of methods and frameworks focused on the early conceptual design of sonic interactions. The aim of the article is to provide novice and expert designers in human computer interaction an introduction to sonic interaction design, to auditory displays, and to the methods used to design the sounds and interactions. A range of the current best practices are analysed. These are discussed with regard to the key methods and concepts, by providing examples from existing work in the field. A complementary framework is presented to highlight how these methods can be used together by an auditory display designer at the early conceptual design stage. These methods are reflected upon and provides a closing discussion on the future directions of research that can be explored using these approaches.

A sound design for acoustic feedback in elite sports

Abstract: Sound (acoustic information) is the naturally evocative, audible result of kinetic events. Humans interact with the world by the everyday experience of listening to perceive and interpret the environment. Elite athletes, especially, rely on sport specific sounds for feedback about successful (or unsuccessful) movements. Visualization plays the dominant role in technique analysis, but the limitations of visual observation (of time related events) compared with auditory perception, which represents information with a clearer time-resolution, mean that acoustic displays offer a promising alternative to visual displays. Sonification, as acoustic representation of information, offers an abundance of applications in elite sports for monitoring, observing movement and detecting changes therein. Appropriate sound is needed to represent specific movement patterns. This article presents conceptual considerations for a sound design to fulfill the specific purpose of movement optimization that would be acceptable to elite athletes, with first practical experience with elite athletes in rowing.

The Application of Axiomatic Design Theory and Conflict Techniques for the Design of Intersections: Part 1

Abstract: Today the design and analysis of traffic intersections is most commonly done using traditional traffic conflict techniques. In this paper, we compare and combine traditional traffic conflict techniques and axiomatic design theory. Both the conflict techniques and axiomatic design theory are applied to a generic 4-way intersection. Strategies to improve the intersection including separation of space and separation of time are considered. The limitations and implications of conflict techniques, axiomatic design theory, and the two strategies are addressed. Finally, the future implications of this work are discussed.

From signal to substance and back: insights from environmental sound research to auditory display design

Abstract: A persistent concern in the field of auditory display design has been how to effectively use environmental sounds, which are naturally occurring familiar non-speech, non-musical sounds Environmental sounds represent physical events in the everyday world, and thus they have a semantic content that enables learning and recognition However, unless used appropriately, their functions in auditory displays may cause problems One of the main considerations in using environmental sounds as auditory icons is how to ensure the identifiability of the sound sources The identifiability of an auditory icon depends on both the intrinsic acoustic properties of the sound it represents, and on the semantic fit of the sound to its context, ie, whether the context is one in which the sound naturally occurs or would be unlikely to occur Relatively recent research has yielded some insights into both of these factors A second major consideration is how to use the source properties to represent events in the auditory display This entails parameterizing the environmental sounds so the acoustics will both relate to source properties familiar to the user and convey meaningful new information to the user Finally, particular considerations come into play when designing auditory displays for special populations, such as hearing impaired listeners who may not have access to all the acoustic information available to a normal hearing listener, or to elderly or other individuals whose cognitive resources may be diminished Some guidelines for designing displays for these populations will be outlined

Teaching Axiomatic Design in the Freshman Year: A Case Study at KAIST

Abstract: Axiomatic design theory was once considered the domain of engineering doctoral candidates and industry experts in search of an improved understanding of design. However, an innovative freshman design course at KAIST is using Axiomatic Design Theory, along with traditional product design and TRIZ, to improve the students’ ability to think independently, consciously, rationally, and synthetically. This paper discusses the basic format, goals, and philosophy for ED100: Introduction to Design and Communication. Finally, the successes, challenges, and the future implications of the course are also discussed.

Tools for designing emotional auditory driver-vehicle interfaces

Abstract: Auditory interfaces are often used in vehicles to inform and alert the driver of various events and hazards. When designed properly, such interfaces can e.g. reduce reaction times and increase the impression of quality of the vehicle. In this paper it is argued that emotional response is an important aspect to consider when designing auditory driver-vehicle interfaces. This paper discusses two applications developed to investigate the emotional dimensions of auditory interfaces. EarconSampler is a tool for designing and modifying earcons. It allows for creating melodic patterns of wav-snippets and adjustment of parameters such as tempo and pitch. It also contains an analysis section where sound quality parameters, urgency and emotional response to the sound is calculated / predicted. SoundMoulder is another tool which offers extended temporal and frequency modifications of earcons. The primary idea with this application is to study how users design sounds given a desired emotional response.

Simulating the soundscape through an analysis/resynthesis methodology

Abstract: This paper presents a graph-based system for the dynamic generation of soundscapes and its implementation in an application that allows for an interactive, real-time exploration of the resulting soundscapes. The application can be used alone, as a pure sonic exploration device, but can also be integrated into a virtual reality engine. In this way, the soundcape can be acoustically integrated in the exploration of an architectonic/urbanistic landscape. The paper is organized as follows: after taking into account the literature on soundscape, we provide a formal definition of the concept; then, a model is introduced, and finally, we describe a software application together with a case-study.

Applying Axiomatic Design to the Educational Process

Abstract: Axiomatic design theory (AD) was originally developed to form a scientific basis for mechanical design. This paper takes an in-depth look at axiomatic design theory as a scientific basis for the design of educational courses and curricula. The implications of the first and second axioms for education are discussed and issues associated with coupling, physical integration, repetition, redundancy, complexity, robustness, and flexibility are addressed.

A graph-based system for the dynamic generation of soundscapes

Abstract: This paper presents a graph-based system for the dynamic generation of soundscapes and its implementation in an application that allows for an interactive, real-time exploration of the resulting soundscapes. The application can be used alone, as a pure sonic exploration device, but it can also be integrated into a virtual reality engine. In this way, the soundcape can be acoustically integrate in the exploration of an architectonic/urbanistic landscape. The paper is organized as follows: after taking into account the literature relative to soundscape, a formal definition of the concept is given; then, a model is introduced; finally, a software application is described together with a case-study.

Navigation performance effects of render method and head-turn latency in mobile audio augmented reality

Abstract: This study assessed participant performance of an outdoor navigation task using a mobile audio augmented reality system. Several quantitative performance measures and one subjective measure were used to compare the perceptual efficacy of Ambisonic and VBAP binaural rendering techniques, and a range of head-turn latencies. The study extends existing indoors research on the effects of head-turn latency for seated listeners. The pilot experiment found that a source capture radius of 2 meters significantly affected the sole participant's navigation distance efficiency compared to other radii. The main experiment, using 8 participants, found that render method significantly affected all performance measures except subjective stability rating, while head-turn latency only affected mean track curvature and subjective stability. Results also showed an interaction in which the choice of rendering method mitigated or potentiated the effects of head-turn latency on perceived source stability.

Multi-touch interactions for model-based sonification

Abstract: This paper presents novel interaction modes for Model-Based Sonification (MBS) via a multi-touch interface. We first lay out details about the constructed multi-touch surface. This is followed by a description of the Data Sonogram Sonification Model and how it is implemented using the system. Modifications from the original sonification model such as the limited space scans are described and discussed with sonification examples. Videos showing examples of interaction are provided for various data sets. Beyond Data Sonograms, the presented system provides a basis for the implementation of known and novel sonification models. We discuss the available interaction modes with multi-touch surfaces and how these interactions can be profitably used to control spatial and non-spatial sonification models.

A communal map of design in auditory display

Abstract: The workshop on Recycling Auditory Displays at ICAD 2008 aimed to capture knowledge about the design of auditory displays from the participants in a manner that would be easy to understand and reuse. The participants introduced themselves by providing examples of a good and a bad sound design. These examples raised issues of culture, identity, aesthetics and context that are more usually associated with product sound design than auditory display. Based on these discussions the themes Users, Applications, Techniques, and Environments were chosen to focus the further development of ideas. A mindmapping session was used to collect over 150 entries under these themes, and more than 30 references. An additional Others theme was needed for ideas that did not fit neatly into the existing categories. The information that has been collected shows that most research in auditory display falls under the themes of Applications and Techniques. The information under the themes of Users and Others shows the overlap with related disciplines such as auditory neuroscience, product design, sound arts, semiotics, and interface design. The Environment theme raised the need for future research to include contextual issues. The outcome of the workshop has been to produce a collaborative understanding of the current state of design knowledge in the Auditory Display community, and to identify future directions for research into the design of Auditory Displays.

Effect of sound source stimuli on the perception of reverberation in large volumes

Abstract: The aim of the presented research is to determine whether the perception of reverberation is dependent on the type of sound stimuli used. We quantified the discrimination thresholds for reverberations that are representative for large rooms such as concert halls (reverberation times around 1.8 s). For exponential decays, simulating an ideal simple room, thresholds are around 6% (Experiment 1). We found no difference in thresholds between a short noise burst and a male voice spoken word, suggesting that discrimination is not dependent on the type, or spectral content, of the sound source (Experiment 2). In two further experiments using a magnitude estimation paradigm we assessed the perceived amount of reverberation as a function of various types of stimuli. Whereas the discrimination of reverberant stimuli does not seem to be affected by the sound stimulus, the perceived amount of reverberation is affected. Vocal stimuli are perceived as being more reverberant than non-vocal stimuli. The results are discussed in light of current neuroscientific models of auditory processing of complex stimuli but also with respect to their consequences for the use of reverberation in auditory display.

Polyphonic alignment algorithms for symbolic music retrieval

Abstract: Melody is an important property for the perceptual description of Western musical pieces. A lot of applications rely on the evaluation of similarity between two melodies. While several existing techniques assume a monophonic context or extract a monophonic melody from polyphonic pieces, in this paper, we propose to consider the whole polyphonic context to evaluate the similarity without reducing to a monophonic melody. We thus propose a new model and a corresponding methodology that takes into account all the notes, even if they sound at the same time or if they overlap. Our model relies on a quotiented sequence representation of music. A quotiented sequence is a sequence graph defined with an additional equivalent relation on its vertices and such that the quotient graph is also a sequence graph. The core of the comparison method is based on an adaptation of edit-distance metrics, regularly applied in bio-informatic context. This algorithm is currently being used to evaluate the similarity between a monophonic or polyphonic query and a database of polyphonic musical pieces. First experiments show that the adaptation to polyphony does not degrade the quality of the algorithm with monophonic musical pieces. Furthermore, the results of experiments with polyphonic pieces are promising, even if they show some limitations.

Using sound to identify correlations in market data

Abstract: Despite intensive study, a comprehensive understanding of the structure of capital market trading data remains elusive. The one known application of audification to market price data reported in 1990 that it was difficult to interpret the results probably because the market does not resonate according to acoustic laws. This paper reports on some techniques for transforming the data so it does resonate; so audification can be used as a means of identifying autocorrelation in capital market trading data. Also reported are some experiments in which the data is sonified using a homomorphic modulation technique. The results obtained indicate that the technique may have a wider application to other similarly structured time-series data.

Spatialized synthesis of noisy environmental sounds

Abstract: In this paper, an overview of the stochastic modeling for analysis/synthesis of noisy sounds is presented. In particular, we focused on the time-frequency domain synthesis based on the inverse fast Fourier transform (IFFT) algorithm from which we proposed the design of a spatialized synthesizer. The originality of this synthesizer remains in its one-stage architecture that efficiently combines the synthesis with 3D audio techniques at the same level of sound generation. This architecture also allowed including a control of the source width rendering to reproduce naturally diffused environments. The proposed approach led to perceptually realistic 3D immersive auditory scenes. Applications of this synthesizer are here presented in the case of noisy environmental sounds such as air swishing, sea wave or wind sound. We finally discuss the limitations but also the possibilities offered by the synthesizer to achieve sound transformations based on the analysis of recorded sounds.

Analysis/synthesis and spatialization of noisy environmental sounds

Abstract: The use of stochastic modeling is discussed for analysis/synthesis and transformation of environmental sounds. The method leads to perceptually relevant synthetic sounds based on the analysis of natural sounds. Applications are presented, such as sound effects using parametric signal transformations, or data compression. Moreover, we propose a method which efficiently combines the stochastic modeling with 3D audio techniques. This architecture offers an efficient control of the source width rendering that is often an important attribute of noisy environmental sounds. This control is of great interest for virtual reality applications to create immersive 3D scenes.

Towards timbre modeling of sounds inside accelerating cars

Abstract: Quality investigations and design of interior car sounds constitute an important challenge for the car industry Such sounds are complex and time-varying, inducing considerable timbre variations depending on the driving conditions An interior car sound is indeed a mixture between several sound sources, with two main contributions, ie the engine noise on the one hand and the aerodynamic and tire-road noise on the other Masking phenomena occur between these two components and should be considered when studying perceptive attributes of interior car sounds in order to identify relevant signal parameters By combining sensory analysis and signal analysis associated with an auditory model, a relation between a reduced number of signal parameters and perceptive attributes can be found This approach has enabled us to propose timbre descriptors based on the tristimulus criterion that reflect the dynamic behavior of a sound inside an accelerating car

Awesome sound design tool: a web based utility that invites end users into the audio design process

Abstract: Previous auditory display research has shown how fundamental aspects of an auditory signal may influence perception and impact the emotional state of the listener. However, a challenge for designers is how to find signals that correspond to user situations and make sense within a user context. In this article we present a web based application called AWESOME Sound Design Tool; a tool that invites users to take part in the design process of auditory signals. The basic idea is to give users control over some aspects of the auditory stimuli and encourage them to manipulate the sound with a specific user scenario in mind. The software may help developers working with applied design to find more appropriate sounds for user situations. It might also help researchers to better understand correlations between the properties of a sound and characteristics of a user situation. A pilot study has been conducted in which car drivers designed warning signals for critical traffic situations. The pilot study illustrated how the tool could be useful for applied audio design.

Quantum harmonic oscillator sonification

Abstract: This work deals with the sonification of a quantum mechanical system and the processes that occur as a result of its quantum mechanical nature and interactions with other systems. The quantum harmonic oscillator is not only regarded as a system with sonifiable characteristics but also as a storage medium for quantum information. By representing sound information quantum mechanically and storing it in the system, every process that unfolds on this level is inherited and reflected by the sound. The main profit of this approach is that the sonification can be used as a first insight for two models: a quantum mechanical system model and a quantum computation model.

Simulator sickness in mobile spatial sound spaces

Abstract: In this paper we summarize, evaluate, and discuss the effect of movement patterns in a spatial sound space on the perceived amount of simulator sickness, the pleasantness of the experience, and the perceived workload. During our user study nearly 48 percent of all participants showed mild to moderate symptoms of simulator sickness, with a trend towards stronger symptoms for those experiencing left to right movements. We found evidence for predictable left to right movements leading to a perceived unpleasantness that is significantly higher than for unpredictable or no movement at all. However none of the movement patterns had a noticable effect on the perceived cognitive load for simple tasks. We also found some differences in the perception of the sound space between men and women. Women tended to have a stronger dislike for the sound space and found the task to be more difficult.

Overlay problems for music and combinatorics

Abstract: Motivated by the identification of the musical structure of pop songs, we introduce combinatorial problems involving overlays (non-overlapping substrings) and the covering of a text t by them. We present 4 problems and suggest solu- tions based on string pattern matching techniques. We show that decision problems of this type can be solved using an Aho-Corasick keyword automaton. We conjecture that one general optimization problem of the type, is NP-complete and introduce a simpler, more pragmatic optimization prob- lem. We solve the latter using suffix trees and finally, we suggest other open problems for further investigation.

Raising awareness about complete automation of live-electronics: a historical perspective

Abstract: The article raises attention to the advantages and disadvantages of complete automation in DSP processes (including the triggering of events) during the performance of interactive music. By proposing a historic summary divided in three main periods according to the technologies, methods and processes available during each of them, (including examples of key works and composers), it shows how the usage of automation in live-electronics was dependent on the development of new technologies, specially digital. Further, it explains how full automation works in two works of mine, describing the features and techniques involved. Considering those examples, the advantages and disadvantages resulting from the introduction of complete automation of DSP in live performance are finally discussed. Even though automation is not a new technique in the field, I am keen to dedicate special attention to completely automated events -including their triggering- given the impact that automation can have on performances.

Algorithms for an automatic transcription of live music performances into symbolic format

Abstract: This paper addresses the problem of the real-time automatic transcription of a live music performance into a symbolic format. The source data are given by any music instrument or other device able to communicate through a performance protocol. During a performance, music events are parsed and their parameters are evaluated thanks to rhythm and pitch detection algorithms. The final step is the creation of a well-formed XML document, validated against the new international standard known as IEEE 1599. This work will shortly describe both the software environment and the XML format, but the main analysis will involve the real-time recognition of music events. Finally, a case study will be presented: PureMX, a set of Pure Data externals, able to perform the automatic transcription of MIDI events.