Showing papers presented at "International Conference on Auditory Display in 2004"

Sound design to enhance presence in photorealistic virtual reality

Abstract: The role of soundscape design to enhance the sense of presence in virtual reality is discussed and tested. Preliminary resul ts which compare real versus virtual soundscapes are reported.

The importance of interaction in sonification

Abstract: This paper argues for a special focus on the use of dynamic human interaction to explore datasets while they are being transformed into sound. We describe why this is a special case of both human computer interaction (HCI) techniques and sonification methods. Humans are adapted for interacting with their physical environment and making continuous use of all their senses. When this exploratory interaction is applied to a dataset (by continuously controlling its transformation into sound) new insights are gained into the data’s macro and micro-structure, which are not obvious in a visual rendering. This paper reviews the importance of interaction in sonification, describes how a certain quality of interaction is required, provides examples of the techniques being applied interactively, and outlines a plan of future work to develop interaction techniques to aid sonification.

Selective and Divided Attention: Extracting Information from Simultaneous Sound Sources.

Abstract: The way in which sounds interact and interfere with each other (both acoustically and perceptually) has an important influence on how well an auditory display can convey information. While spatial separation of simultaneous sound sources has been shown to be very effective when a listener must report the content of one source and ignore another source (a condition known as selective attention), little is known about how spatial separation influences performance in divided-attention tasks, i.e., tasks in which the listener must report the content of more than one simultaneous source. This paper reports preliminary results from a pilot study investigating how perceived spatial separation of sources and consistency in source locations influences performance on selectiveand divided-attention tasks. Results demonstrate that 1) in both selectiveand divided-attention tasks, overall performance is generally better when sources are perceived at different locations than when they are perceived at the same location; 2) in both selectiveand divided-attention tasks, randomly changing the perceived source locations from trial to trial tends to degrade performance compared to conditions where the source locations are fixed; and 3) both of the above effects are larger for selective-attention tasks than dividedattention tasks.

Sonification of geo-referenced data for auditory information seeking: Design principle and pilot study

Abstract: We present an Auditory Information Seeking Principle (AISP) (gist, navigate, filter, and details-on-demand) modeled after the visual information seeking mantra [1]. We propose that data sonification designs should conform to this principle. We also present some design challenges imposed by human auditory perception characteristics. To improve blind access to georeferenced statistical data, we developed two preliminary sonifications adhering to the above AISP, an enhanced table and a spatial choropleth map. Our pilot study shows people can recognize geographic data distribution patterns on a real map with 51 geographic regions, in both designs. The study also shows evidence that AISP conforms to people’s information seeking strategies. Future work is discussed, including the improvement of the choropleth map design.

A toolkit for interactive sonification

Abstract: This paper describes work-in-progress on an Interactive Sonification Toolkit which has been developed in order to aid the analysis of general data sets. The toolkit allows the designer to process and scale data sets, then rapidly change the sonification method. The human user can then interact with the data in a fluid manner, continually controlling the position within the set. The interface used by default is the computer mouse, but we also describe plans for multiparametric interfaces which will allow real-time control of many aspects of the data. Early results of interactive sonic analysis of two example domains are described, but extensive user tests are

The sonification of rhythms in human electroencephalogram

Abstract: We use sonification of temporal information extracted from scalp EEG to characterize the dynamic properties of rhythms in certain frequency bands. Sonification proves particularly useful in the simultaneous monitoring of several EEG channels. Our results suggest sonification as an important tool in the analysis of multivariate data with subtle correlation differences.

Marketbuzz: Sonification of real-time financial dataa

Abstract: A system for the sonification of real-time financial data, currently in use by financial traders in five pilot projects, i s described Anecdotal feedback from the pilot projects suggests that the auditory display is more effective and consistent for monitoring the movement of volatile market indices The same system has also been tested in two experiments carried out at the Department of Psychological and Brain Sciences at Dartmouth College In the first experiment, subjects performed “change of direction” monitoring tasks of varying difficulty with and without auditory display The results indicated a significant increase in accuracy when the auditory display was used In the second experiment, subjects performed the same monitoring task with and without auditory display but were given a second, “number-matching” task which forced them to direct their visual attention away from the “change of direction” task from time to time The auditory display increased accuracy more dramatically than in the first experiment, since the subjects were able to rely on the sonification to perform the “change of direction” monitoring task when they were distracted with the “number-matching” task

Creating Functional and Livable Soundscapes for Peripheral Monitoring of Dynamic Data

Abstract: Sonifications must be studied in order to match listener expectancies about data representation in the form of sound. In this study, a system was designed and implemented for dynamically rendering sonifications of simulated real-time data from the stock market. The system read and parsed the stock data then operated unit generators and mixers through a predefined sound mapping to create a ‘soundscape’ of complementary ecological sounds. The sound mapping consisted of a threshold-based model in which a percentage change in price value was mapped to an ecological sound to be played whenever that threshold or gradient had been reached. The system also provided a generic mechanism for fading and transitioning between gradients. The prototype system was presented to stock trader test subjects in their work-listening environment for evaluation as a stand-alone system and in comparison to their preferred tools.

The interaction between head-tracker latency, source duration, and response time in the localization of virtual sound sources

Abstract: One of the fundamental limitations on the fidelity of interactive virtual audio display systems is the delay that occurs between the time a listener changes his or her head position and the the time the display changes its audio output to reflect the corresponding change in the relative location of the sound source. In this experiment, we examined the impact that six difference headtracker latency values (12, 20, 38, 73, 145 and 243 ms) had on the localization of broadband sound sources in the horizontal plane. In the first part of the experiment, listeners were allowed to take all the time they needed to point their heads in the direction of a continuous sound source and press a response switch. In the second part of the experiment, the stimuli were gated to one of eight different durations (64, 125, 250, 375, 500, 750, 1000 and 2000 ms) and the listeners were required to make their head-pointing responses within two seconds after the onset of the stimulus. In the openended response condition, the results showed that latencies as long as 243 ms had no impact on localization accuracy, but that there was an increase in response time when then latency was longer than 73 ms. In contrast, the data from the time-limited response conditions showed that latencies that exceeded 73 ms had no impact on response time but that they significantly increased the angular localization error and the number of front back confusions. Together with the results of earlier studies, these results suggest that headtracker latency values of less than 70 ms are adequate to obtain acceptable levels of localization accuracy in virtual audio displays.

How to Tackle Auditory Interface Aesthetics? Discussion and Case Study

Abstract: This paper discusses the importance of auditory interface aesthetics and presents an empirical investigation of sound aesthetics in context. The theoretical discussion examines the relationship between sound aesthetics and user satisfaction and concludes that, despite the creation of numerous auditory design methods and guidelines, none are dedicated to achieving aesthetically pleasing designs. In a case study, an empirical investigation is conducted to evaluate the relationship between the functional and aesthetic value of an auditory interface. By investigating two different tasks, this study demonstrates that the nature of the tasks allocated to subjects has a significant impact on the aesthetic judgments made by the subjects. Consequently, functional and aesthetic properties of auditory cannot be dealt with independently.

Cultivating design patterns for auditory displays

Abstract: Auditory Displays are quite well known in the research community, but very little of this experience is being transferred to product designers. The method of Design Patterns is well known to a number of design domains and is used to describe “solutions to problems in context” in such a way that they can be reused again and again. Here we present six new prototype Design Patterns for Auditory Display: SystemMonitoring, SituationalAwareness, SonifiedLineGraph, AuditoryIcon, Attenson, and Personalisation.

Auditory Navigation Performance is Affected by Waypoint Capture Radius.

Abstract: Non-speech audio navigation systems can be very effective mobility aids for persons with either temporary or permanent vision loss. Sound design has certainly been shown to be important in such devices [e.g. 1]. In this study we consider the added factor of capture radius. The capture radius of an auditory beacon is defined as the range at which the system considers a user to have reached the waypoint where the beacon is located. 108 participants successfully navigated paths through a virtual world using only nonspeech beacon cues. Performance differed across the capture radius conditions. Further, there was a speed-accuracy tradeoff, which complicates the design decision process. Implications of these results for the design of auditory navigation aids are discussed, as are other ongoing and future studies.

Capture and recreation of higher order 3d sound fields via reciprocity

Abstract: We propose a unified and simple approach for capturing and recreating 3D sound fields by exploring the reciprocity principle that is satisfied between the two processes. Our approach makes the system easy to build, and practical. Using this approach, we can capture the 3D sound field by a spherical microphone array and recreate it using a spherical loudspeaker array, and ensure that the recreated sound field matches the recorded field up to a high order of spherical harmonics. A design example and simulation results are presented. For some regular or semi-regular microphone layouts, we design an efficient parallel implementation of the multi-directional spherical beamformer by using the rotational symmetries of the beampattern and of the spherical microphone array. This can be implemented in either software or hardware. A simple design example is presented to demonstrate the idea. It can be easily adapted for other regular or semi-regular layouts of microphones.

Auditory versus visual spatial impression: A study of two auditoria

Abstract: Spatial impression refers to the attributes of subjective space beyond localization. In the field of auditorium acoustics, auditory spatial impression is often divided into ‘apparent source width’, ‘envelopment’ and sometimes ‘intimacy’. In separate experiments, this study considers how visual and auditory spatial impression vary within two auditoria, and hence similarities between these two sensory modes. In the visual experiment, the ‘spaciousness’, ‘envelopment’, ‘stage dominance’, ‘intimacy’ and target distance were judged by subjects using grayscale projected photographs, taken from various positions in the audience areas of the two auditoria when a visual target was on stage. In the auditory experiment, the ‘apparent source width’, ‘envelopment’, ‘intimacy’ and performer distance were judged using an anechoic orchestral recording convolved with binaural impulse responses measured from the same positions in the two auditoria. Results show target distance to be of primary importance in auditory and visual spatial impression – thereby providing a basis for covariance between some attributes of auditory and visual spatial impression. Nevertheless, some attributes of spatial impression diverge between the senses.

Auditory display of hyperspectral colon tissue images using vocal synthesis models

Abstract: The human ability to recognize, identify and compare sounds based on their approximation of particular vowels provides an intuitive, easily learned representation for complex data. We describe implementations of vocal tract models specically designed for sonification purposes. The models described are based on classical models including Klatt[1] and Cook[2]. Implementation of these models in MatLab, STK[3], and PD[4] is presented. Various sonication methods were tested and evaluated using data sets of hyperspectral images of colon cells 1 2 .

Interacting with sound an interaction paradigm for virtual auditory worlds

Abstract: The visual and the auditory field of perception respond on different input signals from our environment. Thus, interacting with worlds solely trough sound is a very challenging task. This paper discusses methods and techniques for sonification and interaction in virtual auditory worlds. In particular, it describes auditory elements such as speech, sound and music and discusses their application in diverse auditory situations, as well as interaction techniques for assisted sonification. The work is motivated by the development of a framework for the interactive exploration of auditory environments which will be used to evaluate the later discussed techniques. The main focus for the design of this framework is the use in narrative environments for auditory games, but also for general purpose auditory user interfaces and communication processes.

Emotionface: prototype facial expression display of emotion in music

Abstract: EmotionFace is a software interface for visually displaying the self-reported emotion expressed by music. Taken in reverse, it can be viewed as a facial expression whose auditory connection or exemplar is the time synchronized, associated music. The present instantiation of the software uses a simple schematic face with eyes and mouth moving according to a parabolic model: Smiling and frowning of mouth represents valence (happiness and sadness) and amount of opening of eyes represents arousal. Continuous emotional responses to music collected in previous research have been used to test and calibrate EmotionFace. The interface provides an alternative to the presentation of data on a two-dimensional emotion-space, the same space used for the collection of emotional data in response to music. These synthesized facial expressions make the observation of the emotion data expressed by music easier for the human observer to process and may be a more natural interface between the human and computer. Future research will include optimization of EmotionFace, using more sophisticated algorithms and facial expression databases, and the examination of the lag structure between facial expression and musical structure. Eventually, with more elaborate systems, automation and greater knowledge of emotion and associated musical structure, it may be possible to compose music meaningfully from synthesized and real facial expressions.

Individual differences, cognitive abilities, and the interpretation of auditory graphs

Abstract: Auditory graphs exploit pattern recognition in the auditory system, but questions remain about the relationship between cognitive abilities, demographics, and sonification interpretation. Subjects completed a magnitude estimation task relating sound dimensions to data dimensions. Subjects also completed a working memory task (2-back task) and a spatial reasoning task (Raven's Progressive Matrices) to assess cognitive abilities. Demographics, such as gender, age, handedness, and musical experience, were also reported and included in the analysis. A stepwise multiple regression analysis was performed to determine the relationship between the independent (cognitive abilities and demographics) and dependent (individual slopes and R-squared values) variables. The regression analysis indicates some support for most of the predictor variables, especially predicting R-squared values. The 2-back task does not seem to contribute significantly to the interpretation of sonifications and auditory graphs. However, Raven's and many of the demographic variables do show predictive value for interpretation of auditory graphs.

Space, the final frontearcon: the identification of concurrently presented earcons in a synthetic spatialised auditory environment

Abstract: Two experiments which investigate the impact of spatialised presentation on the identification of concurrently presented earcons are described. The first experiment compared the identification of concurrently presented earcons based on the guidelines for individual earcon design and presentation of Brewster, Wright and Edwards [1] which were presented in spatially distinct locations, to the identification of non-spatially presented earcons which incorporated guidelines for concurrent presentation from McGookin and Brewster [2]. It was found that a significant increase in earcon identification occurred, as well as an increase in earcon register identification when earcons were spatially presented. The second experiment compared the identification of concurrently presented earcons based on the guidelines of Brewster, Wright and Edwards [1] which were presented in spatially distinct locations, to the identification of spatially presented earcons which incorporated guidelines for the presentation of concurrent earcons from McGookin and Brewster [2]. The incorporation of the concurrent earcon guidelines was found to significantly increase identification of the timbre attribute but did not significantly effect the overall identification of earcons.

Mobile spatial audio communication system

Abstract: As a first step in studying the application of the global positioning system (GPS) in a spatial audio communication system, an experimental platform was developed using commercial, off-the-shelf (COTS) GPS units: a Garmin Rino 120 (Rino) and a laptop. Software running on the laptop was developed to create a spatial-audio communication system based on GPS data and the two-way radio link on the Rino units. The audio communications were rendered spatially on headsets and perceptually tested for a number of users. The perceptual results and implementation issues are discussed in this paper. The experimental platform demonstrated that COTS GPS units can be used for a real-time GPS-based spatial audio communication system, but with some limitations.

Geometric shape detection with soundview

Abstract: We present the results of user studies that were performed on sighted people to test their ability to detect simple shapes with SoundView. SoundView is an experimental vision substitution system for the blind. Visual images are mapped onto a virtual surface with a fine-grained color dependent roughness texture. The user explores an image by moving a pointer device over the image which creates sounds. The current prototype uses a Wacom graphics tablet as a pointer device. The pointer acts like a virtual gramophone needle, and the sound produced depends on the motion as well as on the color of the area explored. An extension of SoundView also allows haptic feedback and we have compared the performance of users using auditory and/or haptic feedback.

The design of mixed -use, virtual auditory displays: recent findings with a dual-task paradigm

Abstract: In the third of an ongoing series of exploratory sound information display studies, we augmented a dual task with a mixed-use auditory display designed to provide relevant alert information for each task. The tasks entail a continuous tracking activity and a series of intermittent classification decisions that, in the present study, were presented on separate monitors that were roughly 90° apart. Using a 2-by-3 design that manipulated both the use of sound in each task and where sounds for the decision task were positioned, the following principal questions were addressed: Can tracking performance be improved with a varying auditory alert tied to error? To what degree do listeners use virtual auditory deixis as a cue for improving decision reaction times? Can a previous finding involving participants’ use of sound offsets (cessations) be repeated? And, last, are there performance consequences when auditory displays for separate tasks are combined? Respectively, we found that: Tracking performance as measured by RMS error was not improved and was apparently negatively affected by the use of our auditory design. Listener’s use of even limited virtual auditory deixis is robust, but it is probably also sensitive to the degree it is coincident with the location of corresponding visual stimuli in the task environment. On the basis of manually collected head movement data, listeners do make opportunistic use of sound offsets. And, finally, a significant interaction, as measured by average participant reaction time, was observed between the auditory display used for one task and the manipulation of the degree of auditory deixis encoded in the auditory display used for the other task in our paradigm.

Auditory feedback of human EEG for direct brain-computer communication

Abstract: The Thought-Translation-Device (TTD) is a Brain-ComputerInterface (BCI) that enables completely paralyzed patients to communicate by the use of their brain signals only. Selfregulation of brain signals (e.g. the slow cortical potentials) is achieved by a feedback training. Visual impairment of these patients asks for an auditory feedback mode. The TTD can be entirely operated by combined listening and mental activity. It provides auditory feedback of brain signals which can operate a verbal spelling interface. The extension POSER allows for sonified orchestral real-time feedback of multiple EEG parameters for the training of self-regulation. The properties of the system are reported and the results of some studies and experiments with auditory feedback are presented.

The Design and Evaluation of Auditory Access to Spreadsheets.

Abstract: This project is concerned with the development and evaluation of an integrated approach to sound-based access to spreadsheets. The primary target user population is visually impaired users, employing speech-based screen readers as their main means of accessing the data. Other users may benefit from the approach in situations where sound may provide an effective alternative or complementary means of exploring complex data relationships, when the visual interface being employed is too small to afford effective spreadsheet display or in monitoring situations where the visual attention of users must be employed elsewhere. The paper begins with an introduction to the use of spreadsheets by visually impaired users, including the current state of auditory interfaces as provided by commercial screen reader (SR) systems. The development of a prototype system to enhance speech-based access through the use of data sonification is described. The results of a number of detailed evaluations are discussed, drawn from a number of “real work” situations. The paper concludes with an examination of the issues arising from the evaluations, and a program of future research.

Interaction Sound Feedback in a Haptic Virtual Environment to Improve Motor Skill Acquisition

Abstract: This paper describes the concept and the realisation of a research prototype of a haptic environment that is enhanced with sound feedback to impart implicit knowledge and to teach motor skills to trainees. The sound feedback is to be understood as an additional feedback that provides information to the trainee about his or her actions in relation to objects in a virtual haptic environment. Although the setup simulates the interaction with haptic objects, the additional auditory feedback goes beyond the imitation of real sound behaviour and enables new ways to convey information about user interaction, i.e., movements in space with or without forces applied to tangible objects. The goal of the interaction with additional auditory feedback is to enrich the virtual environment to a full multimodal interaction environment, to support precise movements and to optimise the motor skill learning curve of trainees. The prototype is focused on computer based learning in the context of surgical training and it uses a virtual document planner that provides the appropriate instructions and tailored feedback for the human computer interaction through reasoning on the basis of an explicit knowledge representation. Keyword: auditory feedback, human computer interaction, implicit interaction, motor skill training, virtual document planner.

A generic semantically based design approach for spatial auditory computer displays

Abstract: This paper describes a design approach for creating generic computer user interfaces with spatial auditory displays It proposes a structured depiction process from formulating mode independent descriptions of user interfaces (UIs), to audio rendering methods for virtual environments As the key step in the process a semantic taxonomy of user interface content is proposed Finding semantic classications of UI entities corresponding to properties of auditory objects is the ultimate goal We beleive that this abstract approach detaches the process from visual paradigms and will reveal valuable insights into the representation of user interfaces in the auditory domain Possible ways of accessing operating systems for UI information are discussed along with an overview over common accessibility interfaces Critical aspects are highlighted for the composition of auditory UI entities in spatial environments and state-of-the-art techniques are presented for the creation of 3D audio Besides some possible elds of application, relevant utility and usability engineering aspects are discussed

Atmospherics/weather works: a multi-channel storm sonification project

Abstract: Atmospherics/Weather Works is an interdisciplinary project in the sonification of storms and other meteorological events generated directly from data produced by a highly detailed and physically accurate model of weather systems used for research and forecasting. This paper discusses the background, conception, and execution of a series of sonifications of a historical hurricane and winter snowstorm that resulted in several performances, stereo recordings, a public multi-channel spatialized sound installation, and an online interactive sound listening environment.

Simultaneous manipulation of parameters of auditory icons to convey direction, size, and distance : effects on recognition and interpretation

Abstract: Auditory icons – or environmental sounds – have the potential to convey information by non-verbal means quickly and accurately. In addition, human listeners are quick to determine many qualities of an auditory object, such as location, distance, size, and motion, from acoustics of the signal. An experiment tests these two coupled assumptions in a controlled laboratory context. Stimuli consisted of auditory icons “loaded” with information achieved through systematic manipulation of the acoustic parameters pitch, volume ramping, and reverberation. Sixty adult listeners were asked to recognize and describe four auditory icons wherein object size, distance and direction of motion were captured in the parameters of each 1-second sound. Participants were accurate at recognizing and interpreting the icons 70-80% of the time. Recognition rate was consistently high when participants responded to one, two or three parameters. However, recognition was significantly poorer when in response to all four parameters. There was a significant effect of icon type and parameter manipulation: dog bark was the most easily recognized icon, and the direction parameter interpreted most accurately. Implications of the findings for applied contexts are discussed.

The audio abacus: representing a wide range of values with accuracy and precision

Abstract: Point estimation is a relatively unexplored facet of sonfication. We present a new computer application, the Audio Abacus, designed to transform numbers into tones following the analogy of an abacus. As this is an entirely novel approach to sonifying exact data values, we have begun a systematic line of investigation into the application settings that work most effectively. Results are presented for an initial study. Users were able to perform relatively well with very little practice or training, boding well for this type of display. Further investigations are planned.