The invention relates to a computerized method for correlating a performance, in real time, to a score of music, and a machine based on that method. A score processor accepts a score which a user would like to play and converts it into a useable format. Performance input data is accepted by the input processor and the performance input data is correlated to the score on a note-by-note basis. An apparatus for performing this method includes an input processor that receives input and compares it to the expected score to determine whether an entire chord has been matched, and an output processor which receives a note match signal from the input processor and provides an output stream responsive to the match signals.

Method and apparatus for real-time correlation of a performance to a musical score

The present invention relates to a system and method for processing a video input signal providing for tracking a selected portion in a predefined audiovisual presentation and integrating selected user images into the selected portion of the predefined audiovisual presentation.

Image tracking and substitution system and methodology for audio-visual presentations

A video game adapter interface apparatus has a user input device and an associated video display. The user selects a distinguishable visual image representation for association into a video game's audiovisual presentation, such as where that user is identified. For example, color, size graphics or shape can be used to distinguish users. In one embodiment a digitized image of each user's face is used as the distinguishable representation. Alternatively, the user can create an original image or select one of a predetermined set of visual images as the user's identification for use in the video game audiovisual presentation. In a preferred video game embodiment, the video game adapter interface system is coupled to a video game system and a storage card containing User Images. The video game adapter interface system then integrates a selected User Image from the Storage Card into the video game's audiovisual presentation, and can also integrate the User Image into the video game play. For non-User Image video game software and for non-video game and audiovisual presentations, an Intercept Adapter Interface Systems permits the integration of User Images into the audiovisual presentation in place of and/or in addition to predefined character images otherwise present in the audiovisual presentation.

Image integration with replaceable content

An adapter interface apparatus has a user input device and an associated video display. The user selects a distinguishable visual image representation for association into an audiovisual presentation, such as where that user is identified. For example, color, size graphics or shape can be used to distinguish users. In one embodiment a digitized image of each user's face is used as the distinguishable representation. Alternatively, the user can create an original image or select one of a predetermined set of visual images as the user's identification for use in the audiovisual presentation. In one embodiment, the adapter interface is coupled to an audiovisual presentation system and a storage card containing User Images. The adapter interface system integrates a selected User Image from the Storage Card into the audiovisual presentation.

Image integration, mapping and linking system and methodology

Previous research has shown that listeners use the prosodic structure of utterances in a predictive fashion in sentence comprehension, to direct attention to accented words. Acoustically identical words spliced into sentence contexts arc responded to differently if the prosodic structure of the context is varied; when the preceding prosody indicates that the word will he accented, responses are faster than when the preceding prosody is inconsistent with accent occurring on that word. In the present series of experiments speech hybridisation techniques were first used to interchange the timing patterns within pairs of prosodic variants of utterances, independently of the pitch and intensity contours. The time-adjusted utterances could then serve as a basis lor the orthogonal manipulation of the three prosodic dimensions of pilch, intensity and rhythm. The overall pattern of results showed that when listeners use prosody to predict accent location, they do not simply rely on a single prosodic dimension, but exploit the interaction between pitch, intensity and rhythm. monitoring experiment, in which listeners are asked to respond as quickly as possible to the presence of a specified word-initial phoneme. In ( I ) , the target phoneme is / b / , so the target-bearing word is book. Targets on accented words are responded to faster than targets on unaccented words in this task. In Cutler's experiment, the target-bearing word itself was actually spliced out of both sentence contexts and replaced in each by identical copies of a neutral rendition of the same word. The result of this manipulation was a pair of sentences with acoustical!} identical target-bearing words, which were preceded by identical sequences of words; the only difference between the members of each pair was the prosody applied to the words preceding the target. In one case the prosodic contour in which the target-bearing word occurred was consistent with accent falling upon that word; in the other, it was consistent with the target-bearing word being unaccented. Under these conditions, the 'accented' targets still elicited faster responses than the 'unaccented' targets, and since the only relevant differences between the two sentences in each pair lay in the prosody, Culler concluded that listeners must have used cues in the prosody to direct their attention to the location where sentence accent would fall. Speakers place accent on the most important words in an utterance. Thus by finding accented words, listeners can efficiently locate the most central parts of a speaker's message. Previous studies have shown that listeners do indeed actively use sentence prosody to tell them where accented words arc going to occur. Cutler [2] produced pairs of sentences varying in prosodic contour. An example is (I): ( I ) (a) The couple had quarrelled over a BOOK they had read. (b) The couple had quarrelled over a book they hadn't even READ. Upper case represents sentence accent. In ( la) the main sentence accent falls on book, in ( Ib) on read. These sentences were used as materials in a phonemeProsody, however, is not a unitary phenomenon. The separate dimensions of rhythm, pitch and intensity all contribute to the prosodic structure of an utterance. Culler's experiment did not examine how listeners were exploiting prosody to predict accent, or whether any one prosodic dimension was more informative than others. Culler and Darwin [3] subsequently found that removing pilch information i.e. monotonising the sentences did not remove the accent effect; in monotonised spliced sentences like ( I ) the 'accented' targets are still responded to significantly faster than the 'unaccented' targets. From this, Culler and Darwin concluded that pilch information could not be a necessary component of the accent prediction effect. They speculated that no prosodic dimension might prove necessary for listeners to predict upcoming accents, but variation in any prosodic dimension might prove sufficient.

Components of prosodic effects in speech recognition

A number of applications exist in basic speech research for Dynamic Programming (DP) algorithms that can produce accurate time registration data for aligning one speech signal with a similar speech signal. In this paper, a further application for time-alignment algorithms is described, in which replacement dialogue for a film soundtrack may be automatically synchronized to reference dialogue recorded during filming. This is being carried out in a digital signal processing system that uses a DP algorithm capable of aligning utterances of indeterminate length accurately and efficiently in real-time. The main features of this system and the DP algorithm will be described.

P. Bloom

Papers

Use of dynamic programming for automatic synchronization of two similar speech signals