A method for detecting direction when interfacing with a computer program is provided. The method includes capturing an image presented in front of an image capture device. The image capture device has a capture location in a coordinate space. When a person is captured in the image, the method includes identifying a human head in the image and assigning the human head a head location in the coordinate space. The method also includes identifying an object held by the person in the image and assigning the object an object location in coordinate space. The method further includes identifying a relative position in coordinate space between the head location and the object location when viewed from the capture location. The relative position includes a dimension of depth. The method may be practiced on a computer system, such as one used in the gaming field.

Methods and systems for enabling depth and direction detection when interfacing with a computer program

An invention is provided for affording a real-time three-dimensional interactive environment using a three-dimensional camera. The invention includes obtaining two-dimensional data values for a plurality of pixels representing a physical scene, and obtaining a depth value for each pixel of the plurality of pixels using a depth sensing device. Each depth value indicates a distance from a physical object in the physical scene to the depth sensing device. At least one computer-generated virtual object is inserted into the scene, and an interaction between a physical object in the scene and the virtual object is detected based on coordinates of the virtual object and the obtained depth values.

System and method for providing a real-time three-dimensional interactive environment

Sound processing methods and apparatus are provided. A sound capture unit is configured to identify one or more sound sources. The sound capture unit generates data capable of being analyzed to determine a listening zone at which to process sound to the substantial exclusion of sounds outside the listening zone. Sound captured and processed for the listening zone may be used for interactivity with the computer program. The listening zone may be adjusted based on the location of a sound source. One or more listening zones may be pre-calibrated. The apparatus may optionally include an image capture unit configured to capture one or more image frames. The listening zone may be adjusted based on the image. A video game unit may be controlled by generating inertial, optical and/or acoustic signals with a controller and tracking a position and/or orientation of the controller using the inertial, acoustic and/or optical signal.

Methods and apparatus for targeted sound detection and characterization

Game interface tracks the position of one or more game controllers in 3-dimensional space using hybrid video capture and ultrasonic tracking system. The captured video information is used to identify a horizontal and vertical position for each controller within a capture area. The ultrasonic tracking system analyzes sound communications to determine the distances between the game system and each controller and to determine the distances among the controllers. The distances are then analyzed by the game interface to calculate the depths within the capture area for each controller.

Determination of controller three-dimensional location using image analysis and ultrasonic communication

A method for determining an intensity value of an interaction with a computer program is described. The method and device includes capturing an image of a capture zone, identifying an input object in the image, identifying an initial value of a parameter of the input object, capturing a second image of the capture zone, and identifying a second value of the parameter of the input object. The parameter identifies one or more of a shape, color, or brightness of the input object and is affected by human manipulation of the input object. The extent of change in the parameter is calculated, which is the difference between the second value and the first value. An activity input is provided to the computer program, the activity input including an intensity value representing the extent of change of the parameter. A method for detecting an intensity value from sound generating input objects, and a computer video game are also described. A game controller having LEDs, sound capture and generation, or an accelerometer is also described.

Computer image and audio processing of intensity and input devices for interfacing with a computer program

A method for reducing noise disturbance associated with an audio signal received through a microphone is provided. The method initiates with magnifying a noise disturbance of the audio signal relative to a remaining component of the audio signal. Then, a sampling rate of the audio signal is decreased. Next, an even order derivative is applied to the audio signal having the decreased sampling rate to define a detection signal. Then, the noise disturbance of the audio signal is adjusted according to a statistical average of the detection signal. A system capable of canceling disturbances associated with an audio signal, a video game controller, and an integrated circuit for reducing noise disturbances associated with an audio signal are included.

Method and apparatus to detect and remove audio disturbances

A method for reducing noise disturbance associated with an audio signal received through a microphone that is part of a video game controller, is provided. The method initiates with magnifying a noise disturbance of the audio signal relative to a remaining component of the audio signal. Then, a sampling rate of the audio signal is decreased. Next, an even order derivative is applied to the audio signal having the decreased sampling rate to define a detection signal. Then, the noise disturbance of the audio signal is adjusted according to a statistical average of the detection signal. A system capable of canceling disturbances associated with an audio signal, a video game controller, and an integrated circuit for reducing noise disturbances associated with an audio signal are included.

Xiadong Mao

Papers

Method and apparatus to detect and remove audio disturbances

Method to detect and remove audio disturbances from audio signals captured at video game controllers

Video game controller with noise canceling logic