A depth camera system uses a structured light illuminator and multiple sensors such as infrared light detectors, such as in a system which tracks the motion of a user in a field of view. One sensor can be optimized for shorter range detection while another sensor is optimized for longer range detection. The sensors can have a different baseline distance from the illuminator, as well as a different spatial resolution, exposure time and sensitivity. In one approach, depth values are obtained from each sensor by matching to the structured light pattern, and the depth values are merged to obtain a final depth map which is provided as an input to an application. The merging can involve unweighted averaging, weighted averaging, accuracy measures and/or confidence measures. In another approach, additional depth values which are included in the merging are obtained using stereoscopic matching among pixel data of the sensors.

Depth camera based on structured light and stereo vision

A device for processing data includes a first input port for receiving color image data from a first image sensor and a second input port for receiving depth-related image data from a second image sensor. Processing circuitry generates a depth map using the depth-related image data. At least one output port conveys the depth map and the color image data to a host computer.

Integrated processor for 3D mapping

Imaging apparatus includes an illumination subassembly, which is configured to project onto an object a pattern of monochromatic optical radiation in a given wavelength band. An imaging subassembly includes an image sensor, which is configured both to capture a first, monochromatic image of the pattern on the object by receiving the monochromatic optical radiation reflected from the object and to capture a second, color image of the object by receiving polychromatic optical radiation, and to output first and second image signals responsively to the first and second images, respectively. A processor is configured to process the first and second signals so as to generate and output a depth map of the object in registration with the color image.

Three-dimensional mapping and imaging

Mapping apparatus includes a transmitter, which emits a beam comprising pulses of light, and a scanner, which is configured to scan the beam, within a predefined scan range, over a scene. A receiver receives the light reflected from the scene and to generate an output indicative of a time of flight of the pulses to and from points in the scene. A processor is coupled to control the scanner so as to cause the beam to scan over a selected window within the scan range and to process the output of the receiver so as to generate a 3D map of a part of the scene that is within the selected window.

Scanning depth engine

A system that incorporates teachings of the exemplary embodiments may include, for example, means for generating a disparity map based on a depth map, means for determining accuracy of pixels in the depth map where the determining means identifies the pixels as either accurate or inaccurate based on a confidence map and the disparity map, and means for providing an adjusted depth map where the providing means adjusts inaccurate pixels of the depth map using a cost function associated with the inaccurate pixels. Other embodiments are disclosed.

Apparatus and method for providing three dimensional media content

A method for depth mapping includes projecting a pattern of optical radiation onto an object. A first image of the pattern on the object is captured using a first image sensor, and this image is processed to generate pattern-based depth data with respect to the object. A second image of the object is captured using a second image sensor, and the second image is processed together with another image to generate stereoscopic depth data with respect to the object. The pattern-based depth data is combined with the stereoscopic depth data to create a depth map of the object.

Depth Mapping Based on Pattern Matching and Stereoscopic Information

An optical apparatus includes an image sensor, which outputs a signal responsively to an optical image formed on the image sensor. Objective optics focus optical radiation from an object so as to form the optical image on the image sensor, while superimposing a characteristic structure on the optical image. A processor processes the signal so as to generate an electronic image of the object while detecting, responsively to the characteristic structure, a deviation in an alignment of the objective optics relative to the image sensor.

Automatic detection of lens deviations

A method for projection includes projecting a pattern of structured light with a given average intensity onto a scene. A sequence of images is captured of the scene while projecting the pattern. At least one captured image in the sequence is processed in order to extract a depth map of the scene. A condition is identified in the depth map indicative of a fault in projection of the pattern. Responsively to the identified condition, the average intensity of the projection of the pattern is reduced.

Monitoring DOE performance using software scene evaluation

A method for depth mapping includes acquiring first depth data with respect to an object using a first depth mapping technique and providing first candidate depth coordinates for a plurality of pixels, and acquiring second depth data with respect to the object using a second depth mapping technique, different from the first depth mapping technique, and providing second candidate depth coordinates for the plurality of pixels A weighted voting process is applied to the first and second depth data in order to select one of the candidate depth coordinates at each pixel A depth map of the object is output, including the selected one of the candidate depth coordinates at each pixel

Multi-modal depth mapping

L'invention porte sur un procede pour la cartographie de profondeur qui comprend la projection d'un motif de rayonnement optique sur un objet (22). Une premiere image du motif sur l'objet est capturee a l'aide d'un premier capteur d'image (40), et cette image est traitee pour generer des donnees de profondeur basees sur un motif par rapport a l'objet. Une seconde image de l'objet est capturee a l'aide d'un second capteur d'image (46), et la seconde image est traitee avec une autre image pour generer des donnees de profondeur stereoscopiques par rapport a l'objet. Des donnees de profondeur basees sur le motif sont combinees aux donnees de profondeur stereoscopiques pour creer une carte de profondeur de l'objet.

Ronen Deitch

Papers

Depth Mapping Based on Pattern Matching and Stereoscopic Information

Automatic detection of lens deviations

Monitoring DOE performance using software scene evaluation

Multi-modal depth mapping

Cartographie de profondeur basée sur une correspondance de motifs et des informations stéréoscopiques