Systems and methods are described for generating restricted depth of field depth maps. In one embodiment, an image processing pipeline application configures a processor to: determine a desired focal plane distance and a range of distances corresponding to a restricted depth of field for an image rendered from a reference viewpoint; generate a restricted depth of field depth map from the reference viewpoint using the set of images captured from different viewpoints, where depth estimation precision is higher for pixels with depth estimates within the range of distances corresponding to the restricted depth of field and lower for pixels with depth estimates outside of the range of distances corresponding to the restricted depth of field; and render a restricted depth of field image from the reference viewpoint using the set of images captured from different viewpoints and the restricted depth of field depth map.

Systems and Methods for Synthesizing Images from Image Data Captured by an Array Camera Using Restricted Depth of Field Depth Maps in which Depth Estimation Precision Varies

Methods and apparatus for capturing or generating images using multiple optical chains operating in parallel are described. Pixel values captured by individual optical chains corresponding to the same scene area are combined to provide an image with at least some of the benefits which would have been provided by capturing an image of the scene using a larger lens than that of the individual lenses of the optical chain modules. By using multiple optical chains in parallel at least some benefits normally obtained from using a large lens can be obtained without the need for a large lens. Furthermore in at least some embodiments, a wide dynamic range can be supported through the use of multiple sensors with the overall supported dynamic range being potentially larger than that of the individual sensors. Some lens and/or optical chain configurations are designed for use in small handheld devices, e.g., cell phones.

Methods and apparatus for capturing and/or processing images

Systems and methods in accordance with embodiments of the invention are configured to render images using light field image files containing an image synthesized from light field image data and metadata describing the image that includes a depth map. One embodiment of the invention includes a processor and memory containing a rendering application and a light field image file including an encoded image and metadata describing the encoded image, where the metadata comprises a depth map that specifies depths from the reference viewpoint for pixels in the encoded image. In addition, the rendering application configures the processor to: locate the encoded image within the light field image file; decode the encoded image; locate the metadata within the light field image file; and post process the decoded image by modifying the pixels based on the depths indicated within the depth map to create a rendered image.

Systems and methods for decoding light field image files

Described herein are methods and devices that employ a plurality of image sensors to capture a target image of a scene. As described, positioning at least one reflective or refractive surface near the plurality of image sensors enables the sensors to capture together an image of wider field of view and longer focal length than any sensor could capture individually by using the reflective or refractive surface to guide a portion of the image scene to each sensor. The different portions of the scene captured by the sensors may overlap, and may be aligned and cropped to generate the target image.

Multi-camera system using folded optics

Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.

Capturing and aligning three-dimensional scenes

A method of combining data from multiple sensors is disclosed. The method includes receiving lines of image data at an image processor having an input for a single camera. Each line of the image data includes first line data from a first image captured by a first camera and second line data from a second image captured by a second camera. The method also includes generating an output frame having a first section corresponding to line data of the first image and having a second section corresponding to line data of the second image. The first section and the second section are configured to be used to generate a three-dimensional (3D) image format or a 3D video format.

Combining data from multiple image sensors

Described are methods and apparatus for adjusting images of a stereoscopic image pair based on keypoint matches. The quality of the key point matches is first evaluated to determine whether the quality exceeds a keypoint quality threshold. If the quality level of the keypoint matches exceeds the threshold, the vertical disparity between the images of the stereoscopic image pair can be evaluated based on vertical disparity vectors between the keypoint matches. If the vertical disparity is below a threshold, no adjustment of the stereoscopic image pair may be performed. If the vertical disparity is above the threshold, an affine correction may compensate for pitch, roll, and scale differences between the images. A projective correction may compensate for yaw differences. The vertical disparity between the two images is then evaluated after the corrections to determine if additional adjustment should be performed.

Method and apparatus for calibrating an imaging device

Aspects relate to methods and systems for producing ultra-wide field of view images. In some embodiments, an image capture system for capturing wide field-of-view images comprises an aperture, a central camera positioned to receive light through the aperture, the center camera having an optical axis, a plurality of periphery cameras disposed beside the central camera and pointed towards a portion of the optical axis of the center camera, the plurality of cameras arranged around the center camera, and a plurality of extendible reflectors. The reflectors are configured to move from a first position to a second position and have a mirrored first surface that faces away from the optical axis of the center camera and a second black surface that faces towards the optical axis of the center camera, the plurality of periphery cameras arranged around the center camera.

Wide field of view array camera for hemispheric and spherical imaging

Certain aspects relate to systems and techniques for full well capacity extension. For example, a storage capacitor included in the pixel readout architecture can enable multiple charge dumps from a pixel in the analog domain, extending the full well capacity of the pixel. Further, multiple reads can be integrated in the digital domain using a memory, for example DRAM, in communication with the pixel readout architecture. This also can effectively multiply a small pixel's full well capacity. In some examples, multiple reads in the digital domain can be used to reduce, eliminate, or compensate for kTC noise in the pixel readout architecture.

Pixel readout architecture for full well capacity extension

An electronic device for parallel image processing using multiple processors is disclosed. The electronic device includes multiple image sensors for providing image data. The electronic device also includes multiple processors for processing segmented image data to produce processed segmented image data. Each processor is dedicated to one of the image sensors. A multiple processor interface is also included. The multiple processor interface maps the image data to the processors, segments the image data to produce the segmented image data and synchronizes the segmented image data to processor clock rates.

Sergiu Radu Goma

Papers

Combining data from multiple image sensors

Method and apparatus for calibrating an imaging device

Wide field of view array camera for hemispheric and spherical imaging

Pixel readout architecture for full well capacity extension

Parallel image processing using multiple processors