Showing papers on "Image sensor published in 2013"

Vision 20/20: Single photon counting x-ray detectors in medical imaging

Abstract: Photon counting detectors (PCDs) with energy discrimination capabilities have been developed for medical x-ray computed tomography (CT) and x-ray (XR) imaging. Using detection mechanisms that are completely different from the current energy integrating detectors and measuring the material information of the object to be imaged, these PCDs have the potential not only to improve the current CT and XR images, such as dose reduction, but also to open revolutionary novel applications such as molecular CT and XR imaging. The performance of PCDs is not flawless, however, and it seems extremely challenging to develop PCDs with close to ideal characteristics. In this paper, the authors offer our vision for the future of PCD-CT and PCD-XR with the review of the current status and the prediction of (1) detector technologies, (2) imaging technologies, (3) system technologies, and (4) potential clinical benefits with PCDs.

Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras

Abstract: Plenoptic cameras are gaining attention for their unique light gathering and post-capture processing capabilities. We describe a decoding, calibration and rectification procedure for lenselet-based plenoptic cameras appropriate for a range of computer vision applications. We derive a novel physically based 4D intrinsic matrix relating each recorded pixel to its corresponding ray in 3D space. We further propose a radial distortion model and a practical objective function based on ray reprojection. Our 15-parameter camera model is of much lower dimensionality than camera array models, and more closely represents the physics of lenselet-based cameras. Results include calibration of a commercially available camera using three calibration grid sizes over five datasets. Typical RMS ray reprojection errors are 0.0628, 0.105 and 0.363 mm for 3.61, 7.22 and 35.1 mm calibration grids, respectively. Rectification examples include calibration targets and real-world imagery.

A robust and modular multi-sensor fusion approach applied to MAV navigation

Abstract: It has been long known that fusing information from multiple sensors for robot navigation results in increased robustness and accuracy. However, accurate calibration of the sensor ensemble prior to deployment in the field as well as coping with sensor outages, different measurement rates and delays, render multi-sensor fusion a challenge. As a result, most often, systems do not exploit all the sensor information available in exchange for simplicity. For example, on a mission requiring transition of the robot from indoors to outdoors, it is the norm to ignore the Global Positioning System (GPS) signals which become freely available once outdoors and instead, rely only on sensor feeds (e.g., vision and laser) continuously available throughout the mission. Naturally, this comes at the expense of robustness and accuracy in real deployment. This paper presents a generic framework, dubbed MultiSensor-Fusion Extended Kalman Filter (MSF-EKF), able to process delayed, relative and absolute measurements from a theoretically unlimited number of different sensors and sensor types, while allowing self-calibration of the sensor-suite online. The modularity of MSF-EKF allows seamless handling of additional/lost sensor signals during operation while employing a state buffering scheme augmented with Iterated EKF (IEKF) updates to allow for efficient re-linearization of the prediction to get near optimal linearization points for both absolute and relative state updates. We demonstrate our approach in outdoor navigation experiments using a Micro Aerial Vehicle (MAV) equipped with a GPS receiver as well as visual, inertial, and pressure sensors.

Close-Range Photogrammetry and 3D Imaging

Abstract: This is the second edition of the established guide to close-range photogrammetry which uses accurate imaging techniques to analyse the three-dimensional shape of a wide range of manufactured and natural objects. After more than 20 years of use, close-range photogrammetry, now for the most part entirely digital, has become an accepted, powerful and readily available technique for engineers, scientists and others who wish to utilise images to make accurate 3D measurements of complex objects. Here they will find the photogrammetric fundamentals, details of system hardware and software, and broad range of real-world applications in order to achieve this. Following the introduction, the book provides fundamental mathematics covering subjects such as image orientation, digital imaging processing and 3D reconstruction methods, as well as a discussion of imaging technology, including targeting and illumination, and its implementation in hardware and software. It concludes with an overview of photogrammetric solutions for typical applications in engineering, manufacturing, medical science, architecture, archaeology and other fields.

Method and apparatus for compensating for fixed pattern noise in an imaging system

Abstract: The invention relates to a method and apparatus for configuring an imaging system to compensate for fixed pattern noise, variations in pixel values captured from an image sensor that vary according to a fixed pattern. In a method for configuring an imaging system for compensating an additive term component of fixed pattern noise, a pixel array is exposed to a scene of known radiance and an average white value is determined for each pixel of an array. Each average white value is compared to a predetermined reference value to determine a correction value for each pixel. In a method for configuring an imaging system for compensating both an additive and multiplicative component of fixed pattern noise, a pixel array is exposed to a first scene having a first known radiance, and frames of the scene are captured, and then the array is exposed to a second scene having a second known radiance and frames of the second scene are captured. After first and second average white values are determined for each pixel additive and multiplicative term correction values for each pixel are then determined by solving for a system of equations relating the first and second average white values and first and second scene radiances. Correction values can be utilized to correct pixel values of a frame of image data, and a decoding algorithm for attempting to decode a frame of image data including corrected pixel values can be activated.

Customer facing imaging systems and methods for obtaining images

Abstract: An imaging system for obtaining an image of a display of an electronic device presented by a presenter holding the electronic device includes a housing having an imaging subsystem disposed in the housing. The imaging subsystem comprising an image sensor array and an imaging assembly operative for focusing an image of the display on the electronic device onto the image sensor array. A support comprising a support surface is operable for allowing the presenter to hold the electronic device against the support surface so that the display on the electronic device is positioned towards the imaging system while the image of the display is obtained.

Chip on board based highly integrated imager

Abstract: An apparatus for use in decoding a bar code symbol may include an image sensor integrated circuit having a plurality of pixels, timing and control circuitry for controlling an image sensor, gain circuitry for controlling gain, and analog to digital conversion circuitry for conversion of an analog signal to a digital signal. The apparatus may also include a PCB for mounting the image sensor integrated circuit and light source bank. The connection between the image sensor integrated circuit and/or light source bank and the PCB characterized by a plurality of wires connecting a plurality of bond pads and a plurality of contact pads, where the wires, bond pads, and contact pads provide electrical input/output and mechanical connections between the image sensor integrated circuit and the PCB. The apparatus may be operative for processing image signals generated by the image sensor integrated circuit for attempting to decode the bar code symbol.

Imaging terminal having image sensor and lens assembly

Abstract: There is set forth herein an imaging terminal having and image sensor including an image sensor array having a plurality of pixels. In one embodiment the imaging terminal can include a lens assembly for focusing light on the image sensor array. In one embodiment the lens assembly is a variable setting lens assembly having a first lens setting at which the terminal has a first plane of optimum focus and a second lens setting at which the terminal has a second plane of optimum focus. The imaging terminal can execute one or more processes for determining an operating parameter of the imaging terminal.

An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications

Abstract: Robust velocity and position estimation at high update rates is crucial for mobile robot navigation. In recent years optical flow sensors based on computer mouse hardware chips have been shown to perform well on micro air vehicles. Since they require more light than present in typical indoor and outdoor low-light conditions, their practical use is limited. We present an open source and open hardware design 1 of an optical flow sensor based on a machine vision CMOS image sensor for indoor and outdoor applications with very high light sensitivity. Optical flow is estimated on an ARM Cortex M4 microcontroller in real-time at 250 Hz update rate. Angular rate compensation with a gyroscope and distance scaling using a ultrasonic sensor are performed onboard. The system is designed for further extension and adaption and shown in-flight on a micro air vehicle.

Indicia reading terminals and methods for decoding decodable indicia employing light field imaging

Abstract: A terminal for decoding decodable indicia includes a plenoptic imaging subsystem comprising an image sensor array and plenoptic imaging optics operable to project a plenoptic image of a space containing the decodable indicia onto the image sensor array, a hand held housing encapsulating a least a portion of the plenoptic imaging subsystem, a trigger for initiating operation of the plenoptic imaging subsystem to obtain plenoptic image data of the decodable indicia, and an illumination source for projecting illumination onto the decodable indicia; an aimer for projecting an aimer pattern onto the decodable indicia. The terminal is operable, responsive to detecting that the trigger has been actuated by an operator, to obtain plenoptic image data from the image sensor array, to obtain first rendered image data based on at least a portion of the plenoptic image data, and to attempt to decode the decodable indicia represented in the rendered image data.

LED and CMOS Image Sensor Based Optical Wireless Communication System for Automotive Applications

Abstract: An optical wireless communication (OWC) system based on a light-emitting-diode (LED) transmitter and a camera receiver has been developed for use in the automotive area. The automotive OWC system will require Mb/s-class data rates and the ability to quickly detect LEDs from an image. The key to achieving this is improvements to the capabilities of the image sensor mounted on the camera receiver. In this paper, we report on a novel OWC system equipped with an optical communication image sensor (OCI), which is newly developed using CMOS technology. To obtain higher transmission rates, the OCI employs a specialized “communication pixel (CPx)” capable of responding promptly to optical intensity variations. Furthermore, a new quick LED detection technique, based on a 1-bit flag image which only reacts to high-intensity objects, is formulated. The communication pixels, ordinary image pixels, and associated circuits (including 1-bit flag image output circuits) are then integrated into the OCI. This paper describes the design, fabrication, and capabilities of the OCI, as well as the development of the LED and image sensor based OWC system, which boasts a 20-Mb/s/pixel data rate without LED detection and a 15-Mb/s/pixel data rate with a 16.6-ms real-time LED detection.

CMOS Image Sensor With Per-Column ΣΔ ADC and Programmable Compressed Sensing

Abstract: A CMOS image sensor architecture with built-in single-shot compressed sensing is described. The image sensor employs a conventional 4-T pixel and per-column ΣΔ ADCs. The compressed sensing measurements are obtained via a column multiplexer that sequentially applies randomly selected pixel values to the input of each ΣΔ modulator. At the end of readout, each ADC outputs a quantized value of the average of the pixel values applied to its input. The image is recovered from the random linear measurements off-chip using numerical optimization algorithms. To demonstrate this architecture, a 256x256 pixel CMOS image sensor is fabricated in 0.15 μm CIS process. The sensor can operate in compressed sensing mode with compression ratio 1/4, 1/8, or 1/16 at 480, 960, or 1920 fps, respectively, or in normal capture mode with no compressed sensing at a maximum frame rate of 120 fps. Measurement results demonstrate capture in compressed sensing mode at roughly the same readout noise of 351 μVrms and power consumption of 96.2 mW of normal capture at 120 fps. This performance is achieved with only 1.8% die area overhead. Image reconstruction shows modest quality loss relative to normal capture and significantly higher image quality than downsampling.

Energy characterization and optimization of image sensing toward continuous mobile vision

Abstract: A major hurdle to frequently performing mobile computer vision tasks is the high power consumption of image sensing. In this work, we report the first publicly known experimental and analytical characterization of CMOS image sensors. We find that modern image sensors are not energy-proportional: energy per pixel is in fact inversely proportional to frame rate and resolution of image capture, and thus image sensor systems fail to provide an important principle of energy-aware system design: trading quality for energy efficiency. We reveal two energy-proportional mechanisms, supported by current image sensors but unused by mobile systems: (i) using an optimal clock frequency reduces the power up to 50% or 30% for low-quality single frame (photo) and sequential frame (video) capturing, respectively; (ii) by entering low-power standby mode between frames, an image sensor achieves almost constant energy per pixel for video capture at low frame rates, resulting in an additional 40% power reduction. We also propose architectural modifications to the image sensor that would further improve operational efficiency. Finally, we use computer vision benchmarks to show the performance and efficiency tradeoffs that can be achieved with existing image sensors. For image registration, a key primitive for image mosaicking and depth estimation, we can achieve a 96% success rate at 3 FPS and 0.1 MP resolution. At these quality metrics, an optimal clock frequency reduces image sensor power consumption by 36% and aggressive standby mode reduces power consumption by 95%.

Sensor-synchronized spectrally-structured-light imaging

Abstract: A smartphone is adapted for use as an imaging spectrometer, by synchronized pulsing of different LED light sources as different image frames are captured by the phone's CMOS image sensor. A particular implementation employs the CIE color matching functions, and/or their orthogonally transformed functions, to enable direct chromaticity capture. A great variety of other features and arrangements are also detailed.

Vehicle imaging system

Abstract: A vision system for a vehicle includes an image sensor and image processor. The image sensor is disposed at the vehicle and has a field of view exterior of the vehicle for capturing image data of a scene forward of the vehicle. The field of view encompasses at least a portion of a road surface ahead of and in the direction of travel of the vehicle. The image processor processes captured image data. Responsive to image processing of captured image data by the image processor, the vehicle vision system determines the presence of an animal on the road surface within the field of view. At least in part responsive to determination of the animal on the road surface within the field of view, the vehicle vision system at least one of (a) generates an alert and (b) controls the vehicle.

Color Imaging via Nearest Neighbor Hole Coupling in Plasmonic Color Filters Integrated onto a Complementary Metal-Oxide Semiconductor Image Sensor

Abstract: State-of-the-art CMOS imagers are composed of very small pixels, so it is critical for plasmonic imaging to understand the optical response of finite-size hole arrays and their coupling efficiency to CMOS image sensor pixels. Here, we demonstrate that the transmission spectra of finite-size hole arrays can be accurately described by only accounting for up to the second nearest-neighbor scattering-absorption interactions of hole pairs, thus making hole arrays appealing for close-packed color filters for imaging applications. Using this model, we find that the peak transmission efficiency of a square-shaped hole array with a triangular lattice reaches 90% that of an infinite array at an extent of ~6 × 6 μm^2, the smallest size array showing near-infinite array transmission properties. Finally, we experimentally validate our findings by investigating the transmission and imaging characteristics of a 360 × 320 pixel plasmonic color filter array composed of 5.6 × 5.6 μm^2 RGB color filters integrated onto a commercial black and white 1/2.8 in. CMOS image sensor, demonstrating full-color high resolution plasmonic imaging. Our results show good color fidelity with a 6-color-averaged color difference metric (ΔE) in the range of 16.6–19.3, after white balancing and color-matrix correcting raw images taken with f-numbers ranging from 1.8 to 16. The integrated peak filter transmission efficiencies are measured to be in the 50% range, with a FWHM of 200 nm for all three RGB filters, in good agreement with the spectral response of isolated unmounted color filters.

Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction

Abstract: Light-field imaging systems have got much attention recently as the next generation camera model. A light-field imaging system consists of three parts: data acquisition, manipulation, and application. Given an acquisition system, it is important to understand how a light-field camera converts from its raw image to its resulting refocused image. In this paper, using the Lytro camera as an example, we describe step-by-step procedures to calibrate a raw light-field image. In particular, we are interested in knowing the spatial and angular coordinates of the micro lens array and the resampling process for image reconstruction. Since Lytro uses a hexagonal arrangement of a micro lens image, additional treatments in calibration are required. After calibration, we analyze and compare the performances of several resampling methods for image reconstruction with and without calibration. Finally, a learning based interpolation method is proposed which demonstrates a higher quality image reconstruction than previous interpolation methods including a method used in Lytro software.

A 1/4-inch 8Mpixel back-illuminated stacked CMOS image sensor

Abstract: In recent years, cellphone cameras have come to require much more diversification and increased functionalities, due to the strong growth of the smartphone market. In addition to the image quality, speed, and pixel counts that conventional image sensors require, there is high demand for new functions that can respond to various photo-taking scenes. We developed a stacked CMOS image sensor (CIS), composed of conventional back-illuminated (BI) image-sensor technology and 65nm standard logic technology.

Vision-Based Displacement Sensor for Monitoring Dynamic Response Using Robust Object Search Algorithm

Abstract: This paper develops a vision-based displacement measurement system for remote monitoring of vibration of large-size structures such as bridges and buildings. The system consists of one or multiple video cameras and a notebook computer. With a telescopic lens, the camera placed at a stationary point away from a structure captures images of an object on the structure. The structural displacement is computed in real time through processing the captured images. A robust object search algorithm developed in this paper enables accurate measurement of the displacement by tracking existing features on the structure without requiring a conventional target panel to be installed on the structure. A sub-pixel technique is also proposed to further reduce measurement errors cost-effectively. The efficacy of the vision system in remote measurement of dynamic displacements was demonstrated through a shaking table test and a field experiment on a long-span bridge.

Multispectral imaging with vertical silicon nanowires

Abstract: Multispectral imaging is a powerful tool that extends the capabilities of the human eye. However, multispectral imaging systems generally are expensive and bulky, and multiple exposures are needed. Here, we report the demonstration of a compact multispectral imaging system that uses vertical silicon nanowires to realize a filter array. Multiple filter functions covering visible to near-infrared (NIR) wavelengths are simultaneously defined in a single lithography step using a single material (silicon). Nanowires are then etched and embedded into polydimethylsiloxane (PDMS), thereby realizing a device with eight filter functions. By attaching it to a monochrome silicon image sensor, we successfully realize an all-silicon multispectral imaging system. We demonstrate visible and NIR imaging. We show that the latter is highly sensitive to vegetation and furthermore enables imaging through objects opaque to the eye.

Low-dose phase contrast tomography with conventional x-ray sources

Abstract: Purpose: The edge illumination (EI) x-ray phase contrast imaging (XPCi) method has been recently further developed to perform tomographic and, thus, volumetric imaging. In this paper, the first tomographic EI XPCi images acquired with a conventional x-ray source at dose levels below that used for preclinical small animal imaging are presented. Methods: Two test objects, a biological sample and a custom-built phantom, were imaged with a laboratory-based EI XPCi setup in tomography mode. Tomographic maps that show the phase shift and attenuating properties of the object were reconstructed, and analyzed in terms of signal-to-noise ratio and quantitative accuracy. Dose measurements using thermoluminescence devices were performed. Results: The obtained images demonstrate that phase based imaging methods can provide superior results compared to attenuation based modalities for weakly attenuating samples also in 3D. Moreover, and, most importantly, they demonstrate the feasibility of low-dose imaging. In addition, the experimental results can be considered quantitative within the constraints imposed by polychromaticity. Conclusions: The results, together with the method's dose efficiency and compatibility with conventional x-ray sources, indicate that tomographic EI XPCi can become an important tool for the routine imaging of biomedical samples.

Multi-camera system using folded optics

Abstract: Described herein are methods and devices that employ a plurality of image sensors to capture a target image of a scene. As described, positioning at least one reflective or refractive surface near the plurality of image sensors enables the sensors to capture together an image of wider field of view and longer focal length than any sensor could capture individually by using the reflective or refractive surface to guide a portion of the image scene to each sensor. The different portions of the scene captured by the sensors may overlap, and may be aligned and cropped to generate the target image.

MonoFusion: Real-time 3D reconstruction of small scenes with a single web camera

Abstract: MonoFusion allows a user to build dense 3D reconstructions of their environment in real-time, utilizing only a single, off-the-shelf web camera as the input sensor. The camera could be one already available in a tablet, phone, or a standalone device. No additional input hardware is required. This removes the need for power intensive active sensors that do not work robustly in natural outdoor lighting. Using the input stream of the camera we first estimate the 6DoF camera pose using a sparse tracking method. These poses are then used for efficient dense stereo matching between the input frame and a key frame (extracted previously). The resulting dense depth maps are directly fused into a voxel-based implicit model (using a computationally inexpensive method) and surfaces are extracted per frame. The system is able to recover from tracking failures as well as filter out geometrically inconsistent noise from the 3D reconstruction. Our method is both simple to implement and efficient, making such systems even more accessible. This paper details the algorithmic components that make up our system and a GPU implementation of our approach. Qualitative results demonstrate high quality reconstructions even visually comparable to active depth sensor-based systems such as KinectFusion.

Efficient colour splitters for high-pixel-density image sensors

Abstract: Colour filters that split light by employing near-field interference effects instead of absorption provide enhanced signal levels for dense, small-pixel image sensors.

Undersampled frequency shift ON-OFF keying (UFSOOK) for camera communications (CamCom)

Abstract: Camera communications is the use of an image sensor to receive LED modulated data This paper shows how frequency shift ON-OFF keying - at frequencies that avoid flicker - can be used to encode bits while at the same time being successfully decoded by a low frame rate camera using subsampling This is done by selecting mark and space frequencies that are harmonics of the camera frame rate and then processing the subsampled aliased frequencies to decode bit data

Sensing and Recognizing Surface Textures Using a GelSight Sensor

Abstract: Sensing surface textures by touch is a valuable capability for robots Until recently it was difficult to build a compliant sensor with high sensitivity and high resolution The GelSight sensor is compliant and offers sensitivity and resolution exceeding that of the human fingertips This opens the possibility of measuring and recognizing highly detailed surface textures The GelSight sensor, when pressed against a surface, delivers a height map This can be treated as an image, and processed using the tools of visual texture analysis We have devised a simple yet effective texture recognition system based on local binary patterns, and enhanced it by the use of a multi-scale pyramid and a Hellinger distance metric We built a database with 40 classes of tactile textures using materials such as fabric, wood, and sandpaper Our system can correctly categorize materials from this database with high accuracy This suggests that the GelSight sensor can be useful for material recognition by robots

Line-based extrinsic calibration of range and image sensors

Abstract: Creating rich representations of environments requires integration of multiple sensing modalities with complementary characteristics such as range and imaging sensors. To precisely combine multisensory information, the rigid transformation between different sensor coordinate systems (i.e., extrinsic parameters) must be estimated. The majority of existing extrinsic calibration techniques require one or multiple planar calibration patterns (such as checkerboards) to be observed simultaneously from the range and imaging sensors. The main limitation of these approaches is that they require modifying the scene with artificial targets. In this paper, we present a novel algorithm for extrinsically calibrating a range sensor with respect to an image sensor with no requirement of external artificial targets. The proposed method exploits natural linear features in the scene to precisely determine the rigid transformation between the coordinate frames. First, a set of 3D lines (plane intersection and boundary line segments) are extracted from the point cloud, and a set of 2D line segments are extracted from the image. Correspondences between the 3D and 2D line segments are used as inputs to an optimization problem which requires jointly estimating the relative translation and rotation between the coordinate frames. The proposed method is not limited to any particular types or configurations of sensors. To demonstrate robustness, efficiency and generality of the presented algorithm, we include results using various sensor configurations.

Smart-phone based computational microscopy using multi-frame contact imaging on a fiber-optic array.

Abstract: We demonstrate a cellphone based contact microscopy platform, termed Contact Scope, which can image highly dense or connected samples in transmission mode Weighing approximately 76 grams, this portable and compact microscope is installed on the existing camera unit of a cellphone using an opto-mechanical add-on, where planar samples of interest are placed in contact with the top facet of a tapered fiber-optic array This glass-based tapered fiber array has ∼9 fold higher density of fiber optic cables on its top facet compared to the bottom one and is illuminated by an incoherent light source, eg, a simple light-emitting-diode (LED) The transmitted light pattern through the object is then sampled by this array of fiber optic cables, delivering a transmission image of the sample onto the other side of the taper, with ∼3× magnification in each direction This magnified image of the object, located at the bottom facet of the fiber array, is then projected onto the CMOS image sensor of the cellphone using two lenses While keeping the sample and the cellphone camera at a fixed position, the fiber-optic array is then manually rotated with discrete angular increments of eg, 1–2 degrees At each angular position of the fiber-optic array, contact images are captured using the cellphone camera, creating a sequence of transmission images for the same sample These multi-frame images are digitally fused together based on a shift-and-add algorithm through a custom-developed Android application running on the smart-phone, providing the final microscopic image of the sample, visualized through the screen of the phone This final computation step improves the resolution and also removes spatial artefacts that arise due to non-uniform sampling of the transmission intensity at the fiber optic array surface We validated the performance of this cellphone based Contact Scope by imaging resolution test charts and blood smears

Information communication device

Abstract: An information communication method includes: setting an exposure time of an image sensor to less than or equal to 1/2000 second so that, in an image obtained by capturing a subject by the image sensor, a stripe bright line parallel to a plurality of exposure lines included in the image sensor appears according to a change in luminance of the subject; obtaining the image including the stripe bright line parallel to the plurality of exposure lines by, using the set exposure time, starting exposure sequentially for the plurality of exposure lines each at a different time; and obtaining information by demodulating data according to, in a pattern of the bright line included in the obtained image, a brightness change in a direction perpendicular to the plurality of exposure lines.

Modeling the Performance of Single-Bit and Multi-Bit Quanta Image Sensors

Abstract: Imaging performance metrics of single-bit and multi-bit photo-electron-counting quanta image sensors (QIS) are analyzed using Poisson arrival statistics. Signal and noise as a function of exposure are determined. The D-log H characteristic of single-bit sensors including overexposure latitude is quantified. Linearity and dynamic range are also investigated. Read-noise-induced bit-error rate is analyzed and a read-noise target of less than 0.15 e-rms is suggested.