Preface The Effects of Acute, Chronic & Withdrawal from Chronic Ethanol on Emotional Learning Enhanced Odorant Preference Associative Learning in C. Elegans with Protein Kinase C-Interactive Protein (PKCI)/Hint1 Mutation Brain Derives Neurotrophic Factor (BDNF) & Adult Neurogenesis Neural Mechanisms for Expertise in Mental Imagery Language Processing Across Modalities: Insights from Bimodal Bilingualism Exploiting the Relationship of Natural Language & Computer Science: A Novel Theoretical Approach to Fairness Zero Power & Selflessness: What Meditation & Conscious Perception Have in Common Early Memories of Children & Adults: Implications for Infantile Amnesia APOE ? 4 Allele & Alheimer's Disease: Perspectives on AD Pathogenesis & Therapy The "Egocentric" Americans? Long-Term Memory for Public Events in Five Countries Ecological Validity & the Study of Procedural & Episodic Memory Function in Songbirds Need Dendritic Spines? There's an APP for That Cholesterol & Neuronal Susceptibility to Beta-Amyloid Toxicity Index.

Trends in cognitive sciences

Low illumination underwater light field images reconstruction using deep convolutional neural networks

Existing RGB-D Salient Object Detection (SOD) methods take advantage of depth cues to improve the detection accuracy, while pay insufficient attention to the quality of depth information. In practice, a depth map is often with uneven quality and sometimes suffers from distractors, due to various factors in the acquisition procedure. In this article, to mitigate distractors in depth maps and highlight salient objects in RGB images, we propose a Hierarchical Alternate Interactions Network (HAINet) for RGB-D SOD. Specifically, HAINet consists of three key stages: feature encoding, cross-modal alternate interaction, and saliency reasoning. The main innovation in HAINet is the Hierarchical Alternate Interaction Module (HAIM), which plays a key role in the second stage for cross-modal feature interaction. HAIM first uses RGB features to filter distractors in depth features, and then the purified depth features are exploited to enhance RGB features in turn. The alternate RGB-depth-RGB interaction proceeds in a hierarchical manner, which progressively integrates local and global contexts within a single feature scale. In addition, we adopt a hybrid loss function to facilitate the training of HAINet. Extensive experiments on seven datasets demonstrate that our HAINet not only achieves competitive performance as compared with 19 relevant state-of-the-art methods, but also reaches a real-time processing speed of 43 fps on a single NVIDIA Titan X GPU. The code and results of our method are available at https://github.com/MathLee/HAINet .

Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection

Salient object detection, which simulates human visual perception in locating the most significant object(s) in a scene, has been widely applied to various computer vision tasks. Now, the advent of depth sensors means that depth maps can easily be captured; this additional spatial information can boost the performance of salient object detection. Although various RGB-D based salient object detection models with promising performance have been proposed over the past several years, an in-depth understanding of these models and the challenges in this field remains lacking. In this paper, we provide a comprehensive survey of RGB-D based salient object detection models from various perspectives, and review related benchmark datasets in detail. Further, as light fields can also provide depth maps, we review salient object detection models and popular benchmark datasets from this domain too. Moreover, to investigate the ability of existing models to detect salient objects, we have carried out a comprehensive attribute-based evaluation of several representative RGB-D based salient object detection models. Finally, we discuss several challenges and open directions of RGB-D based salient object detection for future research. All collected models, benchmark datasets, datasets constructed for attribute-based evaluation, and related code are publicly available at https://github.com/taozh2017/RGBD-SODsurvey.

/pdf/rgb-d-salient-object-detection-a-survey-gv47y4gjy6.pdf

RGB-D salient object detection: A survey.

Salient object detection (SOD), which simulates the human visual perception system to locate the most attractive object(s) in a scene, has been widely applied to various computer vision tasks. Now, with the advent of depth sensors, depth maps with affluent spatial information that can be beneficial in boosting the performance of SOD, can easily be captured. Although various RGB-D based SOD models with promising performance have been proposed over the past several years, an in-depth understanding of these models and challenges in this topic remains lacking. In this paper, we provide a comprehensive survey of RGB-D based SOD models from various perspectives, and review related benchmark datasets in detail. Further, considering that the light field can also provide depth maps, we review SOD models and popular benchmark datasets from this domain as well. Moreover, to investigate the SOD ability of existing models, we carry out a comprehensive evaluation, as well as attribute-based evaluation of several representative RGB-D based SOD models. Finally, we discuss several challenges and open directions of RGB-D based SOD for future research. All collected models, benchmark datasets, source code links, datasets constructed for attribute-based evaluation, and codes for evaluation will be made publicly available at this https URL

RGB-D Salient Object Detection: A Survey

Although the light field has been recently recognized helpful in saliency detection, it is not comprehensively explored yet. In this work, we propose a new saliency detection model with light field data. The idea behind the proposed model originates from the following observations. (1) People can distinguish regions at different depth levels via adjusting the focus of eyes. Similarly, a light field image can generate a set of focal slices focusing at different depth levels, which suggests that a background can be weighted by selecting the corresponding slice. We show that background priors encoded by light field focusness have advantages in eliminating background distraction and enhancing the saliency by weighting the light field contrast. (2) Regions at closer depth ranges tend to be salient, while far in the distance mostly belong to the backgrounds. We show that foreground objects can be easily separated from similar or cluttered backgrounds by exploiting their light field depth. Extensive evaluations on the recently introduced Light Field Saliency Dataset (LFSD) [Li et al., 2014], including studies of different light field cues and comparisons with Li et al.'s method (the only reported light field saliency detection approach to our knowledge) and the 2D/3D state-of-the-art approaches extended with light field depth/focusness information, show that the investigated light field properties are complementary with each other and lead to improvements on 2D/3D models, and our approach produces superior results in comparison with the state-of-the-art.

Saliency detection with a deeper investigation of light field

Light field saliency vs. 2D saliency

The invention discloses a method used for measuring the pose of a non-cooperative target based on a complete light field camera The method is applied in the non-cooperative target containing a star arrow docking ring The star arrow docking ring on the non-cooperative target is a circle and is characterized in that step 1, the light field image of the non-cooperative target with circular features is obtained by adopting the compete light field camera; step 2, surface normal vectors, and the coordinates of the circular center of the circular features are calculated in the main lens coordinate system of the complete light field camera; step 3, false solutions are removed; step 4, the pose of the non-cooperative target is acquired According to the invention, the false solutions based on the monocular vision pose estimations of the circle can be effectively removed, so that the pose of the non-cooperative target which meets accuracy requirements can be obtained

Method used for measuring pose of non-cooperative target based on complete light field camera

Light field cameras record not only the spatial information of observed scenes but also the directions of all incoming light rays. The spatial and angular information implicitly contain geometrical characteristics such as multi-view or epipolar geometry, which can be exploited to improve the performance of depth estimation. An Epipolar Plane Image (EPI), the unique 2D spatial-angular slice of the light field, contains patterns of oriented lines. The slope of these lines is associated with the disparity. Benefiting from this property of EPIs, some representative methods estimate depth maps by analyzing the disparity of each line in EPIs. However, these methods often extract the optimal slope of the lines from EPIs while ignoring the relationship between neighboring pixels, which leads to inaccurate depth map predictions. Based on the observation that an oriented line and its neighboring pixels in an EPI share a similar linear structure, we propose an end-to-end fully convolutional network (FCN) to estimate the depth value of the intersection point on the horizontal and vertical EPIs. Specifically, we present a new feature-extraction module, called Oriented Relation Module (ORM), that constructs the relationship between the line orientations. To facilitate training, we also propose a refocusing-based data augmentation method to obtain different slopes from EPIs of the same scene point. Extensive experiments verify the efficacy of learning relations and show that our approach is competitive to other state-of-the-art methods. The code and the trained models are available at this https URL.

EPI-based Oriented Relation Networks for Light Field Depth Estimation

The present invention discloses a multi-thread significance method based on a light field camera. The method comprises: 1, employing a Lytro light field camera to collect five-dimensional light field data in a scene and obtains light field full-focusing images, depth images, a focus stack image sequence and a multi-view image sequence; 2, performing super-pixel segmentation of the full-focusing images, and extracting the color, the position and the depth difference characteristics between the pixel pairs on different light field images; 3, respectively extracting the average optical flow characteristics of an adjacent focal plane image and an adjacent visual angle image, and calculating the focus flow difference characteristics and the visual angle flow difference characteristics; 4, performing weighting and summation of the color, the depth, the focus flow and the visual angle flow difference characteristics, and taking the position difference characteristic as a weight, and obtaining the original substantial results of multiple clues; and 5, optimizing the original substantial results, and obtaining the multiple-clue significance of the light field camera. The multi-thread significance method based on a light field camera is able to solve the defect that the current two-dimensional and three-dimensional significance extraction method cannot obtain and use the vision multiple clues so as to effectively improve the extraction precision of the image significance in the complex changeable scene.

Xudong Zhang

Papers

Saliency detection with a deeper investigation of light field

Light field saliency vs. 2D saliency

Method used for measuring pose of non-cooperative target based on complete light field camera

EPI-based Oriented Relation Networks for Light Field Depth Estimation

Multi-thread significance method based on light field camera