This paper investigates two fundamental problems in computer vision: contour detection and image segmentation. We present state-of-the-art algorithms for both of these tasks. Our contour detector combines multiple local cues into a globalization framework based on spectral clustering. Our segmentation algorithm consists of generic machinery for transforming the output of any contour detector into a hierarchical region tree. In this manner, we reduce the problem of image segmentation to that of contour detection. Extensive experimental evaluation demonstrates that both our contour detection and segmentation methods significantly outperform competing algorithms. The automatically generated hierarchical segmentations can be interactively refined by user-specified annotations. Computation at multiple image resolutions provides a means of coupling our system to recognition applications.

/pdf/contour-detection-and-hierarchical-image-segmentation-2ejknw01zo.pdf

Contour Detection and Hierarchical Image Segmentation

Cell phones and other portable devices are equipped with a variety of technologies by which existing functionality can be improved, and new functionality can be provided. Some relate to visual search capabilities, and determining appropriate actions responsive to different image inputs. Others relate to processing of image data. Still others concern metadata generation, processing, and representation. Yet others relate to coping with fixed focus limitations of cell phone cameras, e.g., in reading digital watermark data. Still others concern user interface improvements. A great number of other features and arrangements are also detailed.

Methods and Systems for Content Processing

A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone's camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user's apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Intuitive computing methods and systems

Automated nuclear detection is a critical step for a number of computer assisted pathology related image analysis algorithms such as for automated grading of breast cancer tissue specimens. The Nottingham Histologic Score system is highly correlated with the shape and appearance of breast cancer nuclei in histopathological images. However, automated nucleus detection is complicated by 1) the large number of nuclei and the size of high resolution digitized pathology images, and 2) the variability in size, shape, appearance, and texture of the individual nuclei. Recently there has been interest in the application of “Deep Learning” strategies for classification and analysis of big image data. Histopathology, given its size and complexity, represents an excellent use case for application of deep learning strategies. In this paper, a Stacked Sparse Autoencoder (SSAE), an instance of a deep learning strategy, is presented for efficient nuclei detection on high-resolution histopathological images of breast cancer. The SSAE learns high-level features from just pixel intensities alone in order to identify distinguishing features of nuclei. A sliding window operation is applied to each image in order to represent image patches via high-level features obtained via the auto-encoder, which are then subsequently fed to a classifier which categorizes each image patch as nuclear or non-nuclear. Across a cohort of 500 histopathological images (2200 $\times$ 2200) and approximately 3500 manually segmented individual nuclei serving as the groundtruth, SSAE was shown to have an improved F-measure 84.49% and an average area under Precision-Recall curve (AveP) 78.83%. The SSAE approach also out-performed nine other state of the art nuclear detection strategies.

/pdf/stacked-sparse-autoencoder-ssae-for-nuclei-detection-on-3ft2os6ho7.pdf

Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images

Automatic segmentation of cell nuclei is an essential step in image cytometry and histometry. Despite substantial progress, there is a need to improve accuracy, speed, level of automation, and adaptability to new applications. This paper presents a robust and accurate novel method for segmenting cell nuclei using a combination of ideas. The image foreground is extracted automatically using a graph-cuts-based binarization. Next, nuclear seed points are detected by a novel method combining multiscale Laplacian-of-Gaussian filtering constrained by distance-map-based adaptive scale selection. These points are used to perform an initial segmentation that is refined using a second graph-cuts-based algorithm incorporating the method of alpha expansions and graph coloring to reduce computational complexity. Nuclear segmentation results were manually validated over 25 representative images (15 in vitro images and 10 in vivo images, containing more than 7400 nuclei) drawn from diverse cancer histopathology studies, and four types of segmentation errors were investigated. The overall accuracy of the proposed segmentation algorithm exceeded 86%. The accuracy was found to exceed 94% when only over- and undersegmentation errors were considered. The confounding image characteristics that led to most detection/segmentation errors were high cell density, high degree of clustering, poor image contrast and noisy background, damaged/irregular nuclei, and poor edge information. We present an efficient semiautomated approach to editing automated segmentation results that requires two mouse clicks per operation.

Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images

Embodiments programmatically analyze each of a plurality of images in order to determine one or more visual characteristics about an item shown in each of the plurality of images. Data is stored corresponding to the one or more visual characteristics. An interface in is provided for which a user is able to specify one or more search criteria. In response to receiving the one or more search criteria, a search operation is performed to identify one or more items that have a visual characteristic that satisfies at least some of the one or more search criteria.

System and method for enabling image recognition and searching of images

Embodiments enable searching of portions of objects in images, including programmatically analyzing each image in a collection in order to determine image data that, for individual images in the collection, represents one or more visual characteristics of a portion of an object shown in that image. A user is enabled to specify one or more search criteria that includes image data, and a search result may be determined based on one or more images in the collection that show a corresponding object that has a portion that satisfies a threshold. The threshold is defined at least in part by the one or more search criteria.

System and method for search portions of objects in images and features thereof

Embodiments described herein provide for a system for creating a data collection of recognized images. The system includes an image analysis module that is configured to programmatically analyze individual images in a collection of images in order to determine information about each image in the collection. The system may also include a manual interface that is configured to (i) interface with one or more human editors, and (ii) displays a plurality of panels concurrently. Individual panels may be provided for one or more analyzed images, and individual panels may be configured to display information that is at least indicative of the one or more images of that panel and/or of the information determined from the one or more images. Additionally, the manual interface enables the one or more human editors to view the plurality of panels concurrently and to interact with each of the plurality of panels in order to correct or remove any information that is incorrectly determined from the image of that panel.

System and method for enabling image searching using manual enrichment, classification, and/or segmentation

Purpose: To develop an automated tool that provides reliable, consistent, and accurate results for counting cell nuclei in tissue sections. Methods: We propose a novel method based on an image processing algorithm to analyze large sets of digital micrographs. The nucleus detector design is based on a Laplacian of Gaussian filter. We use the leave-one-out cross validation method for estimating the generalization error, which is then used to choose the model and parameters of the proposed nucleus detector with both fluorescent and dye stained images. We also evaluate the performance of a nucleus detector by comparing the results with manual counts. Results: When our nucleus detector is applied to previously unanalyzed images of feline retina, it correctly counts nuclei within the outer nuclear layer (ONL) with an average error of 3.67% ranging from 0 to 6.07%, and nuclei within the inner nuclear layer (INL) with an average error of 8.55% ranging from 0 to 13.76%. Our approach accurately identifies the location of cell bodies. Even though we have a relatively large error in the INL due to the large intra-observer variation, both manual counting and nucleus detector result in the same conclusion. This is the first time that cell death in the INL in response to retinal detachment is analyzed quantitatively. We also test the proposed tool with various images and show that it is applicable to a wide range of image types with nuclei varying in size and staining intensity. Conclusions: The proposed method is simple and reliable. It also has widespread applicability to a variety of sample preparation and imaging methods. Our approach will be immediately useful in quantifying cell number in large sets of digital micrographs and from high-throughput imaging. The tool is available as a plug-in for Image J.

/pdf/automated-tool-for-the-detection-of-cell-nuclei-in-digital-4pwd0fuo19.pdf

Automated tool for the detection of cell nuclei in digital microscopic images: application to retinal images.

We present an efficient probabilistic method for identity recognition in personal photo albums. Personal photos are usually taken under uncontrolled conditions -the captured faces exhibit significant variations in pose, expression and illumination that limit the success of traditional face recognition algorithms. We show how to improve recognition rates by incorporating additional cues present in personal photo collections, such as clothing appearance and information about when the photo was taken. This is done by constructing a Markov random field (MRF) that effectively combines all available contextual cues in a principled recognition framework. Performing inference in the MRF produces markedly improved recognition results in a challenging dataset consisting of the personal photo collections of multiple people. At the same time, the computational cost of our approach remains comparable to that of standard face recognition approaches.

/pdf/contextual-identity-recognition-in-personal-photo-albums-5dm5rqtyo3.pdf

Baris Sumengen

Papers

System and method for enabling image recognition and searching of images

System and method for search portions of objects in images and features thereof

System and method for enabling image searching using manual enrichment, classification, and/or segmentation

Automated tool for the detection of cell nuclei in digital microscopic images: application to retinal images.

Contextual Identity Recognition in Personal Photo Albums