The analysis of medical images has been woven into the fabric of the pattern analysis and machine intelligence (PAMI) community since the earliest days of these Transactions. Initially, the efforts in this area were seen as applying pattern analysis and computer vision techniques to another interesting dataset. However, over the last two to three decades, the unique nature of the problems presented within this area of study have led to the development of a new discipline in its own right. Examples of these include: the types of image information that are acquired, the fully three-dimensional image data, the nonrigid nature of object motion and deformation, and the statistical variation of both the underlying normal and abnormal ground truth. In this paper, we look at progress in the field over the last 20 years and suggest some of the challenges that remain for the years to come.

https://hal.inria.fr/inria-00615100/document

Medical image analysis: progress over two decades and the challenges ahead

https://bio3d.colorado.edu/SerialEM/serialEMJSB.pdf

Automated electron microscope tomography using robust prediction of specimen movements.

中枢神経系疾患の治療は正常細胞（ニューロン）の機能維持を目的とするが，脳血管障害のように機能障害の原因が細胞の死滅に基づくことは多い．一方，脳腫瘍の治療においては薬物療法や放射線療法といった腫瘍細胞の死滅を目標とするものが大きな位置を占める．いずれの場合にも，細胞死の機序を理解することは各種病態や治療法の理解のうえで重要である．現在のところ最も研究の進んでいる細胞死の型はアポトーシスである．そのなかで重要な位置を占めるミトコンドリアにおける反応および抗アポトーシス因子について概要を紹介する．

Neuroscience 細胞死：最近の知見

MUCKE aims to mine a large volume of images, to structure them conceptually and to use this conceptual structuring in order to improve large-scale image retrieval. The last decade witnessed important progress concerning low-level image representations. However, there are a number problems which need to be solved in order to unleash the full potential of image mining in applications. The central problem with low-level representations is the mismatch between them and the human interpretation of image content. This problem can be instantiated, for instance, by the incapability of existing descriptors to capture spatial relationships between the concepts represented or by their incapability to convey an explanation of why two images are similar in a content-based image retrieval framework. We start by assessing existing local descriptors for image classification and by proposing to use co-occurrence matrices to better capture spatial relationships in images. The main focus in MUCKE is on cleaning large scale Web image corpora and on proposing image representations which are closer to the human interpretation of images. Consequently, we introduce methods which tackle these two problems and compare results to state of the art methods. Note: some aspects of this deliverable are withheld at this time as they are pending review. Please contact the authors for a preview.

http://ieeexplore.ieee.org/iel2/4524/12820/00592460.pdf

Image Processing

http://www.cvs.rochester.edu/knill_lab/papers/35_TINS_2004.pdf

The Bayesian brain: the role of uncertainty in neural coding and computation

We have been developing the ideal observer formalism for sonography, which is based on the best-possible diagnostic performance. The ideal performance was compared to that of trained human observers to estimate the visual efficiency for discriminating lesion features. We find that humans are generally less than 10% efficient at accessing visual information essential for breast cancer diagnosis. In seeking ways to improve this process, we must first establish a connection between standard ROC observer metrics and instrument properties used in system design. In radiography, that relationship is made through the lesion signal-to-noise ratio SNRI. SNRI2, which describes task information, is simply related to contrast and spatial resolutions and noise power. Those relations break down for sonography due to the quadratic form of the ideal observer. Our goal in this paper is to establish a rigorous connection between ideal performance and engineering design metrics, which has directly applications for sonographic system design and optimization.

Detectability index describes the information conveyed by sonographic images

The footsteps of attention in the Posner paradigm revealed by classification images

Researchers studying human and computer vision have found description and construction of these systems greatly aided
by analysis of the statistical properties of naturally occurring scenes. More specifically, it has been found that receptive
fields with directional selectivity and bandwidth properties similar to mammalian visual systems are more closely
matched to the statistics of natural scenes. It is argued that this allows for sparse representation of the independent
components of natural images [Olshausen and Field, Nature, 1996]. These theories have important implications for
medical image perception. For example, will a system that is designed to represent the independent components of
natural scenes, where objects occlude one another and illumination is typically reflected, be appropriate for X-ray
imaging, where features superimpose on one another and illumination is transmissive?
In this research we begin to examine these issues by evaluating higher-order statistical properties of breast images from
X-ray projection mammography (PM) and dedicated breast computed tomography (bCT). We evaluate kurtosis in
responses of octave bandwidth Gabor filters applied to PM and to coronal slices of bCT scans. We find that kurtosis in
PM rises and quickly saturates for filter center frequencies with an average value above 0.95. By contrast, kurtosis in
bCT peaks near 0.20 cyc/mm with kurtosis of approximately 2. Our findings suggest that the human visual system may
be tuned to represent breast tissue more effectively in bCT over a specific range of spatial frequencies.

Higher-order scene statistics of breast images

We developed a foveated search model based on the channelized Hotelling observer (FCHO) that processes each image in parallel with Gabor channels which center frequencies decrease with retinal eccentricity (distance away from the fovea). The FCHO model is based on how radiologists read 3D volumes by scrolling through 2D slices. The model includes a search component that explores the image by making eye movements guided by peripheral processing and a slice scroll component that changes the current slice for the 3D images. We designed an experiment consisting of a free search in 2D and 3D noise filtered with power-law statistics matching mammograms. Two different targets were embedded in this background: one resembling a microcalcification and another one resembling a mass. Unlike traditional model observers (chanellized Hotelling and non-prewhitening matched filter with an eye filter), performance for the FCHO search model for both 2D and 3D search tasks showed similar dissociations as that of human observers. Detectability of a microcalcification-like target was significantly higher than for a mass-like target in 2D search but comparable or lower in 3D search. Together, our results suggest that 3D search tasks will require more computationally complex 3D search models that take into account the foveated properties of the visual system and eye movements to predict human observer performance in clinically relevant tasks.

A foveated channelized Hotelling search model predicts dissociations in human performance in 2D and 3D images

Cooperating Organizations AAPM—American Association of Physicists in Medicine (United States) • APS—American Physiological Society • CARS—Computer Assisted Radiology and Surgery (Germany) Medical Image Perception Society (United States) • Radiological Society of North America (United States) • The DICOM Standards Committee (United States) • Society for Imaging Informatics in Medicine (United States) • Florida Photonics Cluster (United States) World Molecular Imaging Society

Craig K. Abbey

Papers

Detectability index describes the information conveyed by sonographic images

The footsteps of attention in the Posner paradigm revealed by classification images

Higher-order scene statistics of breast images

A foveated channelized Hotelling search model predicts dissociations in human performance in 2D and 3D images

Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment