As a fundamental and critical task in various visual applications, image matching can identify then correspond the same or similar structure/content from two or more images. Over the past decades, growing amount and diversity of methods have been proposed for image matching, particularly with the development of deep learning techniques over the recent years. However, it may leave several open questions about which method would be a suitable choice for specific applications with respect to different scenarios and task requirements and how to design better image matching methods with superior performance in accuracy, robustness and efficiency. This encourages us to conduct a comprehensive and systematic review and analysis for those classical and latest techniques. Following the feature-based image matching pipeline, we first introduce feature detection, description, and matching techniques from handcrafted methods to trainable ones and provide an analysis of the development of these methods in theory and practice. Secondly, we briefly introduce several typical image matching-based applications for a comprehensive understanding of the significance of image matching. In addition, we also provide a comprehensive and objective comparison of these classical and latest techniques through extensive experiments on representative datasets. Finally, we conclude with the current status of image matching technologies and deliver insightful discussions and prospects for future works. This survey can serve as a reference for (but not limited to) researchers and engineers in image matching and related fields.

/pdf/image-matching-from-handcrafted-to-deep-features-a-survey-27kn6w026w.pdf

Image Matching from Handcrafted to Deep Features: A Survey

U-net is an image segmentation technique developed primarily for image segmentation tasks. These traits provide U-net with a high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in nearly all major image modalities, from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. Given that U-net’s potential is still increasing, this narrative literature review examines the numerous developments and breakthroughs in the U-net architecture and provides observations on recent trends. We also discuss the many innovations that have advanced in deep learning and discuss how these tools facilitate U-net. In addition, we review the different image modalities and application areas that have been enhanced by U-net.

U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications

PadChest: A large chest x-ray image dataset with multi-label annotated reports.

We propose the first fast and certifiable algorithm for the registration of two sets of three-dimensional (3-D) points in the presence of large amounts of outlier correspondences. A certifiable algorithm is one that attempts to solve an intractable optimization problem (e.g., robust estimation with outliers) and provides readily checkable conditions to verify if the returned solution is optimal (e.g., if the algorithm produced the most accurate estimate in the face of outliers) or bound its suboptimality or accuracy. Toward this goal, we first reformulate the registration problem using a truncated least squares (TLS) cost that makes the estimation insensitive to a large fraction of spurious correspondences. Then, we provide a general graph-theoretic framework to decouple scale, rotation, and translation estimation, which allows solving in cascade for the three transformations. Despite the fact that each subproblem (scale, rotation, and translation estimation) is still nonconvex and combinatorial in nature, we show that 1) TLS scale and (component-wise) translation estimation can be solved in polynomial time via an adaptive voting scheme, 2) TLS rotation estimation can be relaxed to a semidefinite program (SDP) and the relaxation is tight, even in the presence of extreme outlier rates, and 3) the graph-theoretic framework allows drastic pruning of outliers by finding the maximum clique. We name the resulting algorithm TEASER ( Truncated least squares Estimation And SEmidefinite Relaxation ). While solving large SDP relaxations is typically slow, we develop a second fast and certifiable algorithm, named TEASER++, that uses graduated nonconvexity to solve the rotation subproblem and leverages Douglas-Rachford Splitting to efficiently certify global optimality. For both algorithms, we provide theoretical bounds on the estimation errors, which are the first of their kind for robust registration problems. Moreover, we test their performance on standard benchmarks, object detection datasets, and the 3DMatch scan matching dataset, and show that 1) both algorithms dominate the state-of-the-art (e.g., RANSAC, branch-&-bound, heuristics) and are robust to more than $\text{99}\%$ outliers when the scale is known, 2) TEASER++ can run in milliseconds and it is currently the fastest robust registration algorithm, and 3) TEASER++ is so robust it can also solve problems without correspondences (e.g., hypothesizing all-to-all correspondences), where it largely outperforms ICP and it is more accurate than Go-ICP while being orders of magnitude faster. We release a fast open-source C++ implementation of TEASER++.

/pdf/teaser-fast-and-certifiable-point-cloud-registration-40gs80w55f.pdf

TEASER: Fast and Certifiable Point Cloud Registration

The establishment of image correspondence through robust image registration is critical to many clinical tasks such as image fusion, organ atlas creation, and tumor growth monitoring and is a very challenging problem. Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning-based approaches and achieved the state-of-the-art in many applications, including image registration. The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey. This requires placing a focus on the different research areas as well as highlighting challenges that practitioners face. This survey, therefore, outlines the evolution of deep learning-based medical image registration in the context of both research challenges and relevant innovations in the past few years. Further, this survey highlights future research directions to show how this field may be possibly moved forward to the next level.

Deep learning in medical image registration: a survey

As the most common examination tool in medical practice, chest radiography has important clinical value in the diagnosis of disease. Thus, the automatic detection of chest disease based on chest radiography has become one of the hot topics in medical imaging research. Based on the clinical applications, the study conducts a comprehensive survey on computer-aided detection (CAD) systems, and especially focuses on the artificial intelligence technology applied in chest radiography. The paper presents several common chest X-ray datasets and briefly introduces general image preprocessing procedures, such as contrast enhancement and segmentation, and bone suppression techniques that are applied to chest radiography. Then, the CAD system in the detection of specific disease (pulmonary nodules, tuberculosis, and interstitial lung diseases) and multiple diseases is described, focusing on the basic principles of the algorithm, the data used in the study, the evaluation measures, and the results. Finally, the paper summarizes the CAD system in chest radiography based on artificial intelligence and discusses the existing problems and trends.

/pdf/computer-aided-detection-in-chest-radiography-based-on-58vuj8x4l6.pdf

Computer-aided detection in chest radiography based on artificial intelligence: a survey

Background/Aims: Augmented reality (AR) technology solves the problem of view switching in traditional image-guided neurosurgery systems by integrating computer-generated objects into the actual scene. However, the state-of-the-art AR solution using head-mounted displays has not been widely accepted in clinical applications because it causes some inconvenience for the surgeon during surgery. Methods: In this paper, we present a Tablet-AR system that transmits navigation information to a movable tablet PC via a wireless local area network and overlays this information on the tablet screen, which simultaneously displays the actual scene captured by its back-facing camera. With this system, the surgeon can directly observe the intracranial anatomical structure of the patient with the overlaid virtual projection images to guide the surgery. Results: The alignment errors in the skull specimen study and clinical experiment were 4.6 pixels (approx. 1.6 mm) and 6 pixels (approx. 2.1 mm), respectively. The system was also used for navigation in 2 actual clinical cases of neurosurgery, which demonstrated its feasibility in a clinical application. Conclusions: The easy-to-use Tablet-AR system presented in this study is accurate and feasible in clinical applications and has the potential to become a routine device in AR neuronavigation.

Easy-to-use augmented reality neuronavigation using a wireless tablet PC.

There are many different types of errors in neuronavigation, and the reasons and results of these errors are complex. For a neurosurgeon using the neuronavigation system, it is important to have a clear understanding of when an error may occur, what the magnitude of it is, and how to avoid it or reduce its influence on the final application accuracy. In this article, we classify all the errors into 2 groups according to the working principle of neuronavigation systems. The first group contains the errors caused by the differences between the anatomic structures in the images and that of the real patient, and the second group contains the errors occurring in transforming the position of surgical tools from the patient space to the image space. Each group is further divided into 2 subgroups. We discuss 16 types of errors and classify each of them into one of the subgroups. The classification and analysis of these errors should help neurosurgeons understand the power and limits of neuronavigation systems and use them more properly.

Classification and analysis of the errors in neuronavigation.

Three-dimensional rigid point cloud registration has many applications in computer vision and robotics. Local methods tend to fail, causing global methods to be needed, when the relative transformation is large or the overlap ratio is small. Most existing global methods utilize BnB optimization over the 6D parameter space of SE(3) . Such methods are usually very slow because the time complexity of BnB optimization is exponential in the dimensionality of the parameter space. In this paper, we decouple the optimization of translation and rotation, and we propose a fast BnB algorithm to globally optimize the 3D translation parameter first. The optimal rotation is then calculated by utilizing the global optimal translation found by the BnB algorithm. The separate optimization of translation and rotation is realized by using a newly proposed rotation invariant feature. Experiments on challenging data sets demonstrate that the proposed method outperforms state-of-the-art global methods in terms of both speed and accuracy.

/pdf/efficient-global-point-cloud-registration-by-matching-1nudd3h2t9.pdf

Efficient Global Point Cloud Registration by Matching Rotation Invariant Features Through Translation Search

Accurate segmentation of organs at risk (OARs) plays a critical role in the treatment planning of image-guided radiotherapy of head and neck cancer. This segmentation task is challenging for both humans and automated algorithms because of the relatively large number of OARs to be segmented, the large variability in size and morphology across different OARs, and the low contrast between some OARs and the background. In this study, we propose a two-stage segmentation framework based on 3D U-Net. In this framework, the segmentation of each OAR is decomposed into two subtasks: locating a bounding box of the OAR and segmenting the OAR from a small volume within the bounding box, and each subtask is fulfilled by a dedicated 3D U-Net. The decomposition makes each subtask much easier so that it can be better completed. We evaluated the proposed method and compared it to state-of-the-art methods using the Medical Image Computing and Computer-Assisted Intervention 2015 Challenge dataset. In terms of the boundary-based metric 95% Hausdorff distance, the proposed method ranked first for seven of nine OARs and ranked second for the other OARs. In terms of the area-based metric dice similarity coefficient, the proposed method ranked first for five of nine OARs and ranked second for the other three OARs with a small difference from the method that ranked first.

Zhijian Song

Papers

Computer-aided detection in chest radiography based on artificial intelligence: a survey

Easy-to-use augmented reality neuronavigation using a wireless tablet PC.

Classification and analysis of the errors in neuronavigation.

Efficient Global Point Cloud Registration by Matching Rotation Invariant Features Through Translation Search

Organ at Risk Segmentation in Head and Neck CT Images Using a Two-Stage Segmentation Framework Based on 3D U-Net