Showing papers on "Corner detection published in 2020"

An Introduction to 3D Computer Vision Techniques and Algorithms

Abstract: Preface. Acknowledgements. Notation and Abbreviations. Part I. 1 Introduction. 1.1 Stereo-pair Images and Depth Perception. 1.2 3D Vision Systems. 1.3 3D Vision Applications. 1.4 Contents Overview: The 3D Vision Task in Stages. 2 Brief History of Research on Vision. 2.1 Abstract. 2.2 Retrospective of Vision Research. 2.3 Closure. Part II. 3 2D and 3D Vision Formation. 3.1 Abstract. 3.2 Human Visual System. 3.3 Geometry and Acquisition of a Single Image. 3.4 Stereoscopic Acquisition Systems. 3.5 Stereo Matching Constraints. 3.6 Calibration of Cameras. 3.7 Practical Examples. 3.8 Appendix: Derivation of the Pin-hole Camera Transformation. 3.9 Closure. 4 Low-level Image Processing for Image Matching. 4.1 Abstract. 4.2 Basic Concepts. 4.3 Discrete Averaging. 4.4 Discrete Differentiation. 4.5 Edge Detection. 4.6 Structural Tensor. 4.7 Corner Detection. 4.8 Practical Examples. 4.9 Closure. 5 Scale-space Vision. 5.1 Abstract. 5.2 Basic Concepts. 5.3 Constructing a Scale-space. 5.4 Multi-resolution Pyramids. 5.5 Practical Examples. 5.6 Closure. 6 Image Matching Algorithms. 6.1 Abstract. 6.2 Basic Concepts. 6.3 Match Measures. 6.4 Computational Aspects of Matching. 6.5 Diversity of Stereo Matching Methods. 6.6 Area-based Matching. 6.7 Area-based Elastic Matching. 6.8 Feature-based Image Matching. 6.9 Gradient-based Matching. 6.10 Method of Dynamic Programming. 6.11 Graph Cut Approach. 6.12 Optical Flow. 6.13 Practical Examples. 6.14 Closure. 7 Space Reconstruction and Multiview Integration. 7.1 Abstract. 7.2 General 3D Reconstruction. 7.3 Multiview Integration. 7.4 Closure. 8 Case Examples. 8.1 Abstract. 8.2 3D System for Vision-Impaired Persons. 8.3 Face and Body Modelling. 8.4 Clinical and Veterinary Applications. 8.5 Movie Restoration. 8.6 Closure. Part III. 9 Basics of the Projective Geometry. 9.1 Abstract. 9.2 Homogeneous Coordinates. 9.3 Point, Line and the Rule of Duality. 9.4 Point and Line at Infinity. 9.5 Basics on Conics. 9.6 Group of Projective Transformations. 9.7 Projective Invariants. 9.8 Closure. 10 Basics of Tensor Calculus for Image Processing. 10.1 Abstract. 10.2 Basic Concepts. 10.3 Change of a Base. 10.4 Laws of Tensor Transformations. 10.5 The Metric Tensor. 10.6 Simple Tensor Algebra. 10.7 Closure. 11 Distortions and Noise in Images. 11.1 Abstract. 11.2 Types and Models of Noise. 11.3 Generating Noisy Test Images. 11.4 Generating Random Numbers with Normal Distributions. 11.5 Closure. 12 Image Warping Procedures. 12.1 Abstract. 12.2 Architecture of the Warping System. 12.3 Coordinate Transformation Module. 12.4 Interpolation of Pixel Values. 12.5 The Warp Engine. 12.6 Software Model of the Warping Schemes. 12.7 Warp Examples. 12.8 Finding the Linear Transformation from Point Correspondences. 12.9 Closure. 13 Programming Techniques for Image Processing and Computer Vision. 13.1 Abstract. 13.2 Useful Techniques and Methodology. 13.3 Design Patterns. 13.4 Object Lifetime and Memory Management. 13.5 Image Processing Platforms. 13.6 Closure. 14 Image Processing Library. References. Index.

Correlation-Guided Attention for Corner Detection Based Visual Tracking

Abstract: Accurate bounding box estimation has recently attracted much attention in the tracking community because traditional multi-scale search strategies cannot estimate tight bounding boxes in many challenging scenarios involving changes to the target. A tracker capable of detecting target corners can flexibly adapt to such changes, but existing corner detection based tracking methods have not achieved adequate success. We analyze the reasons for their failure and propose a state-of-the-art tracker that performs correlation-guided attentional corner detection in two stages. First, a region of interest (RoI) is obtained by employing an efficient Siamese network to distinguish the target from the background. Second, a pixel-wise correlation-guided spatial attention module and a channel-wise correlation-guided channel attention module exploit the relationship between the target template and the RoI to highlight corner regions and enhance features of the RoI for corner detection. The correlation-guided attention modules improve the accuracy of corner detection, thus enabling accurate bounding box estimation. When trained on large-scale datasets using a novel RoI augmentation strategy, the performance of the proposed tracker, running at a high speed of 70 FPS, is comparable with that of state-of-the-art trackers in meeting five challenging performance benchmarks.

Joint Filtering of Intensity Images and Neuromorphic Events for High-Resolution Noise-Robust Imaging

Abstract: We present a novel computational imaging system with high resolution and low noise. Our system consists of a traditional video camera which captures high-resolution intensity images, and an event camera which encodes high-speed motion as a stream of asynchronous binary events. To process the hybrid input, we propose a unifying framework that first bridges the two sensing modalities via a noise-robust motion compensation model, and then performs joint image filtering. The filtered output represents the temporal gradient of the captured space-time volume, which can be viewed as motion-compensated event frames with high resolution and low noise. Therefore, the output can be widely applied to many existing event-based algorithms that are highly dependent on spatial resolution and noise robustness. In experimental results performed on both publicly available datasets as well as our contributing RGB-DAVIS dataset, we show systematic performance improvement in applications such as high frame-rate video synthesis, feature/corner detection and tracking, as well as high dynamic range image reconstruction.

Corner Detection Using Multi-directional Structure Tensor with Multiple Scales

Abstract: Corners are important features for image analysis and computer vision tasks. Local structure tensors with multiple scales are widely used in intensity-based corner detectors. In this paper, the properties of intensity variations of a step edge, L-type corner, Y- or T-type corner, X-type corner, and star-type corner are investigated. The properties that we obtained indicate that the image intensity variations of a corner are not always large in all directions. The properties also demonstrate that existing structure tensor-based corner detection methods cannot depict the differences of intensity variations well between edges and corners which result in wrong corner detections. We present a new technique to extract the intensity variations from input images using anisotropic Gaussian directional derivative filters with multiple scales. We prove that the new extraction technique on image intensity variation has the ability to accurately depict the characteristics of edges and corners in the continuous domain. Furthermore, the properties of the intensity variations of step edges and corners enable us to derive a new multi-directional structure tensor with multiple scales, which has the ability to depict the intensity variation differences well between edges and corners in the discrete domain. The eigenvalues of the multi-directional structure tensor with multiple scales are used to develop a new corner detection method. Finally, the criteria on average repeatability (under affine image transformation, JPEG compression, and noise degradation), region repeatability based on the Oxford dataset, repeatability metric based on the DTU dataset, detection accuracy, and localization accuracy are used to evaluate the proposed detector against ten state-of-the-art methods. The experimental results show that our proposed detector outperforms all the other tested detectors.

A novel noncontact detection method of surgeon's operation for a master-slave endovascular surgery robot

Abstract: Master-slave endovascular interventional surgery (EIS) robots have brought revolutionary advantages to traditional EIS, such as avoiding X-ray radiation to the surgeon and improving surgical precision and safety. However, the master controllers of most of the current EIS robots always lead to bad human-machine interaction, because of the difference in nature between the rigid operating handle and the flexible medical catheter used in EIS. In this paper, a noncontact detection method is proposed, and a novel master controller is developed to realize real-time detection of surgeon's operation without interference to the surgeon. A medical catheter is used as the operating handle. It is enabled by using FAST corner detection algorithm and optical flow algorithm to track the corner points of the continuous markers on a designed sensing pipe. A mathematical model is established to calculate the axial and rotational motion of the sensing pipe according to the moving distance of the corner points in image coordinates. A master-slave EIS robot system is constructed by integrating the proposed master controller and a developed slave robot. Surgical task performance evaluation in an endovascular evaluator (EVE) is conducted, and the results indicate that the proposed detection method breaks through the axial measuring range limitation of the previous marker-based detection method. In addition, the rotational detection error is reduced by 92.5% compared with the previous laser-based detection method. The results also demonstrate the capability and efficiency of the proposed master controller to control the slave robot for surgical task implementation. Graphical abstract A novel master controller is developed to realize real-time noncontact detection of surgeon's operation without interference to the surgeon. The master controller is used to remotely control the slave robot to implement certain surgical tasks.

Camera and Radar Sensor Fusion for Robust Vehicle Localization via Vehicle Part Localization

Abstract: Many production vehicles are now equipped with both cameras and radar in order to provide various driver-assistance systems (DAS) with position information of surrounding objects. These sensors, however, cannot provide position information accurate enough to realize highly automated driving functions and other advanced driver-assistance systems (ADAS). Sensor fusion methods were proposed to overcome these limitations, but they tend to show limited detection performance gain in terms of accuracy and robustness. In this study, we propose a camera-radar sensor fusion framework for robust vehicle localization based on vehicle part (rear corner) detection and localization. The main idea of the proposed method is to reinforce the azimuth angle accuracy of the radar information by detecting and localizing the rear corner part of the target vehicle from an image. This part-based fusion approach enables accurate vehicle localization as well as robust performance with respect to occlusions. For efficient part detection, several candidate points are generated around the initial radar point. Then, a widely adopted deep learning approach is used to detect and localize the left and right corners of target vehicles. The corner detection network outputs their reliability score based on the localization uncertainty of the center point in corner parts. Using these position reliability scores along with a particle filter, the most probable rear corner positions are estimated. Estimated positions (pixel coordinate) are translated into angular data, and the surrounding vehicle is localized with respect to the ego-vehicle by combining the angular data of the rear corner and the radar's range data in the lateral and longitudinal direction. The experimental test results show that the proposed method provides significantly better localization performance in the lateral direction, with greatly reduced maximum errors (radar: 3.02m, proposed method: 0.66m) and root mean squared errors (radar: 0.57m, proposed method: 0.18m).

Mixed Road User Trajectory Extraction From Moving Aerial Videos Based on Convolution Neural Network Detection

Abstract: Vehicle trajectory data under mixed traffic conditions provides critical information for urban traffic flow modeling and analysis. Recently, the application of unmanned aerial vehicles (UAV) creates a potential of reducing traffic video collection cost and enhances flexibility at the spatial-temporal coverage, supporting trajectory extraction in diverse environments. However, accurate vehicle detection is a challenge due to facts such as small vehicle size and inconspicuous object features in UAV videos. In addition, camera motion in UAV videos hardens the trajectory construction procedure. This research aims at proposing a novel framework for accurate vehicle trajectory construction from UAV videos under mixed traffic conditions. Firstly, a Convolution Neural Network (CNN)-based detection algorithm, named You Only Look Once (YOLO) v3, is applied to detect vehicles globally. Then an image registration method based on Shi-Tomasi corner detection is applied for camera motion compensation. Trajectory construction methods are proposed to obtain accurate vehicle trajectories based on data correlation and trajectory compensation. At last, the ensemble empirical mode decomposition (EEMD) is applied for trajectory data denoising. Our framework is tested on three aerial videos taken by an UAV on urban roads with one including intersection. The extracted vehicle trajectories are compared with manual counts. The results show that the proposed framework achieves an average Recall of 91.91% for motor vehicles, 81.98% for non-motorized vehicles and 78.13% for pedestrians in three videos.

Finding Arbitrary-Oriented Ships From Remote Sensing Images Using Corner Detection

Abstract: Ship detection in remote sensing images is a challenging task. In this letter, a novel anchor-free framework is proposed for detecting arbitrary-oriented ships in remote sensing images. First, an end-to-end fully convolutional network is designed to detect the three key points, including the bow, stern, and center of the ship, as well as its angle. Second, the key points of the bow and stern are combined to generate possible rotated bounding boxes. Third, the predicted center and angle information of the ship are used to confirm the bounding box. In the designed network, feature fusion and feature enhancement modules are introduced to improve the performance in complex scenes. The proposed method avoids complicated anchor design compared with anchor-based methods. The experimental results show that with good robustness to haze occlusion, scale variation, and adjacent ship disturbances, our method outperforms other state-of-the-art methods.

Research on the Calibration of Binocular Camera Based on BP Neural Network Optimized by Improved Genetic Simulated Annealing Algorithm

Abstract: The Back Propagation (BP) neural network has the problems of low accuracy and poor convergence in the process of binocular camera calibration. A method based on BP neural network optimized by improved genetic simulated annealing algorithm (IGSAA-BP) is proposed to solve these problems to complete the binocular camera calibration. The method of combining Gaussian scale space and Harris corner detection operator is used for corner detection. A matched algorithm of homonymous corner is proposed by combining point-to-point spatial mapping and grid motion statistics. The pixel values of the homonymous corner and three-dimensional coordinate values are taken as the input and output of BP neural network respectively. The crossover and mutation probability of genetic simulated annealing algorithm and the annealing criterion are improved, the IGSAA-BP neural network is used to calibrate the binocular camera. The average calibration accuracy of BP neural network and IGSAA-BP neural network is 0.71mm and 0.03mm, respectively. The average calibration accuracy of binocular camera is improved by 96%. The iteration speed is increased by 20 times and global optimization ability is improved. It can be seen that the IGSAA-BP neural network can improve the calibration accuracy of binocular camera and accelerate convergence speed.

A Color Image Segmentation Method Based on Region Salient Color and Fuzzy C-Means Algorithm

Abstract: This paper proposes a color image segmentation method based on region salient color and the fuzzy C-means (FCM) algorithm. The method first uses the convex hull theory based on Harris corner detection to detect the object of the image. Thus, the object and the background can be separated. Then, the quantized color histogram can be studied in the HSV color space. By calculating the number of the peak values of both the object and the background histograms, the quantity of the regional salient colors can be obtained. The quantity is the number of the clustering centroids of FCM algorithm. Finally, the FCM algorithm and the noise correction algorithm can be used in the object and the background, respectively. The obtained segmented image consists of the object and the background segmentation. It proves that the method in this paper is an effective segmentation method based on the experiments made by use of Berkeley segmentation dataset. According to the experimental results, it can be concluded that the proposed algorithm has the highest segmentation accuracy and the shortest computing time among the algorithms mentioned in this paper. The algorithm can achieve high-quality, stable and accurate color image segmentation results.

A Novel Tracking Method Based on Improved FAST Corner Detection and Pyramid LK Optical Flow

Abstract: Object tracking is a challenging problem with accuracy and large motion in computer vision. Though optical flow has achieved great performance, it would possibly fail to track fast moving object, and accuracy and big movement both suffer from the size of integration window. Pyramid LK (Lucas-Kanade) method can solve large motion in a small integration window. However, the basic pyramid LK is affected by accuracy. So, a combination of improved FAST corner detection and pyramid LK optical flow is applied. In this method, sub-pixel computation can increase the accuracy. The large movement can be satisfied by scaling the source image. The strongest gray changing corner point can be rapidly extracted by the improved FAST algorithm. This approach can better estimate the optical flow with strong corner to track moving object. In the experiment, a video of actual movement of ball is used to test algorithm validity. Average errors in the x direction and y direction are 0.9386 pixel and 0.6792 pixel separately. The detection results demonstrate that this algorithm can track moving target of large motion accurately.

An Image Registration Method for Multisource High-Resolution Remote Sensing Images for Earthquake Disaster Assessment.

Abstract: For earthquake disaster assessment using remote sensing (RS), multisource image registration is an important step. However, severe earthquakes will increase the deformation between the remote sensing images acquired before and after the earthquakes on different platforms. Traditional image registration methods can hardly meet the requirements of accuracy and efficiency of image registration of post-earthquake RS images used for disaster assessment. Therefore, an improved image registration method was proposed for the registration of multisource high-resolution remote sensing images. The proposed method used the combination of the Shi_Tomasi corner detection algorithm and scale-invariant feature transform (SIFT) to detect tie points from image patches obtained by an image partition strategy considering geographic information constraints. Then, the random sample consensus (RANSAC) and greedy algorithms were employed to remove outliers and redundant matched tie points. Additionally, a pre-earthquake RS image database was constructed using pre-earthquake high-resolution RS images and used as the references for image registration. The performance of the proposed method was evaluated using three image pairs covering regions affected by severe earthquakes. It was shown that the proposed method provided higher accuracy, less running time, and more tie points with a more even distribution than the classic SIFT method and the SIFT method using the same image partitioning strategy.

A fully automatic segmentation algorithm for CT lung images based on random forest.

Abstract: Purpose Several negative factors, such as juxta-pleural nodules, pulmonary vessels, and image noise, make accurately segmenting lungs from computed tomography (CT) images a complex task. We propose a novel hybrid automated algorithm in the paper based on random forest to deal with the issues. Our method aims to eliminate the effect of the factors and generate accurate segmentation of lungs from CT images. Methods Our algorithm consists of five main steps: image preprocessing, lung region extraction, trachea elimination, lung separation, and contour correction. A lung CT image is first preprocessed with a novel normal vector correlation-based image denoising approach and decomposed into a group of multiscale subimages. A modified superpixel segmentation method is then performed on the first-level subimage to generate a set of superpixels, and a random forest classifier is employed to segment the lungs by classifying the superpixels of each subimage-based on the features extracted from them. The initial lung segmentation result is further refined through trachea elimination using an iterative thresholding approach, lung separation based on context information of image sequence, and contour correction with a corner detection technique. Results Our algorithm is tested on a set of CT images affected with interstitial lung diseases, and experiments show that the algorithm achieves high accuracy on lung segmentation with 0.9638 Jaccard's index and 0.9867 Dice similarity coefficient, compared with ground truths. Additionally, our algorithm achieves an average 7.7% better Dice similarity coefficient than compared conventional lung segmentation methods and 1% better than Deep Learning. Conclusions Our algorithm can segment lungs from lung CT images with good performance in a fully automatic fashion, and it is of great assistance for lung disease detection in the computer-aided detection system.

Combining Sparse and Dense Features to Improve Multi-Modal Registration for Brain DTI Images

Abstract: A new solution to overcome the constraints of multimodality medical intra-subject image registration is proposed, using the mutual information (MI) of image histogram-oriented gradients as a new matching criterion. We present a rigid, multi-modal image registration algorithm based on linear transformation and oriented gradients for the alignment of T2-weighted (T2w) images (as a fixed reference) and diffusion tensor imaging (DTI) (b-values of 500 and 1250 s/mm2) as floating images of three patients to compensate for the motion during the acquisition process. Diffusion MRI is very sensitive to motion, especially when the intensity and duration of the gradient pulses (characterized by the b-value) increases. The proposed method relies on the whole brain surface and addresses the variability of anatomical features into an image stack. The sparse features refer to corners detected using the Harris corner detector operator, while dense features use all image pixels through the image histogram of oriented gradients (HOG) as a measure of the degree of statistical dependence between a pair of registered images. HOG as a dense feature is focused on the structure and extracts the oriented gradient image in the x and y directions. MI is used as an objective function for the optimization process. The entropy functions and joint entropy function are determined using the HOGs data. To determine the best image transformation, the fiducial registration error (FRE) measure is used. We compare the results against the MI-based intensities results computed using a statistical intensity relationship between corresponding pixels in source and target images. Our approach, which is devoted to the whole brain, shows improved registration accuracy, robustness, and computational cost compared with the registration algorithms, which use anatomical features or regions of interest areas with specific neuroanatomy. Despite the supplementary HOG computation task, the computation time is comparable for MI-based intensities and MI-based HOG methods.

A study on hovering control of small aerial robot by sensing existing floor features

Abstract: Since precise self-position estimation is required for autonomous flight of aerial robots, there has been some studies on self-position estimation of indoor aerial robots. In this study, we tackle the self-position estimation problem by mounting a small downward-facing camera on the chassis of an aerial robot. We obtain robot position by sensing the features on the indoor floor. In this work, we used the vertex points ( tile corners ) where four tiles on a typical tiled floor connected, as an existing feature of the floor. Furthermore, a small lightweight microcontroller is mounted on the robot to perform image processing for the on-board camera. A lightweight image processing algorithm is developed. So, the real-time image processing could be performed by the microcontroller alone which leads to conduct on-board real time tile corner detection. Furthermore, same microcontroller performs control value calculation for flight commanding. The flight commands are implemented based on the detected tile corner information. The above mentioned all devices are mounted on an actual machine, and the effectiveness of the system was investigated.

Image Denoising and Refinement Based on an Iteratively Reweighted Least Squares Filter

Abstract: This paper presents a method to reduce noise and refine detail features of a scene based on an iteratively reweighted least squares method. The performance of the proposed filter, called the iteratively reweighted least squares filter (IRLSF), was compared with the state-of-the-art filters by checking their ability to recover simulated edge models under various degrees of noise contamination. The results of the simulation comparison show that IRLSF is superior to the other filters in terms of its ability to recover the original edge models. To apply IRLSF to real images of a scene captured by a camera, a procedure composed of corner detection, least squares matching, bilinear resampling, and iteratively reweighted least squares is proposed. The experimental results show that IRLSF produces mean images that are effectively denoised, and that its accuracy is less than one half of grey-level-quantization-unit of test images captured by a commercial camera.

In Situ Measurement of 3D Crystal Size Distribution by Double-View Image Analysis with Case Study on l -Glutamic Acid Crystallization

Abstract: An in situ measurement method is proposed for monitoring three-dimensional (3D) crystal size distribution (CSD) during a crystallization process, based on a binocular microvision system. The stereo particle shape is reconstructed from double-view images captured by two microscopic cameras fixed at different angles outside the crystallizer. To overcome the influence from solution turbulence and uneven illumination background involved with in situ imaging, a microscopic double-view image analysis method is established to identify the key corners of each particle shape in the captured images, including corner detection and corner matching. Two fast algorithms are therefore given for online detection of two typical crystal morphologies of prismatic and needle-like shapes, such as α- and β-forms of l-glutamic acid (LGA) crystals, respectively. On the basis of the identified key corners for different particle shapes, a 3D geometry model is established to approximately reconstruct the 3D shape for each imaged particle, such that 3D sizes of each particle could be quantitatively estimated, along with the particle volume. Experiments on the LGA cooling crystallization are performed to demonstrate the effectiveness of the proposed method.

A robust automated surface-matching registration method for neuronavigation.

Abstract: PURPOSE The surface-matching registration method in the current neuronavigation completes the coarse registration mainly by manually selecting anatomical landmarks, which increases the registration time, makes the automatic registration impossible and sometimes results in mismatch. It may be more practical to use a fast, accurate, and automatic spatial registration method for the patient-to-image registration. METHODS A coarse-to-fine spatial registration method to automatically register the patient space to the image space without placing any markers on the head of the patient was proposed. Three-dimensional (3D) keypoints were extracted by 3D Harris corner detector from the point clouds in the patient and image spaces, and used as input to the 4-points congruent sets (4PCS) algorithm which automatically registered the keypoints in the patient space with the keypoints in the image space without any assumptions about initial alignment. Coarsely aligned point clouds in the patient and image space were then fine-registered with a variant of the iterative closest point (ICP) algorithm. Two experiments were designed based on one phantom and five patients to validate the efficiency and effectiveness of the proposed method. RESULTS Keypoints were extracted within 7.0 s with a minimum threshold 0.001. In the phantom experiment, the mean target registration error (TRE) of 15 targets on the surface of the elastic phantom in the five experiments was 1.17 ± 0.04 mm, and the average registration time was 17.4 s. In the clinical experiments, the mean TRE of the targets on the first, second, third, fourth, and fifth patient's head surface were 1.70 ± 0.32 mm, 1.83 ± 0.38 mm, 1.64 ± 0.3 mm, 1.67 ± 0.35 mm, and 1.72 ± 0.31 mm, respectively, and the average registration time was 21.4 s. Compared with the method only based on the 4PCS and ICP algorithm and the current clinical method, the proposed method has obvious speed advantage while ensuring the registration accuracy. CONCLUSIONS The proposed method greatly improves the registration speed while guaranteeing the equivalent or higher registration accuracy, and avoids a tedious manual process for the coarse registration.

A Multi-Object Grasping Detection Based on the Improvement of YOLOv3 Algorithm

Abstract: YOLOv3 has achieved good results in the field of object detection. In order to achieve multi-object grasping detection, the network structure has been improved. The improved YOLOv3 algorithm is applied to the object position and pose detection in robotic grasping, and a deep learning model is proposed to predict the robot's grasping position, which can detect the occurrence of multiple objects in real time and grasp them in order according to the semantic information. For the specific application scenario, the corresponding dataset is made, and a corner detection method based on YOLOv3 is proposed to grasping position and pose detection. Compared with the traditional corner detection method, this method has semantic information in its detected corner. In the scene, we first classify and locate the object, then detect the corner of the object, and filter the corner of the false detection through the positioning of the object, and design the corresponding algorithm to complete the corner of the missed detection, so that the accuracy of the corner detection is greatly improved, reaching 99% in the self-made dataset. Finally, the position information of the corner is used to calculate the centroid position of the object, that is, the grasping point of the object. The point cloud information is obtained by depth camera, and the grasping pose of the object is calculated. This method can greatly improve the accuracy of grasping detection in specific scenes.

Robust and Efficient Corner Detector Using Non-Corners Exclusion

Abstract: Corner detection is a traditional type of feature point detection method. Among methods used, with its good accuracy and the properties of invariance for rotation, noise and illumination, the Harris corner detector is widely used in the fields of vision tasks and image processing. Although it possesses a good performance in detection quality, its application is limited due to its low detection efficiency. The efficiency is crucial in many applications because it determines whether the detector is suitable for real-time tasks. In this paper, a robust and efficient corner detector (RECD) improved from Harris corner detector is proposed. First, we borrowed the principle of the feature from accelerated segment test (FAST) algorithm for corner pre-detection, in order to rule out non-corners and retain many strong corners as real corners. Those uncertain corners are looked at as candidate corners. Second, the gradients are calculated in the same way as the original Harris detector for those candidate corners. Third, to reduce additional computation amount, only the corner response function (CRF) of the candidate corners is calculated. Finally, we replace the highly complex non-maximum suppression (NMS) by an improved NMS to obtain the resulting corners. Experiments demonstrate that RECD is more competitive than some popular corner detectors in detection quality and speed. The accuracy and robustness of our method is slightly better than the original Harris detector, and the detection time is only approximately 8.2% of its original value.

Fast Corner Detection Using Approximate Form of Second-Order Gaussian Directional Derivative

Abstract: High-efficiency image corner detection, one of the most important and critical basic technology in industrial image processing, is to detect point features from an input image in real-time. In this article, we propose a new corner detection method which has both good performance of corner detection and real-time processing abilities. Firstly, the integral image and the box filter are combined to obtain the second-order derivative response in each direction of the image. Secondly, a new coarse screening mechanism for candidate corners is presented to reduce the complexity of the corner metric. Thirdly, a non-maximum suppression operation is utilized to obtain corners. Finally, the performance evaluation on accuracy of corner detection, localization error, average repeatability, region repeatability, different lighting conditions, and execution time are used to assess the proposed method against twelve state-of-the-art methods. The experimental results show that our proposed detector has good corner detection performance and achieves the requirement of real-time processing.

A Robust Tracking-by-Detection Algorithm Using Adaptive Accumulated Frame Differencing and Corner Features.

Abstract: We propose a tracking-by-detection algorithm to track the movements of meeting participants from an overhead camera. An advantage of using overhead cameras is that all objects can typically be seen clearly, with little occlusion; however, detecting people from a wide-angle overhead view also poses challenges such as people’s appearance significantly changing due to their position in the wide-angle image, and generally from a lack of strong image features. Our experimental datasets do not include empty meeting rooms, and this means that standard motion based detection techniques (e.g., background subtraction or consecutive frame differencing) struggle since there is no prior knowledge for a background model. Additionally, standard techniques may perform poorly when there is a wide range of movement behaviours (e.g. periods of no movement and periods of fast movement), as is often the case in meetings. Our algorithm uses a novel coarse-to-fine detection and tracking approach, combining motion detection using adaptive accumulated frame differencing (AAFD) with Shi-Tomasi corner detection. We present quantitative and qualitative evaluation which demonstrates the robustness of our method to track people in environments where object features are not clear and have similar colour to the background. We show that our approach achieves excellent performance in terms of the multiple object tracking accuracy (MOTA) metrics, and that it is particularly robust to initialisation differences when compared with baseline and state of the art trackers. Using the Online Tracking Benchmark (OTB) videos we also demonstrate that our tracker is very strong in the presence of background clutter, deformation and illumination variation.

Stepwise calibration of plenoptic cameras based on corner features of raw images

Abstract: Plenoptic cameras are increasingly gaining attention in various fields due to their ability to capture both spatial and angular information of light rays. Accurate geometric calibration can lay a solid foundation for the applications that use the plenoptic camera. In this paper, to the best of our knowledge, we first introduce an accurate corner detection method based on a novel selection and refinement strategy. The detected-corner candidates on raw images are selected by a random sample consensus (RANSAC)-based algorithm and optimized by the photometric similarity, as well as the sub-pixel refinement. In addition, a robust and accurate stepwise calibration method is proposed based on separated intrinsic parameters, including parameters related to the pinhole model and those unique to the plenoptic camera. Experiments on both simulated and real data demonstrate that our method outperforms the state-of-the-art methods and is able to support a more accurate calibration of plenoptic cameras.

Dynamic Resource-aware Corner Detection for Bio-inspired Vision Sensors

Abstract: Event-based cameras are vision devices that transmit only brightness changes with low latency and ultra-low power consumption. Such characteristics make event-based cameras attractive in the field of localization and object tracking in resource-constrained systems. Since the number of generated events in such cameras is huge, the selection and filtering of the incoming events are beneficial from both increasing the accuracy of the features and reducing the computational load. In this paper, we present an algorithm to detect asynchronous corners from a stream of events in real-time on embedded systems. The algorithm is called the Three Layer Filtering-Harris or TLF-Harris algorithm. The algorithm is based on an events' filtering strategy whose purpose is 1) to increase the accuracy by deliberately eliminating some incoming events, i.e., noise, and 2) to improve the real-time performance of the system, i.e., preserving a constant throughput in terms of input events per second, by discarding unnecessary events with a limited accuracy loss. An approximation of the Harris algorithm, in turn, is used to exploit its high-quality detection capability with a low-complexity implementation to enable seamless real-time performance on embedded computing platforms. The proposed algorithm is capable of selecting the best corner candidate among neighbors and achieves an average execution time savings of 59 % compared with the conventional Harris score. Moreover, our approach outperforms the competing methods, such as eFAST, eHarris, and FA-Harris, in terms of real-time performance, and surpasses Arc* in terms of accuracy.

A Closed-Loop Controller for a Continuum Surgical Manipulator Based on a Specially Designed Wrist Marker and Stereo Tracking

Abstract: Continuum structures have been widely deployed in MIS (Minimally Invasive Surgery) due to its intrinsic dexterity and compliance. However, the absolute tip positioning accuracy of a continuum manipulator may be low compared to a rigid-linked serial manipulator due to its actual bending behaviors that are often different from the assumed ideal ones. Even though the movement accuracy can be improved by motion calibration and actuation compensation, the errors caused by external loads will still exist. In order to improve the tip positioning accuracy of a continuum surgical manipulator, this paper proposes a proof-of-concept approach via integrating a specially designed wrist marker and a closed-loop controller with the visual feedback from a stereo endoscopic camera. The corner detection of the wrist marker was firstly achieved via a modified corner detection algorithm. Then, the tip position was obtained via a pose estimation algorithm that seeks position and orientation of the wrist marker when the corner points in the two images from the stereo endoscopic camera were aligned with the actual corners of the wrist marker. Experimental verification showed that the tip positioning errors were reduced to 25.23% of the original errors during trajectory tracking, demonstrating the effectiveness of the proposed approach.