In this paper, we present an energy-efficient architecture of the Canny edge detector for advanced mobile vision applications. Three key techniques for reducing computational complexity of the Canny edge detector are presented. First, by exploiting the rank characteristic of the convolution kernel of Gaussian smoothing and Sobel gradient filters, common computations are identified and shared in the image filter design to reduce the number of additions and multiplications. For the gradient magnitude/direction computation, only three directions of neighboring pixels are considered to reduce computation energy with minor degradation on conformance performance (CP). For the adaptive threshold selections, an interesting observation is that the mean values of gradient magnitudes show small variations depending on the classified block types. Thus, the threshold selection process can be simplified as multiplying the mean value of the local block with predecided constants. The proposed low complexity Canny edge detector has been implemented using both field-programmable gate arrays (FPGAs) and a 65-nm standard-cell library. The FPGA implementation with Xilinx Virtex-V (XC5VSX240T) shows that our edge detector achieves 48% of area and 73% of execution time savings over the conventional architecture without seriously sacrificing the detection performance. The proposed edge detector implemented with 65-nm standard-cell library can easily support real-time ultrahigh definition video data processing (50 frames/s) with the power consumption of 5.48 mW (108.84 $\mu \text{J}$ /frame).

Energy Efficient Canny Edge Detector for Advanced Mobile Vision Applications

Artificial intelligence (AI) and machine learning (ML) tools play a significant role in the recent evolution of smart systems. AI solutions are pushing towards a significant shift in many fields such as healthcare, autonomous airplanes and vehicles, security, marketing customer profiling and other diverse areas. One of the main challenges hindering the AI potential is the demand for high-performance computation resources. Recently, hardware accelerators are developed in order to provide the needed computational power for the AI and ML tools. In the literature, hardware accelerators are built using FPGAs, GPUs and ASICs to accelerate computationally intensive tasks. These accelerators provide high-performance hardware while preserving the required accuracy. In this work, we present a systematic literature review that focuses on exploring the available hardware accelerators for the AI and ML tools. More than 169 different research papers published between the years 2009 and 2019 are studied and analysed.

A systematic literature review on hardware implementation of artificial intelligence algorithms

Motion tracking algorithms are widely used in computer vision related research. However, the new video standards, especially those in high resolutions, cause that current implementations, even running on modern hardware, no longer meet the needs of real-time processing. To overcome this challenge several GPU (Graphics Processing Unit) computing approaches have recently been proposed. Although they present a great potential of a GPU platform, hardly any is able to process high definition video sequences efficiently. Thus, a need arose to develop a tool being able to address the outlined problem. In this paper we present software that implements optical flow motion tracking using the Lucas-Kanade algorithm. It is also integrated with the Harris corner detector and therefore the algorithm may perform sparse tracking, i.e. tracking of the meaningful pixels only. This allows to substantially lower the computational burden of the method. Moreover, both parts of the algorithm, i.e. corner selection and tracking, are implemented on GPU and, as a result, the software is immensely fast, allowing for real-time motion tracking on videos in Full HD or even 4K format. In order to deliver the highest performance, it also supports multiple GPU systems, where it scales up very well.

/pdf/real-time-motion-tracking-using-optical-flow-on-multiple-2oxo9039si.pdf

Real-time motion tracking using optical flow on multiple GPUs

This paper proposes a FPGA implementation based on sliding processing window for Harris corner algorithm. It represents one of the most frequently used pre-processing method, for a wide variety of image processing algorithms, such as feature detection, motion tracking, image registration, etc‥ It relies on a series of sequential steps, each processing an image outputted by the previous step. The purpose of the sliding window is to avoid storing intermediate results of processing stages in the external FPGA memory or to avoid utilize large line buffers typically implemented with BRAM blocks. Therefore, the entire processing pipeline benefits from data locality. Implementation results for Virtex5 and Spartan-6 devices show that the proposed solution has very good performance (more than 130 fps for 1280×720 images in Xilinx Spartan-6) with significant less BRAM usage with respect to other approaches.

An FPGA sliding window-based architecture harris corner detector

Detection and description of image features play a vital role in various application domains such as image processing, computer vision, pattern recognition, and machine learning. There are two type of features that can be extracted from an image content; namely global and local features. Global features describe the image as a whole and can be interpreted as a particular property of the image involving all pixels; while, the local features aim to detect keypoints within the image and describe regions around these keypoints. After extracting the features and their descriptors from images, matching of common structures between images (i.e., features matching) is the next step for these applications. This chapter presents a general and brief introduction to topics of feature extraction for a variety of application domains. Its main aim is to provide short descriptions of the chapters included in this book volume.

Detection and Description of Image Features: An Introduction

This work presents a new flexible parameterizable architecture for image and video processing with reduced latency and memory requirements, supporting a variable input resolution. The proposed architecture is optimized for feature detection, more specifically, the Canny edge detector and the Harris corner detector. The architecture contains neighborhood extractors and threshold operators that can be parameterized at runtime. Also, algorithm simplifications are employed to reduce mathematical complexity, memory requirements, and latency without losing reliability. Furthermore, we present the proposed architecture implementation on an FPGA-based platform and its analogous optimized implementation on a GPU-based architecture for comparison. A performance analysis of the FPGA and the GPU implementations, and an extra CPU reference implementation, shows the competitive throughput of the proposed architecture even at a much lower clock frequency than those of the GPU and the CPU. Also, the results show a clear advantage of the proposed architecture in terms of power consumption and maintain a reliable performance with noisy images, low latency and memory requirements.

Paulo Ricardo Possa

Papers

A Multi-Resolution FPGA-Based Architecture for Real-Time Edge and Corner Detection