The current concept of Smart Cities influences urban planners and researchers to provide modern, secured and sustainable infrastructure and give a decent quality of life to its residents. To fulfill this need video surveillance cameras have been deployed to enhance the safety and well-being of the citizens. Despite technical developments in modern science, abnormal event detection in surveillance video systems is challenging and requires exhaustive human efforts. In this paper, we surveyed various methodologies developed to detect anomalies in intelligent video surveillance. Firstly, we revisit the surveys on anomaly detection in the last decade. We then present a systematic categorization of methodologies developed for ease of understanding. Considering the notion of anomaly depends on context, we identify different objects-of-interest and publicly available datasets in anomaly detection. Since anomaly detection is considered a time-critical application of computer vision, our emphasis is on anomaly detection using edge devices and approaches explicitly designed for them. Further, we discuss the challenges and opportunities involved in anomaly detection at the edge.

Anomaly Detection using Edge Computing in Video Surveillance System: Review

Handwritten Digit Recognition (HDR) is one of the most challenging tasks in the domain of Optical Character Recognition (OCR). Irrespective of language, there are some inherent challenges of HDR, which mostly arise due to the variations in writing styles across individuals, writing medium and environment, inability to maintain the same strokes while writing any digit repeatedly, etc. In addition to that, the structural complexities of the digits of a particular language may lead to ambiguous scenarios of HDR. Over the years, researchers have developed numerous offline and online HDR pipelines, where different image processing techniques are combined with traditional Machine Learning (ML)-based and/or Deep Learning (DL)-based architectures. Although evidence of extensive review studies on HDR exists in the literature for languages, such as English, Arabic, Indian, Farsi, Chinese, etc., few surveys on Bengali HDR (BHDR) can be found, which lack a comprehensive analysis of the challenges, the underlying recognition process, and possible future directions. In this paper, the characteristics and inherent ambiguities of Bengali handwritten digits along with a comprehensive insight of two decades of state-of-the-art datasets and approaches towards offline BHDR have been analyzed. Furthermore, several real-life application-specific studies, which involve BHDR, have also been discussed in detail. This paper will also serve as a compendium for researchers interested in the science behind offline BHDR, instigating the exploration of newer avenues of relevant research that may further lead to better offline recognition of Bengali handwritten digits in different application areas.

https://ieeexplore.ieee.org/ielx7/6287639/6514899/09869842.pdf

Two Decades of Bengali Handwritten Digit Recognition: A Survey

Handwritten Digit Recognition (HDR) is one of the most challenging tasks in the domain of Optical Character Recognition (OCR). Irrespective of language, there are some inherent challenges of HDR, which mostly arise due to the variations in writing styles across individuals, writing medium and environment, inability to maintain the same strokes while writing any digit repeatedly, etc. In addition to that, the structural complexities of the digits of a particular language may lead to ambiguous scenarios of HDR. Over the years, researchers have developed numerous offline and online HDR pipelines, where different image processing techniques are combined with traditional Machine Learning (ML)-based and/or Deep Learning (DL)-based architectures. Although evidence of extensive review studies on HDR exists in the literature for languages, such as English, Arabic, Indian, Farsi, Chinese, etc., few surveys on Bengali HDR (BHDR) can be found, which lack a comprehensive analysis of the challenges, the underlying recognition process, and possible future directions. In this paper, the characteristics and inherent ambiguities of Bengali handwritten digits along with a comprehensive insight of two decades of the state-of-the-art datasets and approaches towards offline BHDR have been analyzed. Furthermore, several real-life application-specific studies, which involve BHDR, have also been discussed in detail. This paper will also serve as a compendium for researchers interested in the science behind offline BHDR, instigating the exploration of newer avenues of relevant research that may further lead to better offline recognition of Bengali handwritten digits in different application areas. 

/pdf/two-decades-of-bengali-handwritten-digit-recognition-a-7jcj0924.pdf

Bangla Handwritten digit and character recognition, a complex computer vision problem that is important for the Bengali language as the progress in this segment for the Bengali language is slow. We used two popular datasets, BanglaLekha-Isolated and NumbtaDB, for both digits and characters and used a Convolutional neural network to train our model. We augmented our dataset using a shifting method and ran multiple experiments on vowels, digits, and characters. The result is 96.42% average accuracy on BanglaLekha augmented. Our model also achieved 98.92% accuracy on the NumtaDB dataset. We used our model to sketch up two models, License plate recognition and Smart E-learning application. We used connected component analysis in License plate recognition that helped us to extract essential segments of the license plate. We used Keras as a TensorFlow backend in our research. Bangla OCR research is ongoing and will get better over time with better datasets and learning techniques.

Bangla Handwritten Character and Digit Recognition Using Deep Convolutional Neural Network on Augmented Dataset and Its Applications

BDNet: Bengali Handwritten Numeral Digit Recognition based on Densely connected Convolutional Neural Networks

Despite being one of the major languages in the world, research regarding Bengali handwritten numeral recognition (BHNR) isn't enough in comparison with the other prominent languages. Existing methods mostly rely on feature extraction and some older machine learning algorithms. Recent bloom in machine learning due to deep neural network especially using Convolutional Neural Network (CNN) showing promising results in this field with better accuracy. Some recent works show very good accuracy only in recognizing plain simple digits but perform poor in challenging scenario because of lack of large and versatile training dataset. In this work, we've ensembled our best performing proposed CNN models to recognize numerals with high degree of accuracy beyond 96% even in most challenging noisy conditions. Initially 72000+ specimens from NumtaDB (85000+) have been used for training and 17000+ specimens have been used as test dataset. The improvement in performance in challenging scenarios has been observed, when various noisy training specimens have been augmented to create a training dataset of size about 114000 specimens. The performance of our proposed model has been compared with other existing works also and presented here. These finding are based on Computer Vision Challenge on Bengali HandWritten Digit Recognition (2018) competition submissions.

Handwritten Bangla Numeral Recognition Using Ensembling of Convolutional Neural Network

Anomaly detection (AD) in a surveillance scenario is an emerging and challenging field of research. For autonomous vehicles like drones or cars, it is immensely important to distinguish between normal and abnormal states in real-time. Additionally, we also need to detect any device malfunction. But the nature and degree of abnormality may vary depending upon the actual environment and adversary. As a result, it is impractical to model all cases a-priori and use supervised methods to classify. Also, an autonomous vehicle provides various data types like images and other analog or digital sensor data, all of which can be useful in anomaly detection if leveraged fruitfully. To that effect, in this paper, a heterogeneous system is proposed which estimates the degree of abnormality of an unmanned surveillance drone, analyzing real-time image and IMU (Inertial Measurement Unit) sensor data in an unsupervised manner. Here, we have demonstrated a Convolutional Neural Network (CNN) architecture, named AngleNet to estimate the angle between a normal image and another image under consideration, which provides us with a measure of anomaly of the device. Moreover, the IMU data are used in autoencoder to predict abnormality. Finally, the results from these two algorithms are ensembled to estimate the final degree of abnormality. The proposed method performs satisfactorily on the IEEE SP Cup-2020 dataset with an accuracy of 97.3%. Additionally, we have also tested this approach on an in-house dataset to validate its robustness.

Unsupervised Abnormality Detection Using Heterogeneous Autonomous Systems

Despite being one of the major languages in the world, research regarding Bengali handwritten numeral recognition (BHNR) isn't enough in comparison with the other prominent languages. Existing methods mostly rely on feature extraction and some older machine learning algorithms. Recent bloom in machine learning due to deep neural network especially using Convolutional Neural Network (CNN) showing promising results in this field with better accuracy. Some recent works show very good accuracy only in recognizing plain simple digits but perform poor in challenging scenario because of lack of large and versatile training dataset. In this work, we propose a method where our proposed CNN model which recognizes numerals with high degree of accuracy beyond 96%, even in most challenging noisy conditions. Initially 72000+ specimens were used from NumtaDB (85000+) dataset for training and 1700+ specimens were used as test dataset. The improvement in performance in challenging scenarios is observed, when training specimens are augmented to create a training dataset of size about 114000 specimens. The performance of our proposed model also compared with other existing works and presented here. These findings are based on Computer Vision Challenge on Bengali HandWritten Digit Recognition (2018) competition submissions.

A Deep Convolutional Neural Network for Bangla Handwritten Numeral Recognition

Autonomous aerial surveillance using drone feed is an interesting and challenging research domain. To ensure safety from intruders and potential objects posing threats to the zone being protected, it is crucial to be able to distinguish between normal and abnormal states in real-time. Additionally, we also need to consider any device malfunction. However, the inherent uncertainty embedded within the type and level of abnormality makes supervised techniques less suitable since the adversary may present a unique anomaly for intrusion. As a result, an unsupervised method for anomaly detection is preferable taking the unpredictable nature of attacks into account. Again in our case, the autonomous drone provides heterogeneous data streams consisting of images and other analog or digital sensor data, all of which can play a role in anomaly detection if they are ensembled synergistically. To that end, an ensemble detection mechanism is proposed here which estimates the degree of abnormality of analyzing the real-time image and IMU (Inertial Measurement Unit) sensor data in an unsupervised manner. First, we have implemented a Convolutional Neural Network (CNN) regression block, named AngleNet to estimate the angle between a reference image and current test image, which provides us with a measure of the anomaly of the device. Moreover, the IMU data are used in autoencoders to predict abnormality. Finally, the results from these two pipelines are ensembled to estimate the final degree of abnormality. Furthermore, we have applied adversarial attack to test the robustness and security of the proposed approach and integrated defense mechanism. The proposed method performs satisfactorily on the IEEE SP Cup-2020 dataset with an accuracy of 97.8%. Additionally, we have also tested this approach on an in-house dataset to validate its robustness.

Anomaly Detection in Unsupervised Surveillance Setting Using Ensemble of Multimodal Data with Adversarial Defense

Due to the rise of autonomous vehicles like drones and cars anomaly detection for better and robust surveillance becomes prominent for real-time recognition of normal and abnormal states. But the whole system fails if the unmanned device is unable to detect its own device's anomaly in real-time. Considering the scenario, we can make use of various data of autonomous vehicles like images, video streams, and other digital or analog sensor data to detect device anomaly. In this paper, we have demonstrated a heterogeneous system that estimates the degree of an anomaly in unmanned surveillance drone by inspecting IMU (Inertial Measurement Unit) sensor data and real-time image in an unsupervised approach. We've used AngleNet for detecting images taken in an abnormal state. On top of that, an autoencoder fed by the IMU data has been ensembled with AngleNet for evaluating the final degree of the anomaly. This proposed method is based on the result of the IEEE SP Cup 2020 which achieved 97.3 percent accuracy on the provided dataset. Besides, this approach has been evaluated on an in-house setup for substantiating its robustness.

Rouhan Noor

Papers

Handwritten Bangla Numeral Recognition Using Ensembling of Convolutional Neural Network

Unsupervised Abnormality Detection Using Heterogeneous Autonomous Systems

A Deep Convolutional Neural Network for Bangla Handwritten Numeral Recognition

Anomaly Detection in Unsupervised Surveillance Setting Using Ensemble of Multimodal Data with Adversarial Defense

Unsupervised Abnormality Detection Using Heterogenous Autonomous System