Showing papers by "Michael Blumenstein published in 2018"

Integrated intelligent water-energy metering systems and informatics: Visioning a digital multi-utility service provider

Abstract: Advanced metering technologies coupled with informatics creates an opportunity to form digital multi-utility service providers. These providers will be able to concurrently collect a customers’ medium-high resolution water, electricity and gas demand data and provide user-friendly platforms to feed this information back to customers and supply/distribution utility organisations. Providers that can install low-cost integrative systems will reap the benefits of derived operational synergies and access to mass markets not bounded by historical city, state or country limits. This paper provides a vision of the required transformative process and features of an integrated multi-utility service provider covering the system architecture, opportunities and benefits, impediments and strategies, and business opportunities. The heart of the paper is focused on demonstrating data modelling processes and informatics opportunities for contemporaneously collected demand data, through illustrative examples and four informative water-energy nexus case studies. Finally, the paper provides an overview of the transformative R&D priorities to realise the vision.

Robust Feature-Based Automated Multi-View Human Action Recognition System

Abstract: Automated human action recognition has the potential to play an important role in public security, for example, in relation to the multiview surveillance videos taken in public places, such as train stations or airports. This paper compares three practical, reliable, and generic systems for multiview video-based human action recognition, namely, the nearest neighbor classifier, Gaussian mixture model classifier, and the nearest mean classifier. To describe the different actions performed in different views, view-invariant features are proposed to address multiview action recognition. These features are obtained by extracting the holistic features from different temporal scales which are modeled as points of interest which represent the global spatial-temporal distribution. Experiments and cross-data testing are conducted on the KTH, WEIZMANN, and MuHAVi datasets. The system does not need to be retrained when scenarios are changed which means the trained database can be applied in a wide variety of environments, such as view angle or background changes. The experiment results show that the proposed approach outperforms the existing methods on the KTH and WEIZMANN datasets.

CRISPR/Cas9 cleavage efficiency regression through boosting algorithms and Markov sequence profiling.

Abstract: Motivation CRISPR/Cas9 system is a widely used genome editing tool. A prediction problem of great interests for this system is: how to select optimal single-guide RNAs (sgRNAs), such that its cleavage efficiency is high meanwhile the off-target effect is low. Results This work proposed a two-step averaging method (TSAM) for the regression of cleavage efficiencies of a set of sgRNAs by averaging the predicted efficiency scores of a boosting algorithm and those by a support vector machine (SVM). We also proposed to use profiled Markov properties as novel features to capture the global characteristics of sgRNAs. These new features are combined with the outstanding features ranked by the boosting algorithm for the training of the SVM regressor. TSAM improved the mean Spearman correlation coefficiencies comparing with the state-of-the-art performance on benchmark datasets containing thousands of human, mouse and zebrafish sgRNAs. Our method can be also converted to make binary distinctions between efficient and inefficient sgRNAs with superior performance to the existing methods. The analysis reveals that highly efficient sgRNAs have lower melting temperature at the middle of the spacer, cut at 5'-end closer parts of the genome and contain more 'A' but less 'G' comparing with inefficient ones. Comprehensive further analysis also demonstrates that our tool can predict an sgRNA's cutting efficiency with consistently good performance no matter it is expressed from an U6 promoter in cells or from a T7 promoter in vitro. Availability and implementation Online tool is available at http://www.aai-bioinfo.com/CRISPR/. Python and Matlab source codes are freely available at https://github.com/penn-hui/TSAM. Supplementary information Supplementary data are available at Bioinformatics online.

Signature and Logo Detection using Deep CNN for Document Image Retrieval

Abstract: Signature and logo as a query are important for content-based document image retrieval from a scanned document repository. This paper deals with signature and logo detection from a repository of scanned documents, which can be used for document retrieval using signature or logo information. A large intra-category variance among signature and logo samples poses challenges to traditional hand-crafted feature extraction-based approaches. Hence, the potential of deep learning-based object detectors namely, Faster R-CNN and YOLOv2 were examined for automatic detection of signatures and logos from scanned administrative documents. Four different network models namely ZF, VGG16, VGG_M, and YOLOv2 were considered for analysis and identifying their potential in document image retrieval. The experiments were conducted on the publicly available "Tobacco-800" dataset. The proposed approach detects Signatures and Logos simultaneously. The results obtained from the experiments are promising and at par with the existing methods.

SSBC 2018: Sclera Segmentation Benchmarking Competition

Abstract: This paper summarises the results of the Sclera Segmentation Benchmarking Competition (SSBC 2018). It was organised in the context of the 11th IAPR International Conference on Biometrics (ICB 2018). The aim of this competition was to record the developments on sclera segmentation in the cross-sensor environment (sclera trait captured using multiple acquiring sensors). Additionally, the competition also aimed to gain the attention of researchers on this subject of research. For the purpose of benchmarking, we have developed two datasets of sclera images captured using different sensors. The first dataset was collected using a DSLR camera and the second one was collected using a mobile phone camera. The first dataset is the Multi-Angle Sclera Dataset (MASD version 1), which was used in the context of the previous versions of sclera segmentation competitions. The images in the second dataset were captured using .a mobile phone rear camera of 8-megapixel. As baseline manual segmentation mask of the sclera images from both the datasets were developed. Precision and recall-based statistical measures were employed to evaluate the effectiveness of the submitted segmentation technique and to rank them. Six algorithms were submitted towards the segmentation task. This paper analyses the results produced by these algorithms/system and defines a way forward for this subject of research. Both the datasets along with some of the accompanying ground truth/baseline mask will be freely available for research purposes upon request to authors by email.

Shark Detection from Aerial Imagery Using Region-Based CNN, a Study

Abstract: Shark attacks have been a very sensitive issue for Australians and many other countries. Thus, providing safety and security around beaches is very fundamental in the current climate. Safety for both human beings and underwater creatures (sharks, whales, etc.) in general is essential while people continue to visit and use the beaches heavily for recreation and sports. Hence, an efficient, automated and real-time monitoring approach on beaches for detecting various objects (e.g. human activities, large fish, sharks, whales, surfers, etc.) is necessary to avoid unexpected casualties and accidents. The use of technologies such as drones and machine learning techniques are promising directions in such challenging circumstances. This paper investigates the potential of Region-based Convolutional Neural Networks (R-CNN) for detecting various marine objects, and Sharks in particular. Three network architectures namely Zeiler and Fergus (ZF), Visual Geometry Group (VGG16), and VGG_M were considered for analysis and identifying their potential. A dataset consisting of 3957 video frames were used for experiments. VGG16 architecture with faster-R-CNN performed better than others, with an average precision of 0.904 for detecting Sharks.

Person Head Detection in Multiple Scales Using Deep Convolutional Neural Networks

Abstract: Person detection is an important problem in computer vision with many real-world applications. The detection of a person is still a challenging task due to variations in pose, occlusions and lighting conditions. The purpose of this study is to detect human heads in natural scenes acquired from a publicly available dataset of Hollywood movies. In this work, we have used state-of-the-art object detectors based on deep convolutional neural networks. These object detectors include region-based convolutional neural networks using region proposals for detections. Also, object detectors that detect objects in the single-shot by looking at the image only once for detections. We have used transfer learning for fine-tuning the network already trained on a massive amount of data. During the fine-tuning process, the models having high mean Average Precision (mAP) are used for evaluation of the test dataset. Experimental results show that Faster R-CNN [18] and SSD MultiBox [13] with VGG16 [21] perform better than YOLO [17] and also demonstrate significant improvements against several baseline approaches.

Real-Time Drone Surveillance and Population Estimation of Marine Animals from Aerial Imagery

Abstract: Video analysis is being rapidly adopted by marine biologists to asses the population and migration of marine animals. Manual analysis of videos by human observers is labor intensive and prone to error. The automatic analysis of videos using state-of-the-art deep learning object detectors provides a cost-effective way for the study of marine animals population and their ecosystem. However, there are many challenges associated with video analysis such as background clutter, illumination, occlusions, and deformation. Due to the high-density of objects in the images and sever occlusion, current state-of-the-art object often results in multiple detections. Therefore, customized Non-Maxima-Suppression is proposed after the detections to suppress false positives which significantly improves the counting and mean average precision of the detections. An end-to-end deep learning framework of Faster-RCNN [1] was adopted for detections with base architectures of VGG16 [2], VGGM [3] and ZF [4].

Robust 2D Joint Sparse Principal Component Analysis With F-Norm Minimization For Sparse Modelling: 2D-RJSPCA

Abstract: Principal component analysis (PCA) is widely used methods for dimensionality reduction and Lots of variants have been proposed to improve the robustness of algorithm, however, these methods suffer from the fact that PCA is linear combination which makes it difficult to interpret complex nonlinear data, and sensitive to outliers or cannot extract features consistently, i.e., collectively; PCA may still require measuring all input features. 2DPCA based on $\ell _{1} -norm$ has been recently used for robust dimensionality reduction in the image domain but still sensitive to noise. In this paper, we introduce robust formation of 2DPCA by centering the data using the optimized mean for two-dimensional joint sparse as well as effectively combining the robustness of 2DPCA and the sparsity-inducing lasso regularization. Optimal mean helps to improve the robustness of joint sparse PCA further. The distance in spatial dimension is measure in F-norm and sum of different datapoint uses 1-norm. 2DR-JSPCA imposes joint sparse constraints on its objective function whereas additional plenty term help to deal with outliers efficiently. Both theoretical and empirical results on six publicly available benchmark datasets shows that Optimal mean 2DR-JSPCA provides better performance for dimensionality reduction as compare to non-sparse (2DPCA and 2DPCA-L1) and sparse (SPCA, JSPCA).

Bag-of-Visual-Words for Signature-Based Multi-Script Document Retrieval

Abstract: An end-to-end architecture for multi-script document retrieval using handwritten signatures is proposed in this paper. The user supplies a query signature sample and the system exclusively returns a set of documents that contain the query signature. In the first stage, a component-wise classification technique separates the potential signature components from all other components. A bag-of-visual-words powered by SIFT descriptors in a patch-based framework is proposed to compute the features and a Support Vector Machine (SVM)-based classifier was used to separate signatures from the documents. In the second stage, features from the foreground (i.e. signature strokes) and the background spatial information (i.e. background loops, reservoirs etc.) were combined to characterize the signature object to match with the query signature. Finally, three distance measures were used to match a query signature with the signature present in target documents for retrieval. The `Tobacco' document database and an Indian script database containing 560 documents of Devanagari (Hindi) and Bangla scripts were used for the performance evaluation. The proposed system was also tested on noisy documents and promising results were obtained. A comparative study shows that the proposed method outperforms the state-of-the-art approaches.

Thai Automatic signature verification System Employing Textural Features

Abstract: This study focuses on a comprehensive study of Automatic Signature Verification (ASV) for off-line Thai signatures; an investigation was carried out to characterise the challenges in Thai ASV and to baseline the performance of Thai ASV employing baseline features, being Local Binary Pattern, Local Directional Pattern, Local Binary and Directional Patterns combined (LBDP), and the baseline shape/feature-based hidden Markov model. As there was no publicly available Thai signature database found in the literature, the authors have developed and proposed a database considering real-world signatures from Thailand. The authors have also identified their latent challenges and characterised Thai signature-based ASV. The database consists of 5,400 signatures from 100 signers. Thai signatures could be bi-script in nature, considering the fact that a single signature can contain only Thai or Roman characters or contain both Roman and Thai, which poses an interesting challenge for script-independent SV. Therefore, along with the baseline experiments, the investigation on the influence and nature of bi-script ASV was also conducted. From the equal error rates and Bhattacharyya distance, the score achieved in the experiments indicate that the Thai SV scenario is a script-independent problem. The open research area on this subject of research has also been addressed.

ICFHR 2018 Competition on Thai Student Signatures and Name Components Recognition and Verification (TSNCRV2018)

Abstract: This paper summarises the results of the competition on the 1st Thai Student Signature and Name Components Recognition and Verification (TSNCRV 2018). It was organised in the context of the 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018). The aim of this competition was to record the development and gain attention on Thai student signatures and name component recognition and verification. Two different types of datasets were used for the competition: the first dataset contains Thai student signatures and the second dataset contains Thai student name components. Signatures and name components from 100 volunteers each were included in the competition datasets. For Thai signature dataset, there are 30 genuine signatures, 12 skilled and 12 simple forgeries for each writer. For Thai name components, there are 30 genuine and 12 skilfully forged name components for each writer. For both the datasets the individuals were asked to write their name/signature in the given space on a white piece of paper for number of time (with a pause between each 10 samples). The skilled forgers were asked practice emitting the original signature for certain number of times till they fill skilled to forge. Five teams from distinguish labs submitted their systems. This paper analysed the results produced by these algorithms/systems using a performance measure and defined a way forward for this subject of research. Both the datasets along with some of the accompanying ground truth/baseline mask will be made freely available for research purposes via the TC10/TC11.

A Study on Idiosyncratic Handwriting with Impact on Writer Identification

Abstract: In this paper, we study handwriting idiosyncrasy in terms of its structural eccentricity. In this study, our approach is to find idiosyncratic handwritten text components and model the idiosyncrasy analysis task as a machine learning problem supervised by human cognition. We employ the Inception network for this purpose. The experiments are performed on two publicly available databases and an in-house database of Bengali offline handwritten samples. On these samples, subjective opinion scores of handwriting idiosyncrasy are collected from handwriting experts. We have analyzed the handwriting idiosyncrasy on this corpus which comprises the perceptive ground-truth opinion. We also investigate the effect of idiosyncratic text on writer identification by using the SqueezeNet. The performance of our system is promising.

Offline Bengali Writer Verification by PDF-CNN and Siamese Net

Abstract: Automated handwriting analysis is a popular area of research owing to the variation of writing patterns. In this research area, writer verification is one of the most challenging branches, having direct impact on biometrics and forensics. In this paper, we deal with offline writer verification on complex handwriting patterns. Therefore, we choose a relatively complex script, i.e., Indic Abugida script Bengali (or, Bangla) containing more than 250 compound characters. From a handwritten sample, the probability distribution functions (PDFs) of some handcrafted features are obtained and input to a convolutional neural network (CNN). For such a CNN architecture, we coin the term "PDFCNN", where handcrafted feature PDFs are hybridized with auto-derived CNN features. Such hybrid features are then fed into a Siamese neural network for writer verification. The experiments are performed on a Bengali offline handwritten dataset of 100 writers. Our system achieves encouraging results, which sometimes exceed the results of state-of-the-art techniques on writer verification.

Fourier Transform based Features for Clean and Polluted Water Image Classification

Abstract: Water image classification is challenging because water images of ocean or river share the same properties with images of polluted water such as fungus, waste and rubbish. In this paper, we present a method for classifying clean and polluted water images. The proposed method explores Fourier transform based features for extracting texture properties of clean and polluted water images. Fourier spectrum of each input image is divided into several sub-regions based on angle and spatial information. For each region over the spectrum, the proposed method extracts mean and variance features using intensity values, which results in a feature matrix. The feature matrix is then passed to an SVM classifier for the classification of clean and polluted water images. Experimental results on classes of clean and polluted water images show that the proposed method is effective. Furthermore, a comparative study with the state-of-the-art method shows that the proposed method outperforms the existing method in terms of classification rate, recall, precision and F-measure.

An investigation of discrete Hidden Markov Models on handwritten short answer assessment system

Abstract: This paper presents an investigation of an off-line automatic assessment system utilising discrete Hidden Markov Models. A set of geometric features were extracted from handwritten words and were later classified by HMMs. There were two training datasets employed in the experiments; the first training dataset contained all correct answers to the questions whereas another training dataset contained both correct and incorrect answers to the questions. Datasets contained 3,000 and 3,400 handwritten samples, respectively. The experiments yielded promising results whereby the highest recognition rate of 91.90% with a 100% accuracy was achieved on our database.

Evaluation of Gist Operator for Document Image Retrieval

Abstract: As digitised documents normally contain a large variety of structures, a page segmentation- and layout-free method for document image retrieval is preferable. In this research work, therefore, wavelet transform as a transform-based approach is initially used to provide different under-sampled images from the original image. Then, Gist operator, as a feature extraction technique, is employed to extract a set of global features from the original image as well as the sub-images obtained from the wavelet transform. Moreover, the column-wise variances of the values in each sub-image are computed and they are then concatenated to obtain another set of features. Considering each feature set, locality-sensitive hashing is employed to compute similarity distances between a query and the document images in the database. Finally, a classifier fusion technique using the mean function is taken into account to provide a document image retrieval result. The combination of these features and a clustering score fusion strategy provides higher document image retrieval accuracy. Two different databases of the document image are considered for experimentation. The results obtained from the experimental study are detailed and the results are encouraging.

Matching Pursuit Based on Kernel Non-Second Order Minimization

Abstract: The orthogonal matching pursuit (OMP) is an important sparse approximation algorithm to recover sparse signals from compressed measurements. However, most MP algorithms are based on the mean square error(MSE) to minimize the recovery error, which is suboptimal when there are outliers. In this paper, we present a new robust OMP algorithm based on kernel non-second order statistics (KNS-OMP), which not only takes advantages of the outlier resistance ability of correntropy but also further extends the second order statistics based correntropy to a non-second order similarity measurement to improve its robustness. The resulted framework is more accurate than the second order ones in reducing the effect of outliers. Experimental results on synthetic and real data show that the proposed method achieves better performances compared with existing methods.

More Realistic and Efficient Face-Based Mobile Authentication using CNNs

Abstract: In this work, we propose a more realistic and efficient facebased mobile authentication technique using CNNs. This paper discusses and explores an inevitable problem of using face images for mobile authentication, taken from varying distances with a front/selfie camera of the mobile phone. Incidentally, once an individual comes towards a certain distance from the camera, the face images get large and appear over-sized. Simultaneously sharp features of some portions of the face, such as forehead, cheek, and chin are changed completely. As a result, the face features change and the impact increases exponentially once the individual crosses a certain distance and gradually approaches towards the front camera. This work proposes a solution (achieving better accuracy and facial features, whereby face images were cropped and aligned around its close bounding box) to mitigate the aforementioned identified gap. The work investigated different frontier face detection and recognition techniques to justify the proposed solution. Among all the employed methods evaluated, CNNs worked best. For a quantitative comparison of the proposed method, manually cropped face images/annotations of the face images along with their close boundary were prepared. In turn, we have developed a database considering the above-mentioned scenario for 40 individuals, which will be publicly available for academic research purposes. The experimental results achieved indicate a successful implementation of the proposed method and the performance of the proposed technique is also found to be superior in comparison to the existing state-of-the-art.

Similar Gesture Recognition using Hierarchical Classification Approach in RGB Videos

Abstract: Recognizing human actions from the video streams has become one of the very popular research areas in computer vision and deep learning in the recent years. Action recognition is wildly used in different scenarios in real life, such as surveillance, robotics, healthcare, video indexing and human-computer interaction. The challenges and complexity involved in developing a video-based human action recognition system are manifold. In particular, recognizing actions with similar gestures and describing complex actions is a very challenging problem. To address these issues, we study the problem of classifying human actions using Convolutional Neural Networks (CNN) and develop a hierarchical 3DCNN architecture for similar gesture recognition. The proposed model firstly combines similar gesture pairs into one class, and classify them along with all other class, as a stage-1 classification. In stage-2, similar gesture pairs are classified individually, which reduces the problem to binary classification. We apply and evaluate the developed models to recognize the similar human actions on the HMDB51 dataset. The result shows that the proposed model can achieve high performance in comparison to the state-of-the-art methods.

Cognitive Analysis for Reading and Writing of Bengali Conjuncts

Abstract: In this paper, we study the difficulties arising in reading and writing of Bengali conjunct characters by human-beings. Such difficulties appear when the human cognitive system faces certain obstructions in effortlessly reading/writing. In our computer-based investigation, we consider the reading/writing difficulty analysis task as a machine learning problem supervised by human perception. To this end, we employ two distinct models: (a) an auto-derived feature-based Inception network and (b) a hand-crafted feature-based SVM (Support Vector Machine). Two commonly used Bengali printed fonts and three contemporary handwritten databases are used for collecting subjective opinion scores from human readers/writers. On this corpus, which contains the perceptive ground-truth opinion of reading/writing complications, we have undertaken to conduct the experiments. The experimental results obtained on various types of conjunct characters are promising.

An End-to-End Hierarchical Classification Approach for Similar Gesture Recognition

Abstract: Human action recognition from the RGB video is widely applied on varies real applications. Many works have been done by researchers in computer vision and machine learning area to address the challenges and complexity involved in video-based human action recognition. Deep learning approaches including Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) have been introduced in the human action recognition research area. However, due to the drawbacks of the CNNs, recognizing actions with similar gestures and describing complex actions is still very challenging. Hence, an end-to-end hierarchical classification architecture has been proposed in this paper to resolve the confusion between similar gesture. The proposed approach firstly classifies the whole dataset and generates the accuracy for each class in stage 1. Based on the confusion matrix obtained from stage-1, the approach combines the most confused similar gesture pairs into one class, and classify them along with all other class, in the stage-2. In stage 3, similar gesture pairs will be classified by binary classifiers, which will increase the performance of each class and the overall accuracy. We apply and evaluate the developed models to recognize the similar human actions on the both KTH and UCF101 dataset. The result shows that the proposed approach can boost the classification performance on both the datasets. The proposed architecture is robust and any classification technique can be used in stage 1 and stage 2.

Connectivity Based Method for Clustering Microbial Communities from Metagenomics Data of Water and Soil Samples

Abstract: Understanding microbial community structure of metagenomics water and soil samples is a key process in discovering functions and impact of microorganisms on human and animal health. Evolution of Next Generation Sequencing (NGS) technology has encouraged researchers to sequence large quantity of microbial data from environmental sources. Clustering marker gene sequences into Operational Taxonomic Units (OTU) is the most significant task in microbial community analysis. Several methods have been developed over the years to improve OTU picking strategies. However, building strongly connected OTUs is a major issue in majority of these methods. Herein we present ConClust, a novel method for clustering OTUs that is based on quantifying connectivity among the sequences. Experimental analysis on two synthetic datasets and two real world datasets from water and soil samples demonstrate that our method can mine robust OTUs. Our method can be highly benelicial to study functions of known and unknown microbes and analyze their positive and negative effect on the environment as well as human and animal health.