Face detection and alignment in unconstrained environment are challenging due to various poses, illuminations, and occlusions. Recent studies show that deep learning approaches can achieve impressive performance on these two tasks. In this letter, we propose a deep cascaded multitask framework that exploits the inherent correlation between detection and alignment to boost up their performance. In particular, our framework leverages a cascaded architecture with three stages of carefully designed deep convolutional networks to predict face and landmark location in a coarse-to-fine manner. In addition, we propose a new online hard sample mining strategy that further improves the performance in practice. Our method achieves superior accuracy over the state-of-the-art techniques on the challenging face detection dataset and benchmark and WIDER FACE benchmarks for face detection, and annotated facial landmarks in the wild benchmark for face alignment, while keeps real-time performance.

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Pedestrian detection is a key problem in computer vision, with several applications that have the potential to positively impact quality of life. In recent years, the number of approaches to detecting pedestrians in monocular images has grown steadily. However, multiple data sets and widely varying evaluation protocols are used, making direct comparisons difficult. To address these shortcomings, we perform an extensive evaluation of the state of the art in a unified framework. We make three primary contributions: 1) We put together a large, well-annotated, and realistic monocular pedestrian detection data set and study the statistics of the size, position, and occlusion patterns of pedestrians in urban scenes, 2) we propose a refined per-frame evaluation methodology that allows us to carry out probing and informative comparisons, including measuring performance in relation to scale and occlusion, and 3) we evaluate the performance of sixteen pretrained state-of-the-art detectors across six data sets. Our study allows us to assess the state of the art and provides a framework for gauging future efforts. Our experiments show that despite significant progress, performance still has much room for improvement. In particular, detection is disappointing at low resolutions and for partially occluded pedestrians.

/pdf/pedestrian-detection-an-evaluation-of-the-state-of-the-art-3rhn1vtfew.pdf

Pedestrian Detection: An Evaluation of the State of the Art

Multi-resolution image features may be approximated via extrapolation from nearby scales, rather than being computed explicitly. This fundamental insight allows us to design object detection algorithms that are as accurate, and considerably faster, than the state-of-the-art. The computational bottleneck of many modern detectors is the computation of features at every scale of a finely-sampled image pyramid. Our key insight is that one may compute finely sampled feature pyramids at a fraction of the cost, without sacrificing performance: for a broad family of features we find that features computed at octave-spaced scale intervals are sufficient to approximate features on a finely-sampled pyramid. Extrapolation is inexpensive as compared to direct feature computation. As a result, our approximation yields considerable speedups with negligible loss in detection accuracy. We modify three diverse visual recognition systems to use fast feature pyramids and show results on both pedestrian detection (measured on the Caltech, INRIA, TUD-Brussels and ETH data sets) and general object detection (measured on the PASCAL VOC). The approach is general and is widely applicable to vision algorithms requiring fine-grained multi-scale analysis. Our approximation is valid for images with broad spectra (most natural images) and fails for images with narrow band-pass spectra (e.g., periodic textures).

/pdf/fast-feature-pyramids-for-object-detection-1fqa9ncvlh.pdf

Fast Feature Pyramids for Object Detection

By combining Histograms of Oriented Gradients (HOG) and Local Binary Pattern (LBP) as the feature set, we propose a novel human detection approach capable of handling partial occlusion. Two kinds of detectors, i.e., global detector for whole scanning windows and part detectors for local regions, are learned from the training data using linear SVM. For each ambiguous scanning window, we construct an occlusion likelihood map by using the response of each block of the HOG feature to the global detector. The occlusion likelihood map is then segmented by Mean-shift approach. The segmented portion of the window with a majority of negative response is inferred as an occluded region. If partial occlusion is indicated with high likelihood in a certain scanning window, part detectors are applied on the unoccluded regions to achieve the final classification on the current scanning window. With the help of the augmented HOG-LBP feature and the global-part occlusion handling method, we achieve a detection rate of 91.3% with FPPW= 10−6, 94.7% with FPPW= 10−5, and 97.9% with FPPW= 10−4 on the INRIA dataset, which, to our best knowledge, is the best human detection performance on the INRIA dataset. The global-part occlusion handling method is further validated using synthesized occlusion data constructed from the INRIA and Pascal dataset.

/pdf/an-hog-lbp-human-detector-with-partial-occlusion-handling-vm4uc4vsk3.pdf

An HOG-LBP human detector with partial occlusion handling

We study the performance of ‘integral channel features’ for image classification tasks,
focusing in particular on pedestrian detection. The general idea behind integral channel features is that multiple registered image channels are computed using linear and
non-linear transformations of the input image, and then features such as local sums, histograms, and Haar features and their various generalizations are efficiently computed
using integral images. Such features have been used in recent literature for a variety of
tasks – indeed, variations appear to have been invented independently multiple times.
Although integral channel features have proven effective, little effort has been devoted to
analyzing or optimizing the features themselves. In this work we present a unified view
of the relevant work in this area and perform a detailed experimental evaluation. We
demonstrate that when designed properly, integral channel features not only outperform
other features including histogram of oriented gradient (HOG), they also (1) naturally
integrate heterogeneous sources of information, (2) have few parameters and are insensitive to exact parameter settings, (3) allow for more accurate spatial localization during
detection, and (4) result in fast detectors when coupled with cascade classifiers.

/pdf/integral-channel-features-4xb26p13f5.pdf

Integral Channel Features

We integrate the cascade-of-rejectors approach with the Histograms of Oriented Gradients (HoG) features to achieve a fast and accurate human detection system. The features used in our system are HoGs of variable-size blocks that capture salient features of humans automatically. Using AdaBoost for feature selection, we identify the appropriate set of blocks, from a large set of possible blocks. In our system, we use the integral image representation and a rejection cascade which significantly speed up the computation. For a 320 × 280 image, the system can process 5 to 30 frames per second depending on the density in which we scan the image, while maintaining an accuracy level similar to existing methods.

/pdf/fast-human-detection-using-a-cascade-of-histograms-of-11xedko3vi.pdf

Fast Human Detection Using a Cascade of Histograms of Oriented Gradients

Unsolicited commercial email (UCE), also known as spam, has been a major problem on the Internet. In the past, researchers have addressed this problem as a text classification or categorization problem. However, as spammers' techniques continue to evolve and the genre of email content becomes more and more diverse, text-based anti-spam approaches alone are no longer sufficient. In this paper, we propose a novel anti-spam system which utilizes visual clues, in addition to text information in the email body, to determine whether a message is spam. We analyze a large collection of spam emails containing images and identify a number of useful visual features for this application. We then propose using one-class support vector machines (SVM) as the underlying base classifier for anti-spam filtering. The experimental results demonstrate that the proposed system can add significant filtering power to the existing text-based anti-spam filters.

Using visual features for anti-spam filtering

Due to variations of lighting conditions, camera hardware settings, and the range of skin coloration among human beings, a pre-defined skin-color model cannot accurately capture the wide distribution of skin colors in individual images. In this paper, we propose an adaptive skin-detection method, which allows modeling true skin-color distribution with significantly higher accuracy and flexibility than other methods attain. In principle, the proposed method follows a two-step process. For a given image, we first perform a rough skin classification using a generic skin model, which defines the skin-similar space. In the second step, a Gaussian mixture model (GMM), specific to the image under consideration and refined from the skin-similar space, is derived using the standard expectation-maximization (EM) algorithm. Then, we use an SVM (support vector machine) classifier to identify the skin Gaussian from the trained GMM (which contains two Gaussian components) by incorporating spatial and shape information of the skin pixels. This adaptive method can be applied to both still images and video applications. Results of extensive experiments performed on live video sequences and large image databases have demonstrated the effectiveness and benefits of the proposed model.

http://lbmedia.ece.ucsb.edu/resources/ref/fg04ppt.pdf

Adaptive learning of an accurate skin-color model

We propose an adaptive skin-detection method, which allows modelling and detection of the true skin-color pixels with significantly higher accuracy and flexibility than previous methods. In principle, the proposed approach follows a two-step process. For a given image, we first perform a rough skin classification using a generic skin-model which defines the Skin-Similar space. The Skin-Similar space often contains many non-skin pixels due to the inevitable overlap in the color space between skin pixels and some non-skin pixels under the generic skin-model. The objective of the second step is to reduce the false-positive rate by analyzing the image under consideration. Specifically, in the second step, a Gaussian Mixture Model (GMM), specific to the image under consideration and refined from its Skin-Similar space, is derived using the standard Expectation-Maximization (EM) algorithm. We then use a Support Vector Machine (SVM) classifier to identify the skin Gaussian from the trained GMM by incorporating spatial and shape information of the skin pixels. Moreover, we examine how the improvement on skin detection by this adaptive skin-model impacts the detection accuracy in the application of Objectionable Image Filtering. We further propose a two-level classification scheme based on hierarchical bagging to improve the accuracy. Results of extensive experiments on large databases demonstrate the effectiveness and benefits of our adaptive skin-model.

An adaptive skin model and its application to objectionable image filtering

A method and system is presented for detecting humans in images of a scene acquired by a camera. Gradients of pixels in the image are determined and sorted into bins of a histogram. An integral image is stored for each bin of the histogram. Features are extracted fom the integral images, the extracted features corresponding to a subset of a substantially larger set of variably sized and randomly selected blocks of pixels in the test image. The features are applied to a cascaded classifier to determine whether the test image includes a human or not.

Qiang Zhu

Papers

Fast Human Detection Using a Cascade of Histograms of Oriented Gradients

Using visual features for anti-spam filtering

Adaptive learning of an accurate skin-color model

An adaptive skin model and its application to objectionable image filtering

Method for detecting humans in images