Journal ArticleDOI
HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition
TLDR
The central concept is to use the rich temporal information provided by events to create contexts in the form of time-surfaces which represent the recent temporal activity within a local spatial neighborhood and it is demonstrated that this concept can robustly be used at all stages of an event-based hierarchical model.Abstract:
This paper describes novel event-based spatio-temporal features called time-surfaces and how they can be used to create a hierarchical event-based pattern recognition architecture. Unlike existing hierarchical architectures for pattern recognition, the presented model relies on a time oriented approach to extract spatio-temporal features from the asynchronously acquired dynamics of a visual scene. These dynamics are acquired using biologically inspired frameless asynchronous event-driven vision sensors. Similarly to cortical structures, subsequent layers in our hierarchy extract increasingly abstract features using increasingly large spatio-temporal windows. The central concept is to use the rich temporal information provided by events to create contexts in the form of time-surfaces which represent the recent temporal activity within a local spatial neighborhood. We demonstrate that this concept can robustly be used at all stages of an event-based hierarchical model. First layer feature units operate on groups of pixels, while subsequent layer feature units operate on the output of lower level feature units. We report results on a previously published 36 class character recognition task and a four class canonical dynamic card pip task, achieving near 100 percent accuracy on each. We introduce a new seven class moving face recognition task, achieving 79 percent accuracy.read more
Citations
More filters
Journal ArticleDOI
Understanding Human Reactions Looking at Facial Microexpressions With an Event Camera
TL;DR: In this article , the authors proposed to model expressions with event cameras, bio-inspired vision sensors that have found application within the Industry 4.0 scope, showing that using event cameras can understand human reactions by only observing facial expressions.
Posted Content
Event Data Association via Robust Model Fitting for Event-based Object Tracking
TL;DR: Zhang et al. as discussed by the authors proposed a novel Event Data Association approach (called EDA) to explicitly address the data association problem, which seeks for event trajectories that best fit the event data, in order to perform unifying data association.
Book ChapterDOI
A Noise Filter for Dynamic Vision Sensor Based on Spatiotemporal Correlation and Hot Pixel Detection
Journal ArticleDOI
Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation
Dongsheng Wang,Xu Jia,Yan Zhang,Xinyu Zhang,Yaoyuan Wang,Ziyang Zhang,Rongming Wang,Huchuan Lu +7 more
TL;DR: In this article , a dual-memory aggregation network (DMANet) is proposed to leverage both long and short memory along event streams to aggregate effective information for object detection, where long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars at neighboring time intervals.
Proceedings ArticleDOI
An Interpretable Pixel Intensity Reconstruction Model for Asynchronous Event Camera
TL;DR: In this paper , the amplitude-frequency characteristic of the recovered logarithm of the intensity is used to construct the frame-based image to complete computer vision (CV) tasks.
References
More filters
Journal ArticleDOI
Distinctive Image Features from Scale-Invariant Keypoints
TL;DR: This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene and can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Journal ArticleDOI
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
TL;DR: It is shown that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex.
Proceedings Article
Large Scale Distributed Deep Networks
Jeffrey Dean,Greg S. Corrado,Rajat Monga,Kai Chen,Matthieu Devin,Mark Z. Mao,Marc'Aurelio Ranzato,Andrew W. Senior,Paul A. Tucker,Ke Yang,Quoc V. Le,Andrew Y. Ng +11 more
TL;DR: This paper considers the problem of training a deep network with billions of parameters using tens of thousands of CPU cores and develops two algorithms for large-scale distributed training, Downpour SGD and Sandblaster L-BFGS, which increase the scale and speed of deep network training.