[신간의 별자리x] 우리/미술, 그리고 ‘슬픔의 박물관’

We introduce UCF101 which is currently the largest dataset of human actions.
It consists of 101 action classes, over 13k clips and 27 hours of video data.
The database consists of realistic user uploaded videos containing camera
motion and cluttered background. Additionally, we provide baseline action
recognition results on this new dataset using standard bag of words approach
with overall performance of 44.5%. To the best of our knowledge, UCF101 is
currently the most challenging dataset of actions due to its large number of
classes, large number of clips and also unconstrained nature of such clips.

UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

We consider the automated recognition of human actions in surveillance videos. Most current methods build classifiers based on complex handcrafted features computed from the raw inputs. Convolutional neural networks (CNNs) are a type of deep model that can act directly on the raw inputs. However, such models are currently limited to handling 2D inputs. In this paper, we develop a novel 3D CNN model for action recognition. This model extracts features from both the spatial and the temporal dimensions by performing 3D convolutions, thereby capturing the motion information encoded in multiple adjacent frames. The developed model generates multiple channels of information from the input frames, and the final feature representation combines information from all channels. To further boost the performance, we propose regularizing the outputs with high-level features and combining the predictions of a variety of different models. We apply the developed models to recognize human actions in the real-world environment of airport surveillance videos, and they achieve superior performance in comparison to baseline methods.

/pdf/3d-convolutional-neural-networks-for-human-action-zpxvspu401.pdf

3D Convolutional Neural Networks for Human Action Recognition

We consider the fully automated recognition of actions in uncontrolled environment. Most existing work relies on domain knowledge to construct complex handcrafted features from inputs. In addition, the environments are usually assumed to be controlled. Convolutional neural networks (CNNs) are a type of deep models that can act directly on the raw inputs, thus automating the process of feature construction. However, such models are currently limited to handle 2D inputs. In this paper, we develop a novel 3D CNN model for action recognition. This model extracts features from both spatial and temporal dimensions by performing 3D convolutions, thereby capturing the motion information encoded in multiple adjacent frames. The developed model generates multiple channels of information from the input frames, and the final feature representation is obtained by combining information from all channels. We apply the developed model to recognize human actions in real-world environment, and it achieves superior performance without relying on handcrafted features.

https://icml.cc/Conferences/2010/papers/100.pdf

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Local space-time features capture local events in video and can be adapted to the size, the frequency and the velocity of moving patterns. In this paper, we demonstrate how such features can be used for recognizing complex motion patterns. We construct video representations in terms of local space-time features and integrate such representations with SVM classification schemes for recognition. For the purpose of evaluation we introduce a new video database containing 2391 sequences of six human actions performed by 25 people in four different scenarios. The presented results of action recognition justify the proposed method and demonstrate its advantage compared to other relative approaches for action recognition.

https://www.irisa.fr/vista/Papers/2004_icpr_schuldt.pdf

Recognizing human actions: a local SVM approach

Human adaptability relies crucially on the ability to learn and merge knowledge both from supervised and unsupervised learning: the parents point out few important concepts, but then the children fill in the gaps on their own. This is particularly effective, because supervised learning can never be exhaustive and thus learning autonomously allows to discover invariances and regularities that help to generalize. In this paper we propose to apply a similar approach to the task of object recognition across domains: our model learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals how to solve a jigsaw puzzle on the same images. This secondary task helps the network to learn the concepts of spatial correlation while acting as a regularizer for the classification task. Multiple experiments on the PACS, VLCS, Office-Home and digits datasets confirm our intuition and show that this simple method outperforms previous domain generalization and adaptation solutions. An ablation study further illustrates the inner workings of our approach.

/pdf/domain-generalization-by-solving-jigsaw-puzzles-1mgngt3e9w.pdf

Domain Generalization by Solving Jigsaw Puzzles

Recent advances in rehabilitation robotics suggest that it may be possible for hand-amputated subjects to recover at least a significant part of the lost hand functionality. The control of robotic prosthetic hands using non-invasive techniques is still a challenge in real life: myoelectric prostheses give limited control capabilities, the control is often unnatural and must be learned through long training times. Meanwhile, scientific literature results are promising but they are still far from fulfilling real-life needs. This work aims to close this gap by allowing worldwide research groups to develop and test movement recognition and force control algorithms on a benchmark scientific database. The database is targeted at studying the relationship between surface electromyography, hand kinematics and hand forces, with the final goal of developing non-invasive, naturally controlled, robotic hand prostheses. The validation section verifies that the data are similar to data acquired in real-life conditions, and that recognition of different hand tasks by applying state-of-the-art signal features and machine-learning algorithms is possible. Machine-accessible metadata file describing the reported data (ISA-Tab format)

/pdf/electromyography-data-for-non-invasive-naturally-controlled-hx7jp05mf2.pdf

Electromyography data for non-invasive naturally-controlled robotic hand prostheses

Classifying materials from their appearance is a challenging problem, especially if illumination and pose conditions are permitted to change: highlights and shadows caused by 3D structure can radically alter a sample’s visual texture. Despite these difficulties, researchers have demonstrated impressive results on the CUReT database which contains many images of 61 materials under different conditions. A first contribution of this paper is to further advance the state-of-the-art by applying Support Vector Machines to this problem. To our knowledge, we record the best results to date on the CUReT database.

/pdf/on-the-significance-of-real-world-conditions-for-material-14m5018e0p.pdf

On the Significance of Real‐World Conditions for Material Classification

Although a considerable amount of work has been published on material classification, relatively little of it studies situations with considerable variation within each class. Many experiments use the exact same sample, or different patches from the same image, for training and test sets. Thus, such studies are vulnerable to effectively recognising one particular sample of a material as opposed to the material category. In contrast, this paper places firm emphasis on the capability to generalise to previously unseen instances of materials. We adopt an appearance-based strategy, and conduct experiments on a new database which contains several samples of each of eleven material categories, imaged under a variety of pose, illumination and scale conditions. Together, these sources of intra-class variation provide a stern challenge indeed for recognition. Somewhat surprisingly, the difference in performance between various state-of-the-art texture descriptors proves rather small in this task. On the other hand, we clearly demonstrate that very significant gains can be achieved via different SVM-based classification techniques. Selecting appropriate kernel parameters proves crucial. This motivates a novel recognition scheme based on a decision tree. Each node contains an SVM to split one class from all others with a kernel parameter optimal for that particular node. Hence, each decision is made using a different, optimal, class-specific metric. Experiments show the superiority of this approach over several state-of-the-art classifiers

Barbara Caputo

Papers

Recognizing human actions: a local SVM approach

Domain Generalization by Solving Jigsaw Puzzles

Electromyography data for non-invasive naturally-controlled robotic hand prostheses

On the Significance of Real‐World Conditions for Material Classification

Class-specific material categorisation