Open AccessProceedings Article
Selective Classification for Deep Neural Networks
Yonatan Geifman,Ran El-Yaniv +1 more
- Vol. 30, pp 4878-4887
Reads0
Chats0
TLDR
A method to construct a selective classifier given a trained neural network, which allows a user to set a desired risk level and the classifier rejects instances as needed, to grant the desired risk (with high probability).Abstract:
Selective classification techniques (also known as reject option) have not yet been considered in the context of deep neural networks (DNNs). These techniques can potentially significantly improve DNNs prediction performance by trading-off coverage. In this paper we propose a method to construct a selective classifier given a trained neural network. Our method allows a user to set a desired risk level. At test time, the classifier rejects instances as needed, to grant the desired risk (with high probability). Empirical results over CIFAR and ImageNet convincingly demonstrate the viability of our method, which opens up possibilities to operate DNNs in mission-critical applications. For example, using our method an unprecedented 2% error in top-5 ImageNet classification can be guaranteed with probability 99.9%, with almost 60% test coverage.read more
Citations
More filters
Book ChapterDOI
Into the unknown: Active monitoring of neural networks
TL;DR: This work proposes an algorithmic framework for active monitoring of neural-network classifiers that allows for their deployment in dynamic environments where unknown input classes appear frequently, and adjusts the framework to novel inputs incrementally, thereby improving short-term reliability of the classification.
Posted Content
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples.
Vikash Sehwag,Arjun Nitin Bhagoji,Liwei Song,Chawin Sitawarin,Daniel Cullina,Mung Chiang,Prateek Mittal +6 more
TL;DR: This study uses 11 state-of-the-art neural network models trained on 3 image datasets of varying complexity to demonstrate the success of open-world evasion attacks, where adversarial examples are generated from out- of-distribution inputs (OOD adversarialExamples).
Journal ArticleDOI
Solving math word problems with process- and outcome-based feedback
Jonathan Uesato,Nate Kushman,Ramana Kumar,Francis Song,Noah Siegel,L. Wang,Antonia Creswell,Geoffrey Irving,Irina Higgins +8 more
TL;DR: This paper showed that pure outcome-based supervision produces similar final-answer error rates with less label supervision, but for correct reasoning steps, it is necessary to use process-based supervised or supervision from learned reward models that emulate processbased feedback.
Journal ArticleDOI
A new method to control error rates in automated species identification with deep learning algorithms.
Sébastien Villon,David Mouillot,David Mouillot,Marc Chaumont,Marc Chaumont,Gérard Subsol,Thomas Claverie,Sébastien Villéger +7 more
TL;DR: A new framework to control the error rate of DLAs was proposed, where for each species, a confidence threshold was automatically computed using a training dataset independent from the one used to train the DLAs to minimize the risk of misclassification.
Posted Content
Confidence from Invariance to Image Transformations
TL;DR: A technique for automatically detecting the classification errors of a pre-trained visual classifier and applies it to novelty detection scenarios, establishing a measure of confidence in classifier's decision by analyzing the invariance of its decision under various transformations.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Journal ArticleDOI
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.
Proceedings ArticleDOI
Very deep convolutional neural network based image classification using small training sample size
Shuying Liu,Weihong Deng +1 more
TL;DR: In this article, a modified VGG-16 network was used to fit CIFAR-10 without severe overfitting and achieved 8.45% error rate on the dataset.
Journal ArticleDOI
An optimum character recognition system using decision functions
TL;DR: The character recognition problem, usually resulting from characters being corrupted by printing deterioration and/or inherent noise of the devices, is considered from the viewpoint of statistical decision theory and the optimum recogition is obtained.