In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Very Deep Convolutional Networks for Large-Scale Image Recognition

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.

/pdf/batch-normalization-accelerating-deep-network-training-by-3dkqye6twt.pdf

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

A method for deleting obsolete files from a file system is provided. The method includes: receiving a request to delete a reference to a target file in a file system from a file reference data structure, wherein the file reference data structure includes target file names and reference file names; identifying a reference file name in the file reference data structure, wherein the reference file name includes a file name of the target file; deleting a reference file from the file system, wherein the reference file has the identified reference file name; checking whether the file system includes at least one reference file whose file name matches the file name of the target file; if there is no such reference file in the file system: deleting the target file from the file system; and deleting the file name of the target file from the file reference data structure.

Method and system for deleting obsolete files from a file system

An example method for passive compaction of a cache includes determining first metadata associated with first data and second metadata associated with second data. The first metadata includes a first retrieval time, and the second metadata includes a second retrieval time. The example method further includes obtaining a first metadata key including a first unique identifier and obtaining a second metadata key including a second unique identifier. The example method also includes generating a first data key and generating a second data key. The example method further includes writing, at a client device, the first and second data to the cache. Each of the first and second data occupy one or more contiguous blocks of physical memory in the cache, and the first and second data are stored in the cache in an order based on the relative values of the first and second retrieval times.

Technique for passive cache compaction using a least recently used cache algorithm

PROBLEM TO BE SOLVED: To provide a device which samples transactions used for access to data stored in the memory of a computer system. SOLUTION: This device samples the state of a computer system 100 which is constituted on plural levels and has a hierarchy memory where data are stored in addresses. This device includes a selector to select a memory transaction based on a first state and transaction information. The memory transaction should be processed by the hierarchy memory. A trigger operates the selector based on a second state and transaction information. A sampler stores the state of the computer system 100 discriminated by a selected instruction while the selected memory transaction is processed in the hierarchy memory. COPYRIGHT: (C)1999,JPO

Spatial and temporal sampling device in computer memory system

PROBLEM TO BE SOLVED: To provide a search result of highest relevance, that is a high quality search result to a user by a search engine, in response to a search query by the user. SOLUTION: A system (125) and a method are obtained, which specify a document, acquire one or more history data associated with the document, and generate a score for the document based on at least a part of one or more history data. COPYRIGHT: (C)2007,JPO&INPIT

Method for scoring document

PROBLEM TO BE SOLVED: To provide a method for estimating the statistic value of characteristics of interaction between instructions processed by a processor pipeline. SOLUTION: In a method for estimating the statistic value of characteristics of interaction between instructions processed by a pipeline 111 of a computer system including plural processing steps, the instructions are fetched to the first step of the pipeline 111. A set of the instructions is selected at random from among the fetched instructions and a subset of the selected set of the instructions is mutually executed at the same time. A distance between the selected sets of the instructions is specified and the state information of the computer system is recorded while the selected set of the instructions is processed by the pipeline 111. The recorded state information is transmitted to software where a statistical analysis is applied to the plural sets of the selected instructions and the statistic value of the interaction between selected sets of instructions is estimated.

Jeffrey Dean

Papers

Method and system for deleting obsolete files from a file system

Technique for passive cache compaction using a least recently used cache algorithm

Spatial and temporal sampling device in computer memory system

Method for scoring document

Method for estimating statistic value of characteristics of instruction processed by processor pipeline