scispace - formally typeset

Proceedings ArticleDOI

Image compression with Stochastic Winner-Take-All Auto-Encoder

14 Mar 2017-pp 1512-1516

TL;DR: This paper addresses the problem of image compression using sparse representations with a variant of autoencoder called Stochastic Winner-Take-All Auto-Encoder (SWTA AE), which performs variable rate image compression for images of any size after a single training, which is fundamental for compression.
Abstract: This paper addresses the problem of image compression using sparse representations. We propose a variant of autoencoder called Stochastic Winner-Take-All Auto-Encoder (SWTA AE). “Winner-Take-All” means that image patches compete with one another when computing their sparse representation and “Stochastic” indicates that a stochastic hyperparameter rules this competition during training. Unlike auto-encoders, SWTA AE performs variable rate image compression for images of any size after a single training, which is fundamental for compression. For comparison, we also propose a variant of Orthogonal Matching Pursuit (OMP) called Winner-Take-All Orthogonal Matching Pursuit (WTA OMP). In terms of rate-distortion trade-off, SWTA AE outperforms auto-encoders but it is worse than WTA OMP. Besides, SWTA AE can compete with JPEG in terms of rate-distortion.
Topics: Image compression (58%), Sparse approximation (56%), JPEG (54%), Matching pursuit (54%), Autoencoder (52%)

Content maybe subject to copyright    Report

HAL Id: hal-01493137
https://hal.archives-ouvertes.fr/hal-01493137
Submitted on 21 Mar 2017
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Image Compression with Stochastic Winner-Take-All
Auto-Encoder
Thierry Dumas, Aline Roumy, Christine Guillemot
To cite this version:
Thierry Dumas, Aline Roumy, Christine Guillemot. Image Compression with Stochastic Winner-Take-
All Auto-Encoder. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP 2017), Mar 2017, New Orleans, United States. �hal-01493137�

IMAGE COMPRESSION WITH STOCHASTIC WINNER-TAKE-ALL AUTO-ENCODER
Thierry Dumas, Aline Roumy, Christine Guillemot
INRIA Rennes Bretagne-Atlantique
thierry.dumas@inria.fr, aline.roumy@inria.fr, christine.guillemot@inria.fr
ABSTRACT
This paper addresses the problem of image compression us-
ing sparse representations. We propose a variant of auto-
encoder called Stochastic Winner-Take-All Auto-Encoder
(SWTA AE). “Winner-Take-All” means that image patches
compete with one another when computing their sparse rep-
resentation and “Stochastic” indicates that a stochastic hy-
perparameter rules this competition during training. Unlike
auto-encoders, SWTA AE performs variable rate image com-
pression for images of any size after a single training, which
is fundamental for compression. For comparison, we also
propose a variant of Orthogonal Matching Pursuit (OMP)
called Winner-Take-All Orthogonal Matching Pursuit (WTA
OMP). In terms of rate-distortion trade-off, SWTA AE out-
performs auto-encoders but it is worse than WTA OMP. Be-
sides, SWTA AE can compete with JPEG in terms of rate-
distortion.
Index Terms Image compression, sparse representa-
tions, auto-encoders, Orthogonal Matching Pursuit.
1. INTRODUCTION
Auto-encoders are powerful tools for reducing the dimension-
ality of data. Deep fully-connected auto-encoders [1] are tra-
ditionally used for this task. However, two issues have so far
prevented them from becoming efficient image compression
algorithms: they can only be trained for one image size and
one compression rate [2, 3].
[4] attempts to solve both issues. The authors train an
auto-encoder on image patches so that images of various sizes
can be compressed. Their auto-encoder is a recurrent [5]
residual auto-encoder that performs variable rate image com-
pression after a single training. But all image patches have the
same rate and therefore different distortions due to the texture
complexity variety in image patches. In addition, recurrence,
which is equivalent to scalability in image compression, is not
optimal in terms of rate-distortion trade-off [6, 7].
Instead, we propose to perform learning on whole images
under a global rate-distortion constraint. This is done through
This work has been supported by the French Defense Procurement
Agency (DGA).
Winner-Take-All (WTA), which can be viewed as a competi-
tion between image patches when computing their represen-
tation. Furthermore, auto-encoders architecture must adapt
to different rates. Therefore, during training, the WTA pa-
rameter that controls the rate is stochastically driven. These
contributions give rise to Stochastic Winner-Take-All Auto-
Encoder (SWTA AE).
1.1. Notation
Vectors are denoted by bold lower case letters and matrices
by upper case ones. X
j
denotes the j
th
column of a matrix
X. kXk
F
is the Frobenius norm of X. kXk
0
counts the
number of non-zero elements in X. The support of a vector x
is supp (x) = {i | x
i
6= 0}.
2. STOCHASTIC WINNER-TAKE-ALL
AUTO-ENCODER (SWTA AE)
We now present our Stochastic Winner-Take-All Auto-
Encoder (SWTA AE) whose architecture is shown in Figure 1.
SWTA AE is a type of auto-encoder. An auto-encoder is a
neural network that takes an input and provides a reconstruc-
tion of this input. We justify below two of the most critical
choices for the SWTA AE architecture.
2.1. Strided convolution
A compression algorithm must process images of various
sizes. However, the most efficient neural networks [8, 9] re-
quire that all images have the same size. Indeed, they include
both convolutional layers and fully-connected layers, and the
number of parameters of the latters directly depends on the
image size. This imposes to train one architecture per im-
age size. That is why our proposed SWTA AE only contains
convolutional layers. Its encoder has two convolutional lay-
ers and its decoder has two deconvolutional layers [10]. Each
layer i J1, 4K consists in convolving the layer input with
the bank of filters W
(i)
, adding the biases b
(i)
and applying
a mapping g
(i)
, producing the layer output. For the borders of
the layer input, zero-padding of width p
(i)
is used.
Max-pooling is a core component of neural networks [11]
that downsamples its input representation by appling a max

Fig. 1: SWTA AE architecture.
filter to non-overlapping sub-regions. But max-pooling in-
creases the rate. Indeed, if the encoder contains a max-
pooling layer, the locations of maximum activations selected
during pooling operations must be recorded and transmitted
to the corresponding unpooling layer in the decoder [12, 13].
Instead, for i J1, 2K, we downsample using a fixed stride
s
(i)
> 1 for convolution, which does not need any signaling.
2.2. Semi-sparse bottleneck
The bottleneck is the stack of feature maps denoted Z
R
h×w×65
in Figure 1. Z is the representation of the input
image that is processed in Section 2.3 to give the bitstream.
We propose to apply a global sparse constraint that po-
vides control over the coding cost of Z. This is called
Winner-Take-All (WTA). Let us define WTA via a mapping
g
α
: R
h×w×64
R
h×w×64
, where α ]0, 1[ is the WTA
parameter. g
α
keeps the α × h × w × 64 most representa-
tive coefficients in its input tensor, i.e. those whose absolute
values are the largest, and sets the rest to 0. g
α
only applies
to the output of the convolution in the second layer involving
the first 64 filters in W
(2)
, producing the first 64 sparse fea-
ture maps in Z. Figure 1 displays these sparse feature maps
in orange. Varying α leads to various coding costs of Z. Note
that [14] uses WTA, but our WTA rule is different and g
α
does not apply to specific dimensions of its input tensor as
this constraint is not relevant for image compression.
A patch of the input image might be represented by a por-
tion of the first 64 sparse feature maps in Z that only con-
tains zeros. We want to ensure that each image patch has a
minimum code in Z to guarantee a sufficient quality of recon-
struction per patch. That is why the last feature map in Z is
not sparse. Figure 1 displays it in red. We have noticed that,
during the training in Section 4.2, SWTA AE learns to store in
the last feature map a subsampled version of its input image.
2.3. Bitstream generation
The coefficients of the non-sparse feature map in Z are uni-
formly quantized over 8-bits and coded with a Huffman code.
The non-zero coefficients of the 64 sparse feature maps in Z
are uniformly quantized over 8-bits and coded with a Huff-
man code while their position is coded as explained here-
after. Figure 1 defines a coordinate system (x, y, z) for Z.
The non-zero coefficients in Z are scanned along (x, y, z)
where z changes the fastest. The position along z is coded
with a fixed-length code and, for each pair (x, y), the num-
ber of non-zero coefficients along z is coded with a Huffman
code. This unequivocally characterizes the position of each
non-zero coefficient in Z. We have observed that this pro-
cessing is effective in encoding the position of the non-zero
coefficients.
3. WINNER-TAKE-ALL ORTHOGONAL
MATCHING PURSUIT (WTA OMP)
SWTA AE is similar to Orthogonal Matching Pursuit (OMP)
[15], a common algorithm for image compression using
sparse representations [16]. The difference is that SWTA AE
computes the sparse representation of an image by alternat-
ing convolutions and mappings whereas OMP runs an iter-
ative decomposition of the image patches over a dictionary.
More precisely, let x R
m
be an image patch. Given x and
a dictionary D R
m×n
, OMP finds a vector of coefficients
y R
n
with k < m non-zero coefficients so that Dy equals
to x approximatively.
For the sake of comparison, we build a variant of OMP
called Winner-Take-All Orthogonal Matching Pursuit (WTA
OMP). More precisely, let X R
m×p
be a matrix whose
columns are formed by p image patches of dimension m and
Y R
n×p
be a matrix whose columns are formed by p vec-

tors of coefficients of dimension n. WTA OMP first decom-
poses each image patch over D, see (1). Then, it keeps the
γ × n × p coefficients with largest absolute value for the n-
length sparse representation of the p patches and sets the rest
to 0, see (2). The support of the sparse representation of each
patch has therefore been changed. Hence the need for a final
least-square minimization, see (3).
Algorithm 1 : WTA OMP
Inputs: X R
m×p
, D R
m×n
, k < m and γ ]0, 1[.
For each j J1, pK, Y
j
= OMP (X
j
, D, k) (1)
I = f
γ
(Y) (2)
For each j J1, pK, Z
j
= min
zR
n
kX
j
Dzk
2
2
st.
supp (z) = supp (I
j
)
(3)
Output: Z R
n×p
.
4. TRAINING
Before moving on to the image compression experiment in
Section 5, SWTA AE needs training. Similarly, a dictionary
D R
m×n
must be learned for WTA OMP.
4.1. Training data extraction
We extract 1.0 × 10
5
RGB images from the ILSVRC2012
ImageNet dataset [17]. The RGB color space is transformed
into YCbCr and we only keep the luminance channel.
For SWTA AE, the luminance images are resized to
321 ×321. M R
321×321
denotes the mean of all luminance
images. σ R
+
is the mean of the standard deviation over
all luminance images. Each luminance image is subtracted by
M and divided by σ. These images are concatenated into a
training set R
321×321×
(
1.0×10
5
)
.
For WTA OMP, η = 1.2 × 10
6
image patches of size
m ×
m are randomly sampled from the luminance im-
ages. We remove the DC component from each patch. These
patches are concatenated into a training set Γ R
m×η
.
4.2. SWTA AE training
As explained in Section 2.2, α tunes the coding cost of Z. If α
is fixed during training, all the filters and the biases of SWTA
AE are learned for one rate. That is why we turn α into a
stochastic hyperparameter during training. This justifies the
prefix “Stochastic” in SWTA AE. Since there is no reason to
favor some rates during training, we sample α according to
the uniform distribution U[µ , µ + ], where µ > 0 and
µ + < 1. We select µ = 1.8 × 10
1
and = 1.7 × 10
1
to make the support of α large. At each training epoch, α is
drawn for each training image of .
As shown in Section 2.1, SWTA AE can process images
of various sizes. During training, we feed SWTA AE with
random crops of size 49×49 of the training images of . This
accelerates training considerably. The training objective is to
minimize the mean squared error between these cropped im-
ages and their reconstruction plus l
2
-norm weights decay. We
use stochastic gradient descent. The gradient descent learn-
ing rate is fixed to 2.0 × 10
5
, the momentum is 0.9 and the
size of mini-batches is 5. The weights decay coefficient is
5.0 × 10
4
. Our implementation is based on Caffe [18]. It
adds to Caffe the tools introduced in Sections 2.2 and 2.3.
4.3. Dictionary learning for WTA OMP
Given Γ, k < m and γ ]0, 1[, the dictionary learning prob-
lem is formulated as (4).
min
D,Z
1
,...,Z
η
1
η
η
X
j=1
kΓ
j
DZ
j
k
2
2
st. j J1, ηK, kZ
j
k
0
k
st.
η
X
i=j
kZ
j
k
0
γ × n × η
(4)
(4) is solved by Algorithm 2 which alternates between sparse
coding steps that involve WTA OMP and dictionary updates
that use stochastic gradient descent. Given Γ and p N
+
,
let φ be a function that randomly partitions Γ into η
p
=
η / p mini-batches
X
(1)
, ..., X
(η
p
)
, where, for i J1, η
p
K,
X
(i)
R
m×p
. Mini-batches make learning very fast [19].
Algorithm 2 : dictionary learning for WTA OMP.
Inputs: Γ R
m×η
, k < m, γ ]0, 1[, p N
+
and ε R
+
.
D R
m×n
is randomly initialized.
j [|1, n|], D
j
D
j
/ kD
j
k
2
For several epochs do:
h
X
(1)
, ..., X
(η
p
)
i
= φ (Γ, p)
i [|1, η
p
|], Z
(i)
= WTA OMP
X
(i)
, D, k, γ
D D ε
X
(i)
DZ
(i)
2
F
D
j [|1, n|], D
j
D
j
/ kD
j
k
2
Output: D R
m×n
.
For OMP, given Γ, a dictionary D
0
R
m×n
is learned
using K-SVD [16]
1
, and the parameters m and n are opti-
mized with an exhaustive search. This leads to m = 64 and
n = 1024. For SWTA AE, the same values for m and n
are used for training D via Algorithm 2. Moreover, k = 15,
γ = 4.5 × 10
3
, p = 10 and ε = 2.0 × 10
2
.
1
K-SVD code: http://www.cs.technion.ac.il/ elad/software/

Fig. 2: Evolution of PNSR with the rate.
(a) LENA luminance 512 × 512. (b) BARBARA luminance 480 × 384.
5. IMAGE COMPRESSION EXPERIMENT
After training in Section 4, we compare the rate-distortion
curves of OMP, WTA OMP, SWTA AE, JPEG and JPEG2000
on test luminance images.
5.1. Image CODEC for SWTA AE
Each input test luminance image is pre-processed similarly to
the training in Section 4.1. The mean learned image M is
interpolated to match the size of the input image. Then, the
input image is subtracted by this interpolated mean image and
divided by the learned σ. The encoder of SWTA AE computes
Z. The bitstream is obtained by processing Z as detailed in
Section 2.3.
5.2. Image CODEC for OMP and WTA OMP
A luminance image is split into 8×8 non-overlapping patches.
The DC component is removed from each patch. The DC
components are uniformly quantized over 8-bits and coded
with a fixed-length code. OMP (or WTA OMP) finds the co-
efficients of the sparse decompositions of the image patches
over D
0
(or D). The non-zero coefficients are uniformly
quantized over 8-bits and coded with a Huffman code while
their position is coded with a fixed-length code.
Then, for WTA OMP only, the number of non-zero coef-
ficients of the sparse decomposition of each patch over D is
coded with a Huffman code.
5.3. Comparison of rate-distortion curves
In the literature, there is no reference rate-distortion curve
for auto-encoders. We compare SWTA AE with JPEG and
JPEG2000
2
even though the image CODEC of SWTA AE is
less optimized. Furthermore, we compare SWTA AE with its
2
JPEG and JPEG2000 code: http://www.imagemagick.org/script/index.php
non-sparse Auto-Encoder counterpart (AE). AE has the same
architecture as SWTA AE but its Z only contains non-sparse
feature maps. Note that, to draw a new point in the AE rate-
distortion curve, AE must be first re-trained with a different
number of feature maps in Z.
Figure 2 shows the rate-distortion curves of OMP, WTA
OMP, AE, SWTA AE, JPEG and JPEG2000 for two of the
most common images: LENA and BARBARA. In terms
of rate-distortion trade-off, SWTA AE outperforms AE and
WTA OMP is better than OMP. This highlights the value
of WTA for image compression. When we compare SWTA
AE with WTA OMP, we see that iterative decomposition
is more efficient for image compression using sparse repre-
sentations. Moreover, SWTA AE can compete with JPEG.
We also ran this image compression experiment on several
crops of LENA and BARBARA and observed that the rel-
ative position of the six rate-distortion curves was compa-
rable to the relative positioning in figure 2. The size of
the test image does not affect the performance of SWTA
AE. More simulation results and a complexity analysis for
OMP, WTA OMP and SWTA AE can be found on the
web page https://www.irisa.fr/temics/demos/
NeuralNets/AutoEncoders/swtaAE.htm.
6. CONCLUSIONS AND FUTURE WORK
We have shown that, SWTA AE is more adaptated to image
compression than auto-encoders as it performs variable rate
image compression for any size of image after a single train-
ing and provides better rate-distortion trade-offs.
So far, our work has focused on the layer of auto-encoders
which is dedicated to coding. Yet, many avenues of research
are still to be explored to improve auto-encoders for image
compression. For instance, [20] proves that removing a max-
pooling layer and increasing the stride of the previous con-
volution, as we do, harms neural networks. This has to be
addressed.

Citations
More filters

Journal ArticleDOI
01 Nov 2018-Information Fusion
Abstract: Many of the existing machine learning algorithms, both supervised and unsupervised, depend on the quality of the input characteristics to generate a good model. The amount of these variables is also important, since performance tends to decline as the input dimensionality increases, hence the interest in using feature fusion techniques, able to produce feature sets that are more compact and higher level. A plethora of procedures to fuse original variables for producing new ones has been developed in the past decades. The most basic ones use linear combinations of the original variables, such as PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis), while others find manifold embeddings of lower dimensionality based on non-linear combinations, such as Isomap or LLE (Linear Locally Embedding) techniques. More recently, autoencoders (AEs) have emerged as an alternative to manifold learning for conducting nonlinear feature fusion. Dozens of AE models have been proposed lately, each with its own specific traits. Although many of them can be used to generate reduced feature sets through the fusion of the original ones, there also AEs designed with other applications in mind. The goal of this paper is to provide the reader with a broad view of what an AE is, how they are used for feature fusion, a taxonomy gathering a broad range of models, and how they relate to other classical techniques. In addition, a set of didactic guidelines on how to choose the proper AE for a given task is supplied, together with a discussion of the software tools available. Finally, two case studies illustrate the usage of AEs with datasets of handwritten digits and breast cancer.

146 citations


Journal ArticleDOI
Jiahao Li1, Bin Li2, Xu Jizheng2, Ruiqin Xiong1  +1 moreInstitutions (2)
TL;DR: This paper proposes using a fully connected network to learn an end-to-end mapping from neighboring reconstructed pixels to the current block to generate better prediction using traditional single line-based methods.
Abstract: This paper proposes a deep learning method for intra prediction. Different from traditional methods utilizing some fixed rules, we propose using a fully connected network to learn an end-to-end mapping from neighboring reconstructed pixels to the current block. In the proposed method, the network is fed by multiple reference lines. Compared with traditional single line-based methods, more contextual information of the current block is utilized. For this reason, the proposed network has the potential to generate better prediction. In addition, the proposed network has good generalization ability on different bitrate settings. The model trained from a specified bitrate setting also works well on other bitrate settings. Experimental results demonstrate the effectiveness of the proposed method. When compared with high efficiency video coding reference software HM-16.9, our network can achieve an average of 3.4% bitrate saving. In particular, the average result of 4K sequences is 4.5% bitrate saving, where the maximum one is 7.4%.

99 citations


Cites methods from "Image compression with Stochastic W..."

  • ...In [23], a variant of autoencoder is proposed by using sparse...

    [...]


Journal ArticleDOI
Zhibo Chen1, Tianyu He1, Xin Jin1, Feng Wu1Institutions (1)
TL;DR: The proposed PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks can model spatiotemporal coherence to effectively perform predictive coding inside the learning network and provides a possible new direction to further improve compression efficiency and functionalities of future video coding.
Abstract: One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper, we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis and binarization. The experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.

73 citations


Additional excerpts

  • ...on and optimized a convolutional auto-encoder with an incremental training strategy [6]. Dumas et al. incorporates a stochastic hyperparameter to control a competition mechanism between image patches [20]. As a fast-growing architecture in the field of neural network, GANs have also been proved to be effective on image compression in practice [21], [22]. In the works of [4], [7], [23], [24], a discrete...

    [...]


Posted Content
TL;DR: This paper shows that comparable performances can be obtained with a unique learned transform in the case of autoencoders, and saves a lot of training time.
Abstract: This paper explores the problem of learning transforms for image compression via autoencoders. Usually, the rate-distortion performances of image compression are tuned by varying the quantization step size. In the case of autoen-coders, this in principle would require learning one transform per rate-distortion point at a given quantization step size. Here, we show that comparable performances can be obtained with a unique learned transform. The different rate-distortion points are then reached by varying the quantization step size at test time. This approach saves a lot of training time.

27 citations


Cites methods from "Image compression with Stochastic W..."

  • ...Deep autoencoders have been shown as promising tools for finding alternative transforms [2, 3, 4]....

    [...]


Journal ArticleDOI
Zhibo Chen1, Tianyu He1Institutions (1)
21 Apr 2019-Neurocomputing
TL;DR: A Learning based Facial Image Compression framework with a novel Regionally Adaptive Pooling module whose parameters can be automatically optimized according to gradient feedback from an integrated hybrid semantic fidelity metric, including a successfully exploration to apply Generative Adversarial Network (GAN) as metric directly in image compression scheme.
Abstract: Surveillance and security scenarios usually require high efficient facial image compression scheme for face recognition and identification. While either traditional general image codecs or special facial image compression schemes only heuristically refine codec separately according to face verification accuracy metric. We propose a Learning based Facial Image Compression (LFIC) framework with a novel Regionally Adaptive Pooling (RAP) module whose parameters can be automatically optimized according to gradient feedback from an integrated hybrid semantic fidelity metric, including a successfully exploration to apply Generative Adversarial Network (GAN) as metric directly in image compression scheme. The experimental results verify the framework’s efficiency by demonstrating performance improvement of 71.41%, 48.28% and 52.67% Bit Rate Saving separately over JPEG2000, WebP and neural network-based codecs under the same face verification accuracy distortion metric. We also evaluate LFIC’s superior performance gain compared with latest specific facial image codecs. Visual experiments also show some interesting insight on how LFIC can automatically capture the information in critical areas based on semantic distortion metrics for optimized compression, which is quite different from the heuristic way of optimization in traditional image compression algorithms.

19 citations


Cites background from "Image compression with Stochastic W..."

  • ...[17] introduced a competition mechanism between image patches binding to sparse representation....

    [...]


References
More filters

Proceedings Article
03 Dec 2012-
Abstract: We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overriding in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

73,871 citations


Proceedings ArticleDOI
Jia Deng1, Wei Dong1, Richard Socher1, Li-Jia Li1  +2 moreInstitutions (1)
20 Jun 2009-
TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Abstract: The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called “ImageNet”, a large-scale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond.

31,274 citations


"Image compression with Stochastic W..." refers methods in this paper

  • ...0 × 10(5) RGB images from the ILSVRC2012 ImageNet dataset [17]....

    [...]


Journal ArticleDOI
28 Jul 2006-Science
Abstract: High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

14,206 citations


Posted Content
Yangqing Jia1, Evan Shelhamer2, Jeff Donahue2, Sergey Karayev2  +4 moreInstitutions (2)
Abstract: Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models. The framework is a BSD-licensed C++ library with Python and MATLAB bindings for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures. Caffe fits industry and internet-scale media needs by CUDA GPU computation, processing over 40 million images a day on a single K40 or Titan GPU ($\approx$ 2.5 ms per image). By separating model representation from actual implementation, Caffe allows experimentation and seamless switching among platforms for ease of development and deployment from prototyping machines to cloud environments. Caffe is maintained and developed by the Berkeley Vision and Learning Center (BVLC) with the help of an active community of contributors on GitHub. It powers ongoing research projects, large-scale industrial applications, and startup prototypes in vision, speech, and multimedia.

12,530 citations


Book ChapterDOI
Matthew D. Zeiler1, Rob Fergus1Institutions (1)
06 Sep 2014-
TL;DR: A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark.
Abstract: Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark Krizhevsky et al. [18]. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we explore both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. Used in a diagnostic role, these visualizations allow us to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. We also perform an ablation study to discover the performance contribution from different model layers. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.

11,585 citations


"Image compression with Stochastic W..." refers background in this paper

  • ...Indeed, if the encoder contains a maxpooling layer, the locations of maximum activations selected during pooling operations must be recorded and transmitted to the corresponding unpooling layer in the decoder [12, 13]....

    [...]


Network Information
Related Papers (5)
Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
20221
20211
20205
20192
20188
20171