Open AccessProceedings Article
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
Jie Hu,Li Shen,Samuel Albanie,Gang Sun,Andrea Vedaldi +4 more
- Vol. 31, pp 9401-9411
Reads0
Chats0
TLDR
Gathering and Excite as mentioned in this paper proposes a pair of operators: gather and excite, which redistributes the pooled information to local features, which can be integrated directly in existing architectures to improve their performance.Abstract:
While the use of bottom-up local operators in convolutional neural networks (CNNs) matches well some of the statistics of natural images, it may also prevent such models from capturing contextual long-range feature interactions. In this work, we propose a simple, lightweight approach for better context exploitation in CNNs. We do so by introducing a pair of operators: gather, which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features. The operators are cheap, both in terms of number of added parameters and computational complexity, and can be integrated directly in existing architectures to improve their performance. Experiments on several datasets show that gather-excite can bring benefits comparable to increasing the depth of a CNN at a fraction of the cost. For example, we find ResNet-50 with gather-excite operators is able to outperform its 101-layer counterpart on ImageNet with no additional learnable parameters. We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.read more
Citations
More filters
Posted Content
Squeeze-and-Excitation Networks
TL;DR: Squeeze-and-excitation (SE) as mentioned in this paper adaptively recalibrates channel-wise feature responses by explicitly modeling interdependencies between channels, which can be stacked together to form SENet architectures.
Proceedings ArticleDOI
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
TL;DR: CutMix as discussed by the authors augments the training data by cutting and pasting patches among training images, where the ground truth labels are also mixed proportionally to the area of the patches.
Proceedings ArticleDOI
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
TL;DR: The Efficient Channel Attention (ECA) module as discussed by the authors proposes a local cross-channel interaction strategy without dimensionality reduction, which can be efficiently implemented via 1D convolution, which only involves a handful of parameters while bringing clear performance gain.
Proceedings ArticleDOI
Coordinate Attention for Efficient Mobile Network Design
TL;DR: CoordAttention as mentioned in this paper embeds positional information into channel attention to capture long-range dependencies along one spatial direction and meanwhile precise positional information can be preserved along the other spatial direction.
Posted Content
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
TL;DR: This paper proposes an Efficient Channel Attention (ECA) module, which only involves a handful of parameters while bringing clear performance gain, and develops a method to adaptively select kernel size of 1D convolution, determining coverage of local cross-channel interaction.