The central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informative features by fusing both spatial and channel-wise information within local receptive fields at each layer. A broad range of prior research has investigated the spatial component of this relationship, seeking to strengthen the representational power of a CNN by enhancing the quality of spatial encodings throughout its feature hierarchy. In this work, we focus instead on the channel relationship and propose a novel architectural unit, which we term the “Squeeze-and-Excitation” (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We show that these blocks can be stacked together to form SENet architectures that generalise extremely effectively across different datasets. We further demonstrate that SE blocks bring significant improvements in performance for existing state-of-the-art CNNs at slight additional computational cost. Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017 classification submission which won first place and reduced the top-5 error to 2.251 percent, surpassing the winning entry of 2016 by a relative improvement of ${\sim }$ ∼ 25 percent. Models and code are available at https://github.com/hujie-frank/SENet .

/pdf/squeeze-and-excitation-networks-50icgwqgeu.pdf

Squeeze-and-Excitation Networks

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

We propose Convolutional Block Attention Module (CBAM), a simple yet effective attention module for feed-forward convolutional neural networks. Given an intermediate feature map, our module sequentially infers attention maps along two separate dimensions, channel and spatial, then the attention maps are multiplied to the input feature map for adaptive feature refinement. Because CBAM is a lightweight and general module, it can be integrated into any CNN architectures seamlessly with negligible overheads and is end-to-end trainable along with base CNNs. We validate our CBAM through extensive experiments on ImageNet-1K, MS~COCO detection, and VOC~2007 detection datasets. Our experiments show consistent improvements in classification and detection performances with various models, demonstrating the wide applicability of CBAM. The code and models will be publicly available.

CBAM: Convolutional Block Attention Module

There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source code is at this https URL

YOLOv4: Optimal Speed and Accuracy of Object Detection

The central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informative features by fusing both spatial and channel-wise information within local receptive fields at each layer. A broad range of prior research has investigated the spatial component of this relationship, seeking to strengthen the representational power of a CNN by enhancing the quality of spatial encodings throughout its feature hierarchy. In this work, we focus instead on the channel relationship and propose a novel architectural unit, which we term the "Squeeze-and-Excitation" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We show that these blocks can be stacked together to form SENet architectures that generalise extremely effectively across different datasets. We further demonstrate that SE blocks bring significant improvements in performance for existing state-of-the-art CNNs at slight additional computational cost. Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017 classification submission which won first place and reduced the top-5 error to 2.251%, surpassing the winning entry of 2016 by a relative improvement of ~25%. Models and code are available at this https URL.

This paper formalizes the binarization operations over neural networks from a learning perspective. In contrast to classical hand crafted rules (\eg hard thresholding) to binarize full-precision neurons, we propose to learn a mapping from full-precision neurons to the target binary ones. Each individual weight entry will not be binarized independently. Instead, they are taken as a whole to accomplish the binarization, just as they work together in generating convolution features. To help the training of the binarization mapping, the full-precision neurons after taking sign operations is regarded as some auxiliary supervision signal, which is noisy but still has valuable guidance. An unbiased estimator is therefore introduced to mitigate the influence of the supervision noise. Experimental results on benchmark datasets indicate that the proposed binarization technique attains consistent improvements over baselines.

/pdf/training-binary-neural-networks-through-learning-with-noisy-4ymp7itmx7.pdf

Training Binary Neural Networks through Learning with Noisy Supervision

We present a particle-based method for viscoelastic fluids simulation. In the method, based on the traditional Navier-Stokes equation, an additional elastic stress term is introduced to achieve viscoelastic flow behaviors, which have both fluid and solid features. Benefiting from the Lagrangian nature of Smoothed Particle Hydrodynamics, large flow deformation can be handled more easily and naturally. And also, by changing the viscosity and elastic stress coefficient of the particles according to the temperature variation, the melting and flowing phenomena, such as lava flow and wax melting, are achieved. The temperature evolution is determined with the heat diffusion equation. The method is effective and efficient, and has good controllability. Different kinds of viscoelastic fluid behaviors can be obtained easily by adjusting the very few experimental parameters.

A particle-based method for viscoelastic fluids animation

This paper presents a novel approach to implementing dynamic LOD on GPU. For our purpose, a quadtree structure is created based on seamless geometry image atlas, which is a 3D surface representation in parameter space by combining the features of geometry images and poly-cube maps. All the nodes in the quadtree are packed into the atlas textures. There are two rendering passes in our approach. In the first pass, the LOD selection is performed in the fragment shaders. The resultant buffer is taken as the input texture to the second rendering pass by vertex texturing, and thus the node culling and triangulation can be performed in the vertex shaders. Our LOD algorithm can generate adaptive meshes dynamically, and can be fully implemented on GPU. It improves the efficiency of LOD selection, and alleviates the computing load on CPU.

Dynamic LOD on GPU

We present a new model, called the dual-microfacet, for those materials such as paper and plastic formed by a thin, transparent slab lying between two surfaces of spatially varying roughness. Light transmission through the slab is represented by a microfacet-based BTDF which tabulates the microfacet's normal distribution (NDF) as a function of surface location. Though the material is bounded by two surfaces of different roughness, we approximate light transmission through it by a virtual slab determined by a single spatially-varying NDF. This enables efficient capturing of spatially variant transparent slices. We describe a device for measuring this model over a flat sample by shining light from a CRT behind it and capturing a sequence of images from a single view. Our method captures both angular and spatial variation in the BTDF and provides a good match to measured materials.

Enhua Wu

Papers

Training Binary Neural Networks through Learning with Noisy Supervision

A particle-based method for viscoelastic fluids animation

Dynamic LOD on GPU

The Dual‐microfacet Model for Capturing Thin Transparent Slabs

Technical Section: Point-in-polygon tests by convex decomposition