Huawei

About: Huawei is a company organization based out in Shenzhen, China. It is known for research contribution in the topics: Terminal (electronics) & Signal. The organization has 41417 authors who have published 44698 publications receiving 343496 citations. The organization is also known as: Huawei Technologies & Huawei Technologies Co., Ltd..

Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation

Abstract: Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time (~7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at this https URL.

Revisiting Knowledge Distillation via Label Smoothing Regularization

Abstract: Knowledge Distillation (KD) aims to distill the knowledge of a cumbersome teacher model into a lightweight student model. Its success is generally attributed to the privileged information on similarities among categories provided by the teacher model, and in this sense, only strong teacher models are deployed to teach weaker students in practice. In this work, we challenge this common belief by following experimental observations: 1) beyond the acknowledgment that the teacher can improve the student, the student can also enhance the teacher significantly by reversing the KD procedure; 2) a poorly-trained teacher with much lower accuracy than the student can still improve the latter significantly. To explain these observations, we provide a theoretical analysis of the relationships between KD and label smoothing regularization. We prove that 1) KD is a type of learned label smoothing regularization and 2) label smoothing regularization provides a virtual teacher model for KD. From these results, we argue that the success of KD is not fully due to the similarity information between categories from teachers, but also to the regularization of soft targets, which is equally or even more important. Based on these analyses, we further propose a novel Teacher-free Knowledge Distillation (Tf-KD) framework, where a student model learns from itself or manually-designed regularization distribution. The Tf-KD achieves comparable performance with normal KD from a superior teacher, which is well applied when a stronger teacher model is unavailable. Meanwhile, Tf-KD is generic and can be directly deployed for training deep neural networks. Without any extra computation cost, Tf-KD achieves up to 0.65\% improvement on ImageNet over well-established baseline models, which is superior to label smoothing regularization.

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

Abstract: Instance segmentation is one of the fundamental vision tasks. Recently, fully convolutional instance segmentation methods have drawn much attention as they are often simpler and more efficient than two-stage approaches like Mask R-CNN. To date, almost all such approaches fall behind the two-stage Mask R-CNN method in mask precision when models have similar computation complexity, leaving great room for improvement. In this work, we achieve improved mask prediction by effectively combining instance-level information with semantic information with lower-level fine-granularity. Our main contribution is a blender module which draws inspiration from both top-down and bottom-up instance segmentation approaches. The proposed BlendMask can effectively predict dense per-pixel position-sensitive instance features with very few channels, and learn attention maps for each instance with merely one convolution layer, thus being fast in inference. BlendMask can be easily incorporate with the state-of-the-art one-stage detection frameworks and outperforms Mask R-CNN under the same training schedule while being faster. A light-weight version of BlendMask achieves 36.0 mAP at 27 FPS evaluated on a single 1080Ti. Because of its simplicity and efficacy, we hope that our BlendMask could serve as a simple yet strong baseline for a wide range of instance-wise prediction tasks.

Heterogeneous Cloud Radio Access Networks: A New Perspective for Enhancing Spectral and Energy Efficiencies

Abstract: To mitigate the severe inter-tier interference and enhance limited cooperative gains resulting from the constrained and non-ideal transmissions between adjacent base stations in heterogeneous networks (HetNets), heterogeneous cloud radio access networks (H-CRANs) are proposed as cost-efficient potential solutions through incorporating the cloud computing into HetNets. In this article, state-of-the-art research achievements and challenges on H-CRANs are surveyed. In particular, we discuss issues of system architectures, spectral and energy efficiency performances, and promising key techniques. A great emphasis is given towards promising key techniques in H-CRANs to improve both spectral and energy efficiencies, including cloud computing based coordinated multi-point transmission and reception, large-scale cooperative multiple antenna, cloud computing based cooperative radio resource management, and cloud computing based self-organizing network in the cloud converging scenarios. The major challenges and open issues in terms of theoretical performance with stochastic geometry, fronthaul constrained resource allocation, and standard development that may block the promotion of H-CRANs are discussed as well.

Wireless caching: technical misconceptions and business barriers

Abstract: Caching is a hot research topic and poised to develop into a key technology for the upcoming 5G wireless networks. However, the successful implementation of caching techniques crucially depends on joint research developments in different scientific domains such as networking, information theory, machine learning, and wireless communications. Moreover, there are business barriers related to the complex interactions between the involved stakeholders: users, cellular operators, and Internet content providers. In this article we discuss several technical misconceptions with the aim of uncovering enabling research directions for caching in wireless systems. Ultimately, we make a speculative stakeholder analysis for wireless caching in 5G.