Showing papers by "Huawei published in 2022"

Elastic Deep Learning in Multi-Tenant GPU Clusters

Abstract: We study how to support elasticity, that is, the ability to dynamically adjust the parallelism (i.e., the number of GPUs), for deep neural network (DNN) training in a GPU cluster. Elasticity can benefit multi-tenant GPU cluster management in many ways, for example, achieving various scheduling objectives (e.g., job throughput, job completion time, GPU efficiency) according to cluster load variations, utilizing transient idle resources, and supporting performance profiling, job migration, and straggler mitigation. We propose EDL, which enables elastic deep learning with a simple API and can be easily integrated with existing deep learning frameworks such as TensorFlow and PyTorch. EDL also incorporates techniques that are necessary to reduce the overhead of parallelism adjustments, such as stop-free scaling and dynamic data pipeline. We demonstrate with experiments that EDL can indeed bring significant benefits to the above-listed applications in GPU cluster management.

v Pipe : A Virtualized Acceleration System for Achieving Efficient and Scalable Pipeline Parallel DNN Training

Abstract: The increasing computational complexity of DNNs achieved unprecedented successes in various areas such as machine vision and natural language processing (NLP), e.g., the recent advanced Transformer has billions of parameters. However, as large-scale DNNs significantly exceed GPU’s physical memory limit, they cannot be trained by conventional methods such as data parallelism. Pipeline parallelism that partitions a large DNN into small subnets and trains them on different GPUs is a plausible solution. Unfortunately, the layer partitioning and memory management in existing pipeline parallel systems are fixed during training, making them easily impeded by out-of-memory errors and the GPU under-utilization. These drawbacks amplify when performing neural architecture search (NAS) such as the evolved Transformer, where different network architectures of Transformer needed to be trained repeatedly. vPipe is the first system that transparently provides dynamic layer partitioning and memory management for pipeline parallelism. vPipe has two unique contributions, including (1) an online algorithm for searching a near-optimal layer partitioning and memory management plan, and (2) a live layer migration protocol for re-balancing the layer distribution across a training pipeline. vPipe improved the training throughput of two notable baselines (Pipedream and GPipe) by 61.4-463.4 percent and 24.8-291.3 percent on various large DNNs and training settings.

How does working from home affect developer productivity? — A case study of Baidu during the COVID-19 pandemic

Abstract: Nowadays, working from home (WFH) has become a popular work arrangement due to its many potential benefits for both companies and employees (e.g., increasing job satisfaction and retention of employees). Many previous studies have investigated the impact of WFH on the productivity of employees. However, most of these studies usually use a qualitative analysis method such as surveys and interviews, and the studied participants do not work from home for a long continuing time. Due to the outbreak of coronavirus disease 2019 (COVID-19), a large number of companies asked their employees to work from home, which provides us an opportunity to investigate whether WFH affects their productivity. In this study, to investigate the difference in developer productivity between WFH and working onsite, we conduct a quantitative analysis based on a dataset of developers’ daily activities from Baidu Inc., one of the largest IT companies in China. In total, we collected approximately four thousand records of 139 developers’ activities of 138 working days. Out of these records, 1103 records are submitted when developers work from home due to the COVID-19 pandemic. We find that WFH has both positive and negative impacts on developer productivity in terms of different metrics, e.g., the number of builds/commits/code reviews. We also notice that WFH has different impacts on projects with different characteristics including programming language, project type/age/size. For example, WFH has a negative impact on developer productivity for large projects. Additionally, we find that productivity varies for different developers. Based on these findings, we get some feedback from developers of Baidu and understand some reasons why WFH has different impacts on developer productivity. We also conclude several implications for both companies and developers.

Toward practical quantum embedding simulation of realistic chemical systems on near-term quantum computers

Abstract: Quantum computing has recently exhibited great potential in predicting chemical properties for various applications in drug discovery, material design, and catalyst optimization. Progress has been made in simulating small molecules, such as LiH and hydrogen chains of up to 12 qubits, by using quantum algorithms such as variational quantum eigensolver (VQE). Yet, originating from the limitations of the size and the fidelity of near-term quantum hardware, the accurate simulation of large realistic molecules remains a challenge. Here, integrating an adaptive energy sorting strategy and a classical computational method-the density matrix embedding theory, which respectively reduces the circuit depth and the problem size, we present a means to circumvent the limitations and demonstrate the potential of near-term quantum computers toward solving real chemical problems. We numerically test the method for the hydrogenation reaction of C6H8 and the equilibrium geometry of the C18 molecule, using basis sets up to cc-pVDZ (at most 144 qubits). The simulation results show accuracies comparable to those of advanced quantum chemistry methods such as coupled-cluster or even full configuration interaction, while the number of qubits required is reduced by an order of magnitude (from 144 qubits to 16 qubits for the C18 molecule) compared to conventional VQE. Our work implies the possibility of solving industrial chemical problems on near-term quantum devices.

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

Abstract: This paper studies the Binary Neural Networks (BNNs) in which weights and activations are both binarized into 1-bit values, thus greatly reducing the memory usage and computational complexity. Since the modern deep neural networks are of sophisticated design with complex architecture for the accuracy reason, the diversity on distributions of weights and activations is very high. Therefore, the conventional sign function cannot be well used for effectively binarizing full-precision values in BNNs. To this end, we present a simple yet effective approach called AdaBin to adaptively obtain the optimal binary sets $$\{b_1, b_2\}$$ ( $$b_1, b_2\in \mathbb {R}$$ ) of weights and activations for each layer instead of a fixed set (i.e., $$\{-1, +1\}$$ ). In this way, the proposed method can better fit different distributions and increase the representation ability of binarized features. In practice, we use the center position and distance of 1-bit values to define a new binary quantization function. For the weights, we propose an equalization method to align the symmetrical center of binary distribution to real-valued distribution, and minimize the Kullback-Leibler divergence of them. Meanwhile, we introduce a gradient-based optimization method to get these two parameters for activations, which are jointly trained in an end-to-end manner. Experimental results on benchmark models and datasets demonstrate that the proposed AdaBin is able to achieve state-of-the-art performance. For instance, we obtain a 66.4% Top-1 accuracy on the ImageNet using ResNet-18 architecture, and a 69.4 mAP on PASCAL VOC using SSD300.

CodeMatcher: Searching Code Based on Sequential Semantics of Important Query Words

Abstract: To accelerate software development, developers frequently search and reuse existing code snippets from a large-scale codebase, e.g., GitHub. Over the years, researchers proposed many information re...

Understanding in-app advertising issues based on large scale app review analysis

Abstract: Context: In-app advertising closely relates to app revenue. Reckless ad integration could adversely impact app quality and user experience, leading to loss of income. It is very challenging to balance the ad revenue and user experience for app developers. Objective: Towards tackling the challenge, we conduct a study on analyzing user concerns about in-app advertisement. Method: Specifically, we present a large-scale analysis on ad-related user feedback. The large user feedback data from App Store and Google Play allow us to summarize ad-related app issues comprehensively and thus provide practical ad integration strategies for developers. We first define common ad issues by manually labeling a statistically representative sample of ad-related feedback, and then build an automatic classifier to categorize ad-related feedback. We study the relations between different ad issues and user ratings to identify the ad issues poorly scored by users. We also explore the fix durations of ad issues across platforms for extracting insights into prioritizing ad issues for ad maintenance. Results: (1) We summarize 15 types of ad issues by manually annotating 903 out of 36,309 ad-related user reviews. From a statistical analysis of 36,309 ad-related reviews, we find that users care most about the number of unique ads and ad display frequency during usage. (2) Users tend to give relatively lower ratings when they report the security and notification related issues. (3) Regarding different platforms, we observe that the distributions of ad issues are significantly different between App Store and Google Play. (4) Some ad issue types are addressed more quickly by developers than other ad issues. Conclusion: We believe the findings we discovered can benefit app developers towards balancing ad revenue and user experience while ensuring app quality.

CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving

Abstract: Contemporary deep-learning object detection methods for autonomous driving usually presume fixed categories of common traffic participants, such as pedestrians and cars. Most existing detectors are unable to detect uncommon objects and corner cases (e.g., a dog crossing a street), which may lead to severe accidents in some situations, making the timeline for the real-world application of reliable autonomous driving uncertain. One main reason that impedes the development of truly reliably self-driving systems is the lack of public datasets for evaluating the performance of object detectors on corner cases. Hence, we introduce a challenging dataset named CODA that exposes this critical problem of vision-based detectors. The dataset consists of 1500 carefully selected real-world driving scenes, each containing four object-level corner cases (on average), spanning more than 30 object categories. On CODA, the performance of standard object detectors trained on large-scale autonomous driving datasets significantly drops to no more than 12.8% in mAR. Moreover, we experiment with the state-of-the-art open-world object detector and find that it also fails to reliably identify the novel objects in CODA, suggesting that a robust perception system for autonomous driving is probably still far from reach. We expect our CODA dataset to facilitate further research in reliable detection for real-world autonomous driving. Our dataset is available at https://coda-dataset.github.io .

Towards a Consistent Interpretation of AIOps Models

Abstract: Artificial Intelligence for IT Operations (AIOps) has been adopted in organizations in various tasks, including interpreting models to identify indicators of service failures. To avoid misleading p...

Metabolic syndrome in patients with first-ever ischemic stroke: prevalence and association with coronary heart disease

Abstract: The metabolic syndrome (MetS) has been well linked with coronary heart disease (CHD) in the general population, but studies have rarely explored their association among patients with stroke. We examine prevalence of MetS and its association with CHD in patients with first-ever ischemic stroke. This hospital-based study included 1851 patients with first-ever ischemic stroke (mean age 61.2 years, 36.5% women) who were hospitalized into two university hospitals in Shandong, China (January 2016-February 2017). Data were collected through interviews, physical examinations, and laboratory tests. MetS was defined following the National Cholesterol Education Program (NCEP) criteria, the International Diabetes Federation (IDF) criteria, and the Chinese Diabetes Society (CDS) criteria. CHD was defined following clinical criteria. Data were analyzed using binary logistic regression models. The overall prevalence of MetS was 33.4% by NECP criteria, 47.2% by IDF criteria, and 32.5% by CDS criteria, with the prevalence being decreased with age and higher in women than in men (p < 0.05). High blood pressure, high triglycerides, and low HDL-C were significantly associated with CHD (multi-adjusted odds ratio [OR] range 1.27-1.38, p < 0.05). The multi-adjusted OR of CHD associated with MetS defined by the NECP criteria, IDF criteria, and CDS criteria (vs. no MetS) was 1.27 (95% confidence interval 1.03-1.57), 1.44 (1.18-1.76), and 1.27 (1.03-1.57), respectively. In addition, having 1-2 abnormal components (vs. none) of MetS was associated with CHD (multi-adjusted OR range 1.66-1.72, p < 0.05). MetS affects over one-third of patients with first-ever ischemic stroke. MetS is associated with an increased likelihood of CHD in stroke patients.

<i>Mangifera indica</i> Leaf (MIL) as a Novel Material in Atmospheric Water Collection

Abstract: Here, Mangifera indica leaves (MILs) have been used to collect atmospheric water for the first time. This novel material has been viewed by mankind as environmental waste and is mostly discarded or incinerated, causing environmental pollution. By turning waste into wealth, MILs have proven resourceful and can help ameliorate the water crisis, especially in tropical countries. The unprecedented water collection result is enough to describe MILs as an ideal material for atmospheric water collection when compared to other natural plants. Both the physical and chemical surface morphologies were extensively characterized. This comparative study shows that MIL surface droplet termination and hydrophilic nature differ from those of other materials, with the apex playing a key role in the roll-off of the droplet. The surface wettability and its interaction with the droplet are of keen interest in this study.

Cordyceps Improves Obesity and its Related Inflammation via Modulation of <i>Enterococcus cecorum</i> Abundance and Bile Acid Metabolism

Abstract: Dysbiotic gut microbiota has been identified as a primary mediator of inherent inflammation that underlies the pathogenesis of obesity. Cordyceps comprises the larval body and the stroma of Cordyceps sinensis (BerK.) Sacc. parasiting on Hepialidae larvae of moths (H. pialusoberthur) with potent metabolic regulation functions. The underlying anti-obesity mechanisms, however, remain largely unknown. Here, we demonstrate that the water extract of Cordyceps attenuates glucose and lipid metabolism disorders and its associated inflammation in high-fat diet (HFD)-fed mice. 16S rRNA gene sequencing and microbiomic analysis showed that Cordyceps reduced the amounts of Enterococcus cecorum, a bile-salt hydrolase-producing microbe to regulate the metabolism of bile acids in the gut. Importantly, E. cecorum transplantation or liver-specific knockdown of farnesoid X receptor (FXR), a bile acid receptor, diminished the protective effect of Cordyceps against HFD-induced obesity. Together, our results shed light on the mechanisms that underlie the glucose- and lipid-lowering effects of Cordyceps and suggest that targeting intestinalE. cecorum or hepatic FXR are potential anti-obesity and anti-inflammation therapeutic avenues.

On the Reproducibility and Replicability of Deep Learning in Software Engineering

Abstract: Context: Deep learning (DL) techniques have gained significant popularity among software engineering (SE) researchers in recent years. This is because they can often solve many SE challenges withou...

On Decoding of Reed-Muller Codes Using a Local Graph Search

Abstract: We present a novel iterative decoding algorithm for Reed-Muller (RM) codes, which takes advantage of a graph representation of the code. Vertices of the considered graph correspond to codewords, with two vertices being connected by an edge if and only if the Hamming distance between the corresponding codewords equals the minimum distance of the code. The algorithm uses a greedy local search to find a node optimizing a metric, e.g. the correlation between the received vector and the corresponding codeword. In addition, the cyclic redundancy check can be used to terminate the search as soon as a valid codeword is found, leading to an improvement in the average computational complexity of the algorithm. Simulation results for both binary symmetric channel and additive white Gaussian noise channel show that the presented decoder approaches the performance of maximum likelihood decoding for RM codes of length less than 1024 and for the second-order RM codes of length less than 4096. Moreover, it is demonstrated that the considered decoding approach outperforms state-of-the-art decoding algorithms of RM codes with similar computational complexity for a wide range of block lengths and rates.

Carnosol inhibits the growth and biofilm of Candida albicans

Abstract: This study was to explore the inhibitory effects of carnosol on the growth and biofilm of Candida albicans.Our results showed that carnosol inhibited the planktonic growth of C. albicans with a MIC of 100 μg/mL. Carnosol can also inhibit the biofilm formation and development of C. albicans. 25-100 μg/mL of carnosol can obviously inhibit the yeast-to-hyphal transition in four kinds of hyphal-inducing media and the adhesion of C. albicans to polystyrene surfaces. Results from PI staining indicated that carnosol may disrupt cell membrane of C. albicans.Carnosol can inhibit the planktonic growth and virulence factors of C. albicans, such as biofilm formation, adhesion and hyphal growth. The antifungal mechanism may involve the increase in cell membrane permeability.