Background. Recent years have seen an increasing interest in cross-project defect prediction (CPDP), which aims to apply defect prediction models built on source projects to a target project. Currently, a variety of (complex) CPDP models have been proposed with a promising prediction performance.Problem. Most, if not all, of the existing CPDP models are not compared against those simple module size models that are easy to implement and have shown a good performance in defect prediction in the literature.Objective. We aim to investigate how far we have really progressed in the journey by comparing the performance in defect prediction between the existing CPDP models and simple module size models.Method. We first use module size in the target project to build two simple defect prediction models, ManualDown and ManualUp, which do not require any training data from source projects. ManualDown considers a larger module as more defect-prone, while ManualUp considers a smaller module as more defect-prone. Then, we take the following measures to ensure a fair comparison on the performance in defect prediction between the existing CPDP models and the simple module size models: using the same publicly available data sets, using the same performance indicators, and using the prediction performance reported in the original cross-project defect prediction studies.Result. The simple module size models have a prediction performance comparable or even superior to most of the existing CPDP models in the literature, including many newly proposed models.Conclusion. The results caution us that, if the prediction performance is the goal, the real progress in CPDP is not being achieved as it might have been envisaged. We hence recommend that future studies should include ManualDown/ManualUp as the baseline models for comparison when developing new CPDP models to predict defects in a complete target project.

How Far We Have Progressed in the Journey? An Examination of Cross-Project Defect Prediction

Software defect prediction is one of the most popular research topics in software engineering. It aims to predict defect-prone software modules before defects are discovered, therefore it can be used to better prioritise software quality assurance effort. In recent years, especially for recent 3 years, many new defect prediction studies have been proposed. The goal of this study is to comprehensively review, analyse and discuss the state-of-the-art of defect prediction. The authors survey almost 70 representative defect prediction papers in recent years (January 2014-April 2017), most of which are published in the prominent software engineering journals and top conferences. The selected defect prediction papers are summarised to four aspects: machine learning-based prediction algorithms, manipulating the data, effort-aware prediction and empirical studies. The research community is still facing a number of challenges for building methods and many research opportunities exist. The identified challenges can give some practical guidelines for both software engineering researchers and practitioners in future software defect prediction.

Progress on approaches to software defect prediction

An empirical study to investigate oversampling methods for improving software defect prediction using imbalanced data

Heterogeneous defect prediction (HDP) refers to predicting defect-proneness of software modules in a target project using heterogeneous metric data from other projects. Existing HDP methods mainly focus on predicting target instances with single source. In practice, there exist plenty of external projects. Multiple sources can generally provide more information than a single project. Therefore, it is meaningful to investigate whether the HDP performance can be improved by employing multiple sources. However, a precondition of conducting HDP is that the external sources are available. Due to privacy concerns, most companies are not willing to share their data. To facilitate data sharing, it is essential to study how to protect the privacy of data owners before they release their data. In this paper, we study the above two issues in HDP. Specifically, to utilize multiple sources effectively, we propose a multi-source selection based manifold discriminant alignment (MSMDA) approach. To protect the privacy of data owners, a sparse representation based double obfuscation algorithm is designed and applied to HDP. Through a case study of 28 projects, our results show that MSMDA can achieve better performance than a range of baseline methods. The improvement is 3.4-$15.3$15.3 percent in g-measure and 3.0-$19.1$19.1 percent in AUC.

On the Multiple Sources and Privacy Preservation Issues for Heterogeneous Defect Prediction

It is envisioned that the Internet of Things (IoT) will provide promising opportunities to users, manufacturers, and service providers with a wide applicability in many fields. By employing social networking and device-to-device (D2D) communications in the IoT, the resulting social IoT can potentially provide services more effectively and efficiently. This paper focuses on content sharing among smart objects (devices) in the social IoT with D2D-based cooperative coded caching. Generally, complete content items or coded fragments are allowed to be delivered via multi-hop cooperative D2D communications. First, aiming at maximizing the overall success rate of multi-hop-based content sharing, the interplay between coding parameter optimization and wireless resource allocation is investigated by considering both physical and social characteristics. Moreover, a Roth and Vande Vate-based distributed scheme is proposed to solve the dynamic matching problem between the content helpers and content requesters. Numerical results demonstrate that the proposed scheme can achieve a good tradeoff between system performance and computational complexity.

https://ieeexplore.ieee.org/ielaam/7693/8334009/8234659-aam.pdf

Multi-Hop Cooperative Caching in Social IoT Using Matching Theory

Software defect prediction generally builds models from intra-project data. Lack of training data at the early stage of software testing limits the efficiency of prediction in practice. Thereby researchers proposed cross-project defect prediction using the data from other projects. Most previous efforts assumed the cross-project defect data have the same metrics set which means the metrics used and size of metrics set are same in the data of projects. However, in real scenarios, this assumption may not hold. In addition, software defect datasets have the class imbalance problem increasing the difficulty for the learner to predict defects. In this paper, we advance canonical correlation analysis for deriving a joint feature space for associating crossproject data and propose a novel support vector machine algorithm which incorporates the correlation transfer information into classifier design for cross-project prediction. Moreover, we take different misclassification costs into consideration to make the classification inclining to classify a module as a defective one, alleviating the impact of imbalanced data. Experiments on public heterogeneous datasets from different projects show that our method is more effective, compared to state-of-the-art methods. Keywords-defect prediction; heterogeneous metrics; class imbalance; canonical correlation analysis; support vector machine

/pdf/heterogeneous-defect-prediction-via-exploiting-correlation-owj8hxus41.pdf

Heterogeneous Defect Prediction via Exploiting Correlation Subspace.

A novel webpage layout aesthetic evaluation model for quantifying webpage layout design

Coding-based cooperative caching in on-demand data broadcast environments

Thesaurus‐based, code‐related, and software‐specific query expansion techniques are the main contributions in free‐form query search. However, these techniques still could not put the most relevant query result in the first position because they lack the ability to infer the expansion words that represent the user needs based on a given query. In this paper, we discover that code changes can imply what users want and propose a novel query expansion technique with code changes (QECC). It exploits (changes, contexts) pairs from changed methods. On the basis of statistical learning from pairs, it can infer code changes for a given query. In this way, it expands a query with code changes and recommends the query results that meet actual needs perfectly. In addition, we implement InstaRec to perform QECC and evaluate it with 195 039 change commits from GitHub and our code tracker. The results show that QECC can improve the precision of 3 code search algorithms (ie, IR, Portfolio, and VF) by up to 52% to 62% and outperform the state‐of‐the‐art query expansion techniques (ie, query expansion based on crowd knowledge and CodeHow) by 13% to 16% when the top 1 result is inspected.

Query expansion based on statistical learning from code changes

We present a semi-supervised approach for software defect prediction. The proposed method is designed to address the special problematic characteristics of software defect datasets, namely, lack of labeled samples and class-imbalanced data. To alleviate these problems, the proposed method features the following components. Being a semi-supervised approach, it exploits the wealth of unlabeled samples in software systems by evaluating the confidence probability of the predicted labels, for each unlabeled sample. And we propose to jointly optimize the classifier parameters and the dictionary by a task-driven formulation, to ensure that the learned features (sparse code) are optimal for the trained classifier. Finally, during the dictionary learning process we take the different misclassification costs into consideration to improve the prediction performance. Experimental results demonstrate that our method outperforms several representative state-of-the-art defect prediction methods.

https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/cje.2016.08.034

Guoqing Wu

Papers

Heterogeneous Defect Prediction via Exploiting Correlation Subspace.

A novel webpage layout aesthetic evaluation model for quantifying webpage layout design

Coding-based cooperative caching in on-demand data broadcast environments

Query expansion based on statistical learning from code changes

Semi-supervised Software Defect Prediction Using Task-Driven Dictionary Learning