he Kyoto Protocol is a symbolically important expression of governments' concern about climate change. But as an instrument for achieving emissions reductions, it has failed 1. It has produced no demonstrable reductions in emissions or even in anticipated emissions growth. And it pays no more than token attention to the needs of societies to adapt to existing climate change. The impending United Nations Climate Change Conference being held in Bali in December — to decide international policy after 2012 — needs to radically rethink climate policy. Kyoto's supporters often blame non-signatory governments, especially the United States and Australia, for its woes. But the Kyoto Protocol was always the wrong tool for the nature of the job. Kyoto was constructed by quickly borrowing from past treaty regimes dealing with stratospheric ozone depletion, acid rain from sulphur emissions and nuclear weapons. Drawing on these plausible but partial analogies , Kyoto's architects assumed that climate change would be best attacked directly through global emissions controls, treating tonnes of carbon dioxide like stockpiles of nuclear weapons to be reduced via mutually verifiable targets and timetables. Unfortunately this borrowing simply failed to accommodate the complexity of the climate-change issue 2. Kyoto has failed in several ways, not just in its failure to slow global warming , but also because it has stifled discussion of alternative policy approaches that could both combat climate change and adapt to its unavoidable consequences. As Kyoto became a litmus test of political correctness, those who were concerned about climate change, but sceptical of the top–down approach adopted by the Protocol were sternly admonished that \" Kyoto is the only game in town \". We are anxious that the same mistake is not repeated in the current round of negotiations. Already, in the post-Kyoto discussions, we are witnessing that well-documented human response to failure, especially where political or emotional capital is involved, which is to insist on more of what is not working: in this case more stringent targets and timetables, involving more countries. The next round of negotiations needs to open up new approaches, not to close them down as Kyoto did. Economic theory recognizes the futility of throwing good money after bad. In politics, however, sunk costs are often seen as political capital or as an investment of reputation and status. So we acknowledge that those advocating the Kyoto regime will be reluctant to embrace alternatives because it means …

No silver bullet

Increasingly, software is making autonomous decisions in case of criminal sentencing, approving credit cards, hiring employees, and so on. Some of these decisions show bias and adversely affect certain social groups (e.g. those defined by sex, race, age, marital status). Many prior works on bias mitigation take the following form: change the data or learners in multiple ways, then see if any of that improves fairness. Perhaps a better approach is to postulate root causes of bias and then applying some resolution strategy. This paper postulates that the root causes of bias are the prior decisions that affect- (a) what data was selected and (b) the labels assigned to those examples. Our Fair-SMOTE algorithm removes biased labels; and rebalances internal distributions such that based on sensitive attribute, examples are equal in both positive and negative classes. On testing, it was seen that this method was just as effective at reducing bias as prior approaches. Further, models generated via Fair-SMOTE achieve higher performance (measured in terms of recall and F1) than other state-of-the-art fairness improvement algorithms. To the best of our knowledge, measured in terms of number of analyzed learners and datasets, this study is one of the largest studies on bias mitigation yet presented in the literature.

Bias in Machine Learning Software: Why? How? What to do?

From the Publisher: 
This new work from Watts Humphrey, author of the influential book, Managing the Software Process, broadens his orderly view of software process management, and lays the foundation for a disciplined approach to software engineering. In his earlier book, the author developed concrete methods for managing software development and maintenance. These methods, now commonly practiced in industry, provide programmers and managers with specific steps they can take to evaluate and improve their software capabilities. In this new book, Humphrey scales those methods down to a personal level, helping software engineers develop the skills and habits needed to plan, track, and analyze large, complex projects. Humphrey and others have used material from this book to train professionals and students around the world in a projects-oriented software engineering course. First establishing the need for discipline in software engineering, and the benefits to practitioners of learning how to manage their personal software process, Humphrey then develops a model that they can use to monitor, test, and improve their work. Examples drawn from industry enhance the practical focus of the book, while project exercises give readers the opportunity to practice software process management as they learn it. Features: presents concepts and methods for a disciplined software engineering process; scales down industrial practices for planning, tracking, analysis, and defect management to fit the needs of small-scale program development; and shows how small project disciplines provide a solid base for larger projects.

Discipline for Software Engineering a

Machine learning software is increasingly being used to make decisions that affect people's lives. But sometimes, the core part of this software (the learned model), behaves in a biased manner that gives undue advantages to a specific group of people (where those groups are determined by sex, race, etc.). This "algorithmic discrimination" in the AI software systems has become a matter of serious concern in the machine learning and software engineering community. There have been works done to find "algorithmic bias" or "ethical bias" in the software system. Once the bias is detected in the AI software system, the mitigation of bias is extremely important. In this work, we a)explain how ground-truth bias in training data affects machine learning model fairness and how to find that bias in AI software,b)propose a methodFairwaywhich combines pre-processing and in-processing approach to remove ethical bias from training data and trained model. Our results show that we can find bias and mitigate bias in a learned model, without much damaging the predictive performance of that model. We propose that (1) test-ing for bias and (2) bias mitigation should be a routine part of the machine learning software development life cycle. Fairway offers much support for these two purposes.

Fairway: A Way to Build Fair ML Software

Context: Continuous Integration (CI) environments allow frequent integration of software changes, making software evolution more rapid and cost-effective. In such environments, the regression test plays an important role, as well as the use of Test Case Prioritization (TCP) techniques. Such techniques attempt to identify the test case order that maximizes certain goals, such as early fault detection. This research subject has been raising interest because some new challenges are faced in the CI context, as TCP techniques need to consider time constraints of the CI environments. Objective: This work presents the results of a systematic mapping study on Test Case Prioritization in Continuous Integration environments (TCPCI) that reports the main characteristics of TCPCI approaches and their evaluation aspects. Method: The mapping was conducted following a plan that includes the definition of research questions, selection criteria and search string, and the selection of search engines. The search returned 35 primary studies classified based on the goal and kind of used TCP technique, addressed CI particularities and testing problems, and adopted evaluation measures. Results: The results show a growing interest in this research subject. Most studies have been published in the last four years. 80% of the approaches are history-based, that is, are based on the failure and test execution history. The great majority of studies report evaluation results by comparing prioritization techniques. The preferred measures are Time and number/percentage of Faults Detected. Few studies address CI testing problems and characteristics, such as parallel execution and test case volatility. Conclusions: We observed a growing number of studies in the field. Future work should explore other information sources such as models and requirements, as well as CI particularities and testing problems, such as test case volatility, time constraint, and flaky tests, to solve existing challenges and offer cost-effective approaches to the software industry.

Test Case Prioritization in Continuous Integration environments: A systematic mapping study

Keeping track of and managing Self-Admitted Technical Debts (SATDs) are important to maintaining a healthy software project. This requires much time and effort from human experts to identify the SATDs manually. The current automated solutions do not have satisfactory precision and recall in identifying SATDs to fully automate the process. To solve the above problems, we propose a two-step framework called Jitterbug for identifying SATDs. Jitterbug first identifies the "easy to find" SATDs automatically with close to 100% precision using a novel pattern recognition technique. Subsequently, machine learning techniques are applied to assist human experts in manually identifying the remaining "hard to find" SATDs with reduced human effort. Our simulation studies on ten software projects show that Jitterbug can identify SATDs more efficiently (with less human effort) than the prior state-of-the-art methods.

Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach

We assert that it is the ethical duty of software engineers to strive to reduce software discrimination. This paper discusses how that might be done. This is an important topic since machine learning software is increasingly being used to make decisions that affect people's lives. Potentially, the application of that software will result in fairer decisions because (unlike humans) machine learning software is not biased. However, recent results show that the software within many data mining packages exhibits "group discrimination"; i.e. their decisions are inappropriately affected by "protected attributes"(e.g., race, gender, age, etc.). 
There has been much prior work on validating the fairness of machine-learning models (by recognizing when such software discrimination exists). But after detection, comes mitigation. What steps can ethical software engineers take to reduce discrimination in the software they produce? 
This paper shows that making \textit{fairness} as a goal during hyperparameter optimization can (a) preserve the predictive power of a model learned from a data miner while also (b) generates fairer results. To the best of our knowledge, this is the first application of hyperparameter optimization as a tool for software engineers to generate fairer software.

Software Engineering for Fairness: A Case Study with Hyperparameter Optimization.

Automated UI testing is an important component of the continuous integration process of software development. A modern web-based UI is an amalgam of reports from dozens of microservices written by multiple teams. Queries on a page that opens up another will fail if any of that page's microservices fails. As a result, the overall cost for automated UI testing is high since the UI elements cannot be tested in isolation. For example, the entire automated UI testing suite at LexisNexis takes around 30 hours (3-5 hours on the cloud) to execute, which slows down the continuous integration process. To mitigate this problem and give developers faster feedback on their code, test case prioritization techniques are used to reorder the automated UI test cases so that more failures can be detected earlier. Given that much of the automated UI testing is "black box" in nature, very little information (only the test case descriptions and testing results) can be utilized to prioritize these automated UI test cases. Hence, this paper evaluates 17 "black box" test case prioritization approaches that do not rely on source code information. Among these, we propose a novel TCP approach, that dynamically re-prioritizes the test cases when new failures are detected, by applying and adapting a state of the art framework from the total recall problem. Experimental results on LexisNexis automated UI testing data show that our new approach (which we call TERMINATOR), outperformed prior state of the art approaches in terms of failure detection rates with negligible CPU overhead.

https://dl.acm.org/doi/pdf/10.1145/3338906.3340448

TERMINATOR: better automated UI test case prioritization

Block-based programming environments are widely used by novices who are learning computer science. However, even in block-based coding environments that have been carefully developed to serve novices, students frequently struggle and require additional support. A promising avenue to provide this support is the use of intelligent tutoring systems, which offer adaptive hints to assist learners. In order to provide students with the adaptive hints they need, we must investigate their help-seeking behaviors and identify patterns surrounding their need for support. In this experience report, we examine data collected from 174 college students in an introductory engineering course, who used an intelligent block-based coding environment to learn computer science. These students made more than 1,000 hint requests, which we represent in two-dimensional space along axes of elapsed time and code completeness. Analysis revealed five major clusters of hint requests, which we further characterized through qualitative examination of the coding trajectories that preceded each hint request. We also analyzed how students' incoming knowledge and perceived computer skill were related to their help-seeking behaviors. Students with higher incoming knowledge requested hints when their code was more complete than students with lower incoming knowledge. Students with high perceived computer skill asked for hints when their code was less complete than those with low perceived computer skill. The results presented here provide insight into student help-seeking behavior in computer science education, informing CS educators and system designers on how best to develop support strategies.

Exploring Novice Programmers' Hint Requests in an Intelligent Block-Based Coding Environment

Software analytics can be improved by surveying; i.e. rechecking and (possibly) revising the labels offered by prior analysis. Surveying is a time-consuming task and effective surveyors must carefully manage their time. Specifically, they must balance the cost of further surveying against the additional benefits of that extra effort. This paper proposes SURVEY0, an incremental Logistic Regression estimation method that implements cost/benefit analysis. Some classifier is used to rank the as-yet-unvisited examples according to how interesting they might be. Humans then review the most interesting examples, after which their feedback is used to update an estimator for estimating how many examples are remaining. This paper evaluates SURVEY0 in the context of self-admitted technical debt. As software project mature, they can accumulate "technical debt" i.e. developer decisions which are sub-optimal and decrease the overall quality of the code. Such decisions are often commented on by programmers in the code; i.e. it is self-admitted technical debt (SATD). Recent results show that text classifiers can automatically detect such debt. We find that we can significantly outperform prior results by SURVEYing the data. Specifically, for ten open-source JAVA projects, we can find 83% of the technical debt via SURVEY0 using just 16% of the comments (and if higher levels of recall are required, SURVEY0can adjust towards that with some additional effort).

Fahmid Morshed Fahid

Papers

Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach

Software Engineering for Fairness: A Case Study with Hyperparameter Optimization.

TERMINATOR: better automated UI test case prioritization

Exploring Novice Programmers' Hint Requests in an Intelligent Block-Based Coding Environment

Better Technical Debt Detection via SURVEYing