Teachers deal with plagiarism on a regular basis, so they try to prevent and detect plagiarism, a task that is complicated by the large size of some classes. Students who cheat often try to hide their plagiarism (obfuscate), and many different similarity detection engines (often called plagiarism detection tools) have been built to help teachers. This article focuses only on plagiarism detection and presents a detailed systematic review of the field of source-code plagiarism detection in academia. This review gives an overview of definitions of plagiarism, plagiarism detection tools, comparison metrics, obfuscation methods, datasets used for comparison, and algorithm types. Perspectives on the meaning of source-code plagiarism detection in academia are presented, together with categorisations of the available detection tools and analyses of their effectiveness. While writing the review, some interesting insights have been found about metrics and datasets for quantitative tool comparison and categorisation of detection algorithms. Also, existing obfuscation methods classifications have been expanded together with a new definition of “source-code plagiarism detection in academia.”

Source-code Similarity Detection and Detection Tools Used in Academia: A Systematic Review

Traditionally, education relies on a linear relationship between enrollment and staff; rising enrollment dictates increases to staff with some expertise (such as teaching assistants, TAs) for evaluation. This relationship is expensive, so learning at scale has largely deemphasized expert evaluation and feedback. Two organizations, though, have used different models to scale up class size online while retaining this expert evaluation and feedback. In this paper, we analyze the methods these two organizations have used to increase enrollment while preserving scalability and feedback. We observe an academic program has scaled feedback with traditional TAs by relying on unique characteristics of its student body, while a commercial program has done so with a novel, network-based model. These successes show the potential of learning from experts at scale.

Scaling Expert Feedback: Two Case Studies

Meta evaluation theory and methods were used to evaluate the review experts of science and technology projects. Dozens of meta evaluation criteria can be found within two categories: the objective data of the experts' review results, i.e. the coefficient of deviation, the Spearman rank correlation coefficient, the reliability coefficient of binary value; the subjective data of the experts, i.e., the degree of experts' participation, the degree of review punctuality, the qualification of experts, and the degree of review seriousness. How these criteria impact each other has few been examined, and how to integrate the objective and subjective criteria need a comprehensive model with considering the fuzzy characteristics of the subjective criteria. This study targeted these two questions. An empirical study was adopted with hundreds of experts taking part in reviewing hundreds of Sci-Tech projects. The impacting relationships among the criteria were analyzed based on the empirical study. In order to deal with the intuitionistic fuzzy data on the subjective criteria and improve the estimation efficiency, an IVIF-BWM (best worst method under interval-valued intuitionistic fuzzy environment) was proposed by combining IVIF and the classical BWM to generate the importance weight for each criterion. The MULTIMOORA (multi-objective optimization by ratio analysis plus the full multiplicative form) was used to determine how to combine these criteria. At last, the proposed meta-evaluation model based on IVIF-BWM and MULTIMOORA was applied in a real case. The case study results supported the accuracy and reliability of the proposed model.

A meta-evaluation model on science and technology project review experts using IVIF-BWM and MULTIMOORA

In recent years, non-credit options for learning at scale have outpaced for-credit options. To scale for-credit options, workflows and policies must be devised to preserve the characteristics of accredited higher education---such as the presumption of human evaluation and an assertion of academic integrity---despite increased scale. These efforts must follow as well with shifting from offering isolated courses (or informal collections thereof) to offering full degree programs with additional administrative elements. We see this shift as one from Massive Open Online Courses (MOOCs) to Large, Internet-Mediated Asynchronous Degrees (Limeades). In this work, we perform a qualitative research study on one such program that has scaled to 6,500 students while retaining full accreditation. We report a typology of policies and workflows employed by the individual classes to deliver this experience.

Squeezing the limeade: policies and workflows for scalable online degrees

We outline how we detected programming plagiarism in an introductory online course for a master's of science in computer science program, how we achieved a statistically significant reduction in programming plagiarism by combining a clear explanation of university and class policy on academic honesty reinforced with a short but formal assessment, and how we evaluated plagiarism rates before SIGand after implementing our policy and assessment.

Collaboration Versus Cheating.

Large classes, both online and residential, typically demand many graders for evaluating students' written work. Some classes attempt to use autograding or peer grading, but these both present challenges to assigning grades at for-credit institutions, such as the difficulty of autograding to evaluate free-response answers and the lack of expert oversight in peer grading. In a large, online class at Georgia Tech in Summer 2015, we experimented with a new approach to grading: framing graders as meta-reviewers, charged with evaluating the original work in the context of peer reviews. To evaluate this approach, we conducted a pair of controlled experiments and a handful of qualitative analyses. We found that having access to peer reviews improves the perceived quality of feedback provided by graders without decreasing the graders' efficiency and with only a small influence on the grades assigned.

Graders as Meta-Reviewers: Simultaneously Scaling and Improving Expert Evaluation for Large Online Classrooms

Cheating in computer science classes can damage the reputation of institutions and their students. It is therefore essential to routinely authenticate student submissions with available software plagiarism detection algorithms such as Measure of Software Similarity (MOSS). Scaling this task for large classes where assignments are repeated each semester adds complexity and increases the instructor workload. The MOSS Tool for Addressing Plagiarism at Scale (MOSS-TAPS), organizes the MOSS submission task in courses that repeat coding assignments. In a recent use-case in the Online Master of Science in Computer Science (OMSCS) program at the Georgia Institute of Technology, the instructor time spent was reduced from 50 hours to only 10 minutes using the managed submission tool design presented here. MOSS-TAPS provides persistent configuration, supports a mixture of software languages and file organizations, and is implemented in pure Java for cross-platform compatibility.

Dana Sheahen

Papers

Graders as Meta-Reviewers: Simultaneously Scaling and Improving Expert Evaluation for Large Online Classrooms

TAPS: A MOSS Extension for Detecting Software Plagiarism at Scale