Powergrading: a Clustering Approach to Amplify Human Effort for Short Answer Grading
Sumit Basu,Charles E. Jacobs,Lucy Vanderwende +2 more
- Vol. 1, pp 391-402
TLDR
This paper used a similarity metric between student responses, and then used this metric to group responses into clusters and subclusters, which allowed teachers to grade multiple responses with a single action, provide rich feedback to groups of similar answers, and discover modalities of misunderstanding among students.Abstract:
We introduce a new approach to the machine-assisted grading of short answer questions. We follow past work in automated grading by first training a similarity metric between student responses, but then go on to use this metric to group responses into clusters and subclusters. The resulting groupings allow teachers to grade multiple responses with a single action, provide rich feedback to groups of similar answers, and discover modalities of misunderstanding among students; we refer to this amplification of grader effort as “powergrading.” We develop the means to further reduce teacher effort by automatically performing actions when an answer key is available. We show results in terms of grading progress with a small “budget” of human actions, both from our method and an LDA-based approach, on a test corpus of 10 questions answered by 698 respondents.read more
Citations
More filters
Proceedings Article
Semi-Supervised Clustering for Short Answer Scoring
Andrea Horbach,Manfred Pinkal +1 more
TL;DR: This paper proposes to re-allocate some of the human annotation effort to before and during the clustering process for (i) feature selection, (ii) for creating pairwise constraints and (iii) for metric learning.
Proceedings Article
ESCRITO - An NLP-Enhanced Educational Scoring Toolkit.
Torsten Zesch,Andrea Horbach +1 more
TL;DR: This article proposed ESCRITO, a toolkit for scoring student writings using NLP techniques that addresses two main user groups: teachers and NLP researchers, and it provides a ready-made testbed for applying the latest developments from NLP areas like text similarity, paraphrase detection, textual entailment, and argument mining.
Proceedings ArticleDOI
Preventing Critical Scoring Errors in Short Answer Scoring with Confidence Estimation.
Hiroaki Funayama,Shota Sasaki,Yuichiroh Matsubayashi,Tomoya Mizumoto,Jun Suzuki,Masato Mita,Kentaro Inui +6 more
TL;DR: It is demonstrated that a typical SAS system can predict scores with zero CSE for approximately 50% of test data at maximum by filtering out low-reliablility predictions on the basis of a certain confidence estimation, which indicates the possibility of reducing half the scoring cost of human raters.
Proceedings ArticleDOI
A Machine Learning Approach for Suggesting Feedback in Textual Exercises in Large Courses
TL;DR: In this paper, a machine learning approach called CoFee is proposed to suggest computer-aided feedback in open-ended textual exercises, which uses topic modeling to split student answers into text segments and language embeddings to transform these segments.
Proceedings ArticleDOI
Elicast: embedding interactive exercises in instructional programming screencasts
TL;DR: Elicast is introduced, a screencast tool for recording and viewing programming lectures with embedded programming exercises, to provide hands-on programming experiences in the screen-cast and found that instructors structured the lectures into small learning units using embedded exercises as checkpoints.
References
More filters
Journal ArticleDOI
Latent dirichlet allocation
TL;DR: This work proposes a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hofmann's aspect model.
Proceedings Article
Latent Dirichlet Allocation
TL;DR: This paper proposed a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI).
Book
Finding Groups in Data: An Introduction to Cluster Analysis
TL;DR: An electrical signal transmission system, applicable to the transmission of signals from trackside hot box detector equipment for railroad locomotives and rolling stock, wherein a basic pulse train is transmitted whereof the pulses are of a selected first amplitude and represent a train axle count.
Journal ArticleDOI
An algorithm for suffix stripping
TL;DR: An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
Journal ArticleDOI
Finding Groups in Data: An Introduction to Chster Analysis
TL;DR: This book make understandable the cluster analysis is based notion of starsmodern treatment, which efficiently finds accurate clusters in data and discusses various types of study the user set explicitly but also proposes another.