by J. Biggs and C. Tang, Maidenhead, England,
Open University Press, 2007, 360 pp., £29.99, ISBN-13: 978-0-335-22126-4

Teaching for quality learning at university

This paper presents a systematic literature review of the recent (2006--2010) development of automatic assessment tools for programming exercises. We discuss the major features that the tools support and the different approaches they are using both from the pedagogical and the technical point of view. Examples of these features are ways for the teacher to define tests, resubmission policies, security issues, and so forth. We have also identified a list of novel features, like assessing web software, that are likely to get more research attention in the future. As a conclusion, we state that too many new systems are developed, but also acknowledge the current reasons for the phenomenon. As one solution we encourage opening up the existing systems and joining efforts on developing those further. Selected systems from our survey are briefly described in Appendix A.

Review of recent systems for automatic assessment of programming assignments

Web-Based Education for All: A Tool for Development Adaptive Courseware.

Three decades of active research on the teaching of introductory programming has had limited effect on classroom practice. Although relevant research exists across several disciplines including education and cognitive science, disciplinary differences have made this material inaccessible to many computing educators. Furthermore, computer science instructors have not had access to a comprehensive survey of research in this area. This paper collects and classifies this literature, identifies important work and mediates it to computing educators and professional bodies.We identify research that gives well-supported advice to computing academics teaching introductory programming. Limitations and areas of incomplete coverage of existing research efforts are also identified. The analysis applies publication and research quality metrics developed by a previous ITiCSE working group [74].

A survey of literature on the teaching of introductory programming

Efforts to improve computer science education are underway, and teachers of computer science are challenged in introductory programming courses to help learners develop their understanding of programming and computer science. Identifying and addressing students’ misconceptions is a key part of a computer science teacher's competence. However, relevant research on this topic is not as fully developed in the computer science education field as it is in mathematics and science education. In this article, we first review relevant literature on general definitions of misconceptions and studies about students’ misconceptions and other difficulties in introductory programming. Next, we investigate the factors that contribute to the difficulties. Finally, strategies and tools to address difficulties including misconceptions are discussed.Based on the review of literature, we found that students exhibit various misconceptions and other difficulties in syntactic knowledge, conceptual knowledge, and strategic knowledge. These difficulties experienced by students are related to many factors including unfamiliarity of syntax, natural language, math knowledge, inaccurate mental models, lack of strategies, programming environments, and teachers’ knowledge and instruction. However, many sources of students’ difficulties have connections with students’ prior knowledge. To better understand and address students’ misconceptions and other difficulties, various instructional approaches and tools have been developed. Nevertheless, the dissemination of these approaches and tools has been limited. Thus, first, we suggest enhancing the dissemination of existing tools and approaches and investigating their long-term effects. Second, we recommend that computing education research move beyond documenting misconceptions to address the development of students’ (mis)conceptions by integrating conceptual change theories. Third, we believe that developing and enhancing instructors’ pedagogical content knowledge (PCK), including their knowledge of students’ misconceptions and ability to apply effective instructional approaches and tools to address students’ difficulties, is vital to the success of teaching introductory programming.

https://dl.acm.org/doi/pdf/10.1145/3077618

Students’ Misconceptions and Other Difficulties in Introductory Programming: A Literature Review

This article is a survey of program visualization systems intended for teaching beginners about the runtime behavior of computer programs. Our focus is on generic systems that are capable of illustrating many kinds of programs and behaviors. We inclusively describe such systems from the last three decades and review findings from their empirical evaluations. A comparable review on the topic does not previously exist; ours is intended to serve as a reference for the creators, evaluators, and users of educational program visualization systems. Moreover, we revisit the issue of learner engagement which has been identified as a potentially key factor in the success of educational software visualization and summarize what little is known about engagement in the context of the generic program visualization systems for beginners that we have reviewed; a proposed refinement of the frameworks previously used by computing education researchers to rank types of learner engagement is a side product of this effort. Overall, our review illustrates that program visualization systems for beginners are often short-lived research prototypes that support the user-controlled viewing of program animations; a recent trend is to support more engaging modes of user interaction. The results of evaluations largely support the use of program visualization in introductory programming education, but research to date is insufficient for drawing more nuanced conclusions with respect to learner engagement. On the basis of our review, we identify interesting questions to answer for future research in relation to themes such as engagement, the authenticity of learning tasks, cognitive load, and the integration of program visualization into introductory programming pedagogy.

A Review of Generic Program Visualization Systems for Introductory Programming Education

Interaction and feedback are key factors supporting the learning process. Therefore many automatic assessment and feedback systems have been developed for computer science courses dur- ing the past decade. In this paper we present a new framework, TRAKLA2, for building interactive algorithm simulation exercises. Exercises constructed in TRAKLA2 are viewed as learning objects in which students manipulate conceptual visualizations of data structures in order to simulate the working of given algorithms. The framework supports randomized input values for the assignments, as well as automatic feedback and grading of students' simulation sequences. Moreover, it supports automatic generation of model solutions as algorithm animations and the logging of statistical data about the interaction process resulting as students s olve exercises. The system has been used in two universities in Finland for several courses invol ving over 1000 students. Student response has been very positive.

/pdf/visual-algorithm-simulation-exercise-system-with-automatic-4lorv2qh5h.pdf

Visual Algorithm Simulation Exercise System with Automatic Assessment: TRAKLA2

The process of solving a programming assignment is generally invisible to the teacher. We only see the end result and maybe a few snapshots along the way. In order to investigate this process with regard to Parsons problems, we used an online environment for Parsons problems in Python to record a detailed trace of all the interaction during the solving session. In these assignments, learners are to correctly order and indent a given set of code fragments in order to build a functioning program that meets the set requirements. We collected data from students of two programming courses and among other analyses present a visualization of the solution path as an interactive graph that can be used to explore such patterns and anomalies as backtracking and loops in the solution. The results provide insights into students' solving process for these types of problems and ideas on how to improve the assignment environment and its use in programming education.

/pdf/how-do-students-solve-parsons-programming-problems-an-3tj7wc5q5j.pdf

How do students solve parsons programming problems?: an analysis of interaction traces

In this paper, we present our experiences in using two automatic assessment tools, TRAKLA and TRAKLA2, in a second course of programming. In this course, 500--700 students have been enrolled annually during the period 1993--2004. The tools are specifically designed for assessing algorithm simulation exercises in which students simulate the working of algorithms at a conceptual level. Both of these tools allow students to resubmit their solutions after getting feedback. However, the resubmission policy has changed considerably during the period. Those changes reflect the students performance in the exercises. We conclude that an encouraging grading policy, i.e., the more exercises they solve the better grades they achive, combined with an option to resubmit the solution is a very important factor promoting students' learning. However, in order to prevent aimless trial-and-error problem solving method, the number of resubmissions allowed per assignment should be carefully controlled.

Ville Karavirta

Papers

Review of recent systems for automatic assessment of programming assignments

A Review of Generic Program Visualization Systems for Introductory Programming Education

Visual Algorithm Simulation Exercise System with Automatic Assessment: TRAKLA2

How do students solve parsons programming problems?: an analysis of interaction traces

Experiences on automatically assessed algorithm simulation exercises with different resubmission policies