Web-based peer assessment : feedback for students with various thinking-styles

doi:10.1046/J.0266-4909.2001.00198.X

Home
/
Papers
/
Web-based peer assessment : feedback for students with various thinking-styles

Journal Article•DOI•

Web-based peer assessment : feedback for students with various thinking-styles

Sunny S. J. Lin¹, Eric Zhi Feng Liu¹, Shyan-Ming Yuan¹•Institutions (1)

National Chiao Tung University¹

01 Dec 2001-Journal of Computer Assisted Learning (Wiley-Blackwell)-Vol. 17, Iss: 4, pp 420-432

TL;DR: While students with high executive thinking styles significantly improved over two rounds of peer assessment, low executive students did not improve through the cycles, suggesting that future web-based peer assessment adopts a specific feedback format for all students.

read less

Abstract: This study used aptitude treatment interaction design to examine how feedback formats (specific vs. holistic) and executive thinking styles (high vs. low) affect web-based peer assessment. An Internet-based (anonymous) peer-assessment system was developed and used by 58 computer science students who submitted assignments for peer review. The results indicated that while students with high executive thinking styles significantly improved over two rounds of peer assessment, low executive students did not improve through the cycles. In addition, high executive students contributed substantially better feedback than their low executive counterparts. In the second round of peer assessment, thinking style and feedback format interactively affected student learning. Low executive students receiving specific feedback significantly outperformed those receiving holistic feedback. In receiving holistic feedback, high executive thinkers outperformed their low executive counterparts. This study suggests that future web-based peer assessment adopts a specific feedback format for all students.

...read moreread less

Citations

PDF

Open Access

More filters

Book Chapter•DOI•

Defining Twenty-First Century Skills

[...]

Marilyn Binkley¹, Ola Erstad², Joan L. Herman³, Senta Raizen, Martin Ripley, May Miller-Ricci⁴, Mike Rumble - Show less +3 more•Institutions (4)

University of Luxembourg¹, University of Oslo², University of California³, WestEd⁴

01 Jan 2012

TL;DR: In this article, the authors argue that success lies in being able to communicate, share, and use information to solve complex problems, in adapting and innovating in response to new demands and changing circumstances, in marshaling and expanding the power of technology to create new knowledge, and in expanding human capacity and productivity.

...read moreread less

Abstract: As the previous chapter indicates, there has been a significant shift in advanced economies from manufacturing to information and knowledge services. Knowledge itself is growing ever more specialized and expanding exponentially. Information and communication technology is transforming the nature of how work is conducted and the meaning of social relationships. Decentralized decision making, information sharing, teamwork, and innovation are key in today’s enterprises. No longer can students look forward to middle class success in the conduct of manual labor or use of routine skills – work that can be accomplished by machines. Rather, whether a technician or a professional person, success lies in being able to communicate, share, and use information to solve complex problems, in being able to adapt and innovate in response to new demands and changing circumstances, in being able to marshal and expand the power of technology to create new knowledge, and in expanding human capacity and productivity.

...read moreread less

1,056 citations

Book•DOI•

International handbook of information technology in primary and secondary education

[...]

Joke Voogt¹, Gerald Knezek²•Institutions (2)

University of Amsterdam¹, University of North Texas²

01 Jan 2008

681 citations

Journal Article•DOI•

The nature of feedback: how different types of peer feedback affect writing performance

[...]

Melissa M. Nelson¹, Christian D. Schunn¹•Institutions (1)

University of Pittsburgh¹

01 Jul 2009-Instructional Science

TL;DR: Five main predictions were developed from the feedback literature in writing regarding feedback features as they relate to potential causal mediators of problem or solution understanding and problem or solutions agreement, leading to the final outcome of feedback implementation.

...read moreread less

Abstract: Although providing feedback is commonly practiced in education, there is no general agreement regarding what type of feedback is most helpful and why it is helpful. This study examined the relationship between various types of feedback, potential internal mediators, and the likelihood of implementing feedback. Five main predictions were developed from the feedback literature in writing, specifically regarding feedback features (summarization, identifying problems, providing solutions, localization, explanations, scope, praise, and mitigating language) as they relate to potential causal mediators of problem or solution understanding and problem or solution agreement, leading to the final outcome of feedback implementation. To empirically test the proposed feedback model, 1,073 feedback segments from writing assessed by peers was analyzed. Feedback was collected using SWoRD, an online peer review system. Each segment was coded for each of the feedback features, implementation, agreement, and understanding. The correlations between the feedback features, levels of mediating variables, and implementation rates revealed several significant relationships. Understanding was the only significant mediator of implementation. Several feedback features were associated with understanding: including solutions, a summary of the performance, and the location of the problem were associated with increased understanding; and explanations of problems were associated with decreased understanding. Implications of these results are discussed.

...read moreread less

436 citations

Journal Article•DOI•

Effective peer assessment processes: Research findings and future directions

[...]

Marjo van Zundert¹, Dominique Sluijsmans¹, Jeroen J. G. van Merriënboer¹•Institutions (1)

Open University¹

01 Aug 2010-Learning and Instruction

TL;DR: In this article, a literature review on peer assessment (PA) found that the development of domain-specific skills benefits from PA-based revision, while the development and development of PA skills benefit from training and are related to students' thinking style and academic achievement.

...read moreread less

428 citations

Cites background from "Web-based peer assessment : feedbac..."

...The study by Lin et al. (2001) also revealed an interaction effect, namely low executive thinkers did better when receiving specific feedback instead of holistic feedback, whereas the feedback specificity did not matter for high executive thinkers....
[...]
...Lin, Liu, and Yuan (2001) found that in the case of PA and revision, students’ thinking style influences the quality of their work when they get the opportunity to revise it on the basis of feedback....
[...]

Journal Article•DOI•

On-line peer assessment and the role of the peer feedback: A study of high school computer course

[...]

Sheng-Chau Tseng¹, Chin Chung Tsai¹•Institutions (1)

National Chiao Tung University¹

01 Dec 2007-Computer Education

TL;DR: It was found that Reinforcing peer feedback was useful in helping students' development of better projects; however, Didactic feedback and perhaps Corrective feedback provided by peers might play an unfavorable role for subsequent improvement of students' projects.

...read moreread less

Abstract: The purposes of this study were to explore the effects and the validity of on-line peer assessment in high schools and to analyze effects of various types of peer feedback on students. The participating students, a total of 184 10th graders, developed their individual course projects by involving the on-line peer assessment learning activities in a computer course. The peer assessment activities consisted of three rounds, and each of the students acted as an author and a reviewer. Research data as evaluated from peers and experts indicated that students significantly improved their projects as involving the peer assessment activities. The scores determined by the learning peers were highly correlated with those marked by the experts, indicating that peer assessment in high school could be perceived as a valid assessment method. Moreover, this study also examined the relationships between the types of peer feedbacks in which students obtained from peer assessment and the subsequent performance of their projects. We categorized peer feedbacks into four types: Reinforcing, Didactic, Corrective and Suggestive. It was found that Reinforcing peer feedback was useful in helping students' development of better projects; however, Didactic feedback and perhaps Corrective feedback provided by peers might play an unfavorable role for subsequent improvement of students' projects. The Suggestive feedback may be helpful in the beginning of peer assessment activities; however, in the later parts of peer assessment, the effect of this type of feedback on learning might not be significant.

...read moreread less

278 citations

Cites background from "Web-based peer assessment : feedbac..."

...In addition, when Lin et al. (2001b) managed webbased peer-review activities, they observed that students did learn effectively from reading numerous peers’ work and feedbacks....
[...]
...…identify their own strengths and weaknesses, target areas for remedial action, develop metacognitive and professional transferable skills, and enhance their reflective thinking and problem solving abilities during the learning experience (Sluijsmans, Dochy, & Moerkerke, 1999; Smith, Cooper, &…...
[...]
...The peer assessment activities consisted of three rounds, and each of the students acted as an author and a reviewer....
[...]
...Lin et al. (2001a) found many students did not improve over two rounds in an on-line peer assessment study....
[...]
...Moreover, Lin et al. (2001a) suggested that web-based peer assessment at least has the following advantages over traditional paper-and-pencil peer assessment: 1....
[...]

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

Collapse

References

PDF

Open Access

More filters

A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ).

[...]

Paul R. Pintrich

01 Jan 1991

TL;DR: Pintrich and Pintrich as discussed by the authors presented a manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ) for teaching and learning at the National Center for Research to improve postsecondary education.

...read moreread less

Abstract: AUTHOR Pintrich, Paul R.; And Others TITLE A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). INSTITUTION National Center for Research to Improve Postsecondary Teaching and Learning, Ann Arbor, MI. SPONS AGENCY Office of Educational Research and Improvement (ED), Washington, DC. REPORT NO NCRIPTAL-91-B-004 PUB DATE 91 CONTRACT OERI-86-0010 NOTE 76p.; From the National Center for Research to Improve Postsecondary Teaching and Learning Project on Instructional Processes and Edrcational Outcomes. For a related document, see HE 024 936. PUB TYPE Guides Non-Classroom Use (055) -Tests/Evaluation Instruments (160)

...read moreread less

2,913 citations

Journal Article•DOI•

Peer Assessment Between Students in Colleges and Universities

[...]

Keith Topping¹•Institutions (1)

University of Dundee¹

01 Sep 1998-Review of Educational Research

TL;DR: A definition and typology of peer assessment between students in higher education is proposed, and the theoretical underpinnings of the method are discussed in this paper, and a review of the developing literature follows, including both process and outcome studies.

...read moreread less

Abstract: A definition and typology of peer assessment between students in higher education is proposed, and the theoretical underpinnings of the method are discussed. A review of the developing literature follows, including both process and outcome studies. This indicates that peer assessment is of adequate reliability and validity in a wide variety of applications. Peer assessment of writing and peer assessment using marks, grades, and tests have shown positive formative effects on student achievement and attitudes. These effects are as good as or better than the effects of teacher assessment. Evidence for such effects from other types of peer assessment (of presentation skills, group work or projects, and professional skills) is, as yet, more limited. Computer-assisted peer assessment is an emerging growth area. Important factors in successful implementation are summarized, and recommendations for future research and practice are made.

...read moreread less

1,579 citations

"Web-based peer assessment : feedbac..." refers background in this paper

...While reviewing recent developments in peer assessment, Topping (1998) found that most students benefit from this assessment method....
[...]
...This study has provided further insight into individual difference and feedback effect on peer assessment that was not addressed in the review by Topping (1998)....
[...]
...This study has provided further insight into individual difference and feedback effect on peer assessment that was not addressed in the review by Topping (1998). High executive students appear to benefit from peer assessment the most....
[...]

Journal Article•DOI•

The Impact of Classroom Evaluation Practices on Students

[...]

Terence J. Crooks

01 Jan 1988-Review of Educational Research

TL;DR: In this article, a review summarizes results from 14 specific fields of research that cast light on the relationships between classroom evaluation practices and student outcomes and concludes that classroom evaluation has powerful direct and indirect impacts, which may be positive or negative.

...read moreread less

Abstract: In most educational programs, a substantial proportion of teacher and student time is devoted to activities which involve (or lead directly to) evaluation by the teacher of student products or behavior. This review summarizes results from 14 specific fields of research that cast light on the relationships between classroom evaluation practices and student outcomes. Particular attention is given to outcomes involving learning strategies, motivation, and achievement. Where possible, mechanisms are suggested that could account for the reported effects. The conclusions derived from the individual fields are then merged to produce an integrated summary with clear implications for effective educational practice. The primary conclusion is that classroom evaluation has powerful direct and indirect impacts, which may be positive or negative, and thus deserves very thoughtful planning and implementation.

...read moreread less

1,370 citations

"Web-based peer assessment : feedbac..." refers background in this paper

...Previous studies (Crooks, 1988; Kulik & Kulik, 1988; Bangert-Drowns et al. 1991) indicate that receiving feedback is correlated with effective learning. Receiving abundant and immediate peer feedback can prevent some errors and provide hints for making progress. In addition to the positive effects of peer assessment, Lin et al. (2001a) observed that some students had negative feelings about this learning strategy. Some students disliked peer assessment because raters were also competitors. In one case, students could change their previous score during a certain period. Upon receiving an unexpectedly low score from peers, students often reduced the previous scores they had given to others as a form of retaliation. Moreover, students often believe that only teachers have the ability and knowledge to evaluate and provide critical feedback (Zhao, 1998). They may suspect peers’ ability; in particular, those who receive lower scores regard peer assessment as inaccurate (McDowell, 1995). Furthermore, many educators refuse to adopt peer assessment owing to the possibility of overmarking or undermarking peers’ performance. Web-based peer assessment has some advantages over ordinary peer assessment. First, students evaluate peers’ work through the web (not in a face-to-face presentation), thereby ensuring anonymity and facilitating a willingness to critique. Second, web-based peer assessment allows teachers to monitor students’ progress at any period of the assessment process. Teachers can always determine how well an assessor or assessee performs and constantly monitors the process whereas this is nearly impossible during ordinary peer assessment when several rounds are involved. Third, web-based peer assessment can decrease photocopying time and expense since assessees do not need to photocopy their assignments for their peer assessors. Some researchers (Downing & Brown, 1997; Davis & Berrow, 1998; Zhao, 1998) explored the feasibility of Internet supported peer assessment. However these studies mainly relied on general-purpose applications or commercial software, such as email, electronic communication applications or ftp. Using applications not specifically designed for peer assessment may either increase management load or cause difficulty in maintaining anonymity among peers. Kwok & Ma (1999) and Rada (1998) are among the few researchers who have implemented web-based peer assessment. Kwok & Ma (1999) used Group Support Systems (GSS) to support collaborative and peer assessment....
[...]
...Previous studies (Crooks, 1988; Kulik & Kulik, 1988; Bangert-Drowns et al. 1991) indicate that receiving feedback is correlated with effective learning. Receiving abundant and immediate peer feedback can prevent some errors and provide hints for making progress. In addition to the positive effects of peer assessment, Lin et al. (2001a) observed that some students had negative feelings about this learning strategy. Some students disliked peer assessment because raters were also competitors. In one case, students could change their previous score during a certain period. Upon receiving an unexpectedly low score from peers, students often reduced the previous scores they had given to others as a form of retaliation. Moreover, students often believe that only teachers have the ability and knowledge to evaluate and provide critical feedback (Zhao, 1998). They may suspect peers’ ability; in particular, those who receive lower scores regard peer assessment as inaccurate (McDowell, 1995). Furthermore, many educators refuse to adopt peer assessment owing to the possibility of overmarking or undermarking peers’ performance. Web-based peer assessment has some advantages over ordinary peer assessment. First, students evaluate peers’ work through the web (not in a face-to-face presentation), thereby ensuring anonymity and facilitating a willingness to critique. Second, web-based peer assessment allows teachers to monitor students’ progress at any period of the assessment process. Teachers can always determine how well an assessor or assessee performs and constantly monitors the process whereas this is nearly impossible during ordinary peer assessment when several rounds are involved. Third, web-based peer assessment can decrease photocopying time and expense since assessees do not need to photocopy their assignments for their peer assessors. Some researchers (Downing & Brown, 1997; Davis & Berrow, 1998; Zhao, 1998) explored the feasibility of Internet supported peer assessment. However these studies mainly relied on general-purpose applications or commercial software, such as email, electronic communication applications or ftp. Using applications not specifically designed for peer assessment may either increase management load or cause difficulty in maintaining anonymity among peers. Kwok & Ma (1999) and Rada (1998) are among the few researchers who have implemented web-based peer assessment. Kwok & Ma (1999) used Group Support Systems (GSS) to support collaborative and peer assessment. Rada (1998) supervised three classes of computer science students to solve exercise problems and submitted solutions for peer review using a Many Using and Creating Hypermedia system (MUCH)....
[...]
...Previous studies (Crooks, 1988; Kulik & Kulik, 1988; Bangert-Drowns et al. 1991) indicate that receiving feedback is correlated with effective learning. Receiving abundant and immediate peer feedback can prevent some errors and provide hints for making progress. In addition to the positive effects of peer assessment, Lin et al. (2001a) observed that some students had negative feelings about this learning strategy. Some students disliked peer assessment because raters were also competitors. In one case, students could change their previous score during a certain period. Upon receiving an unexpectedly low score from peers, students often reduced the previous scores they had given to others as a form of retaliation. Moreover, students often believe that only teachers have the ability and knowledge to evaluate and provide critical feedback (Zhao, 1998). They may suspect peers’ ability; in particular, those who receive lower scores regard peer assessment as inaccurate (McDowell, 1995). Furthermore, many educators refuse to adopt peer assessment owing to the possibility of overmarking or undermarking peers’ performance. Web-based peer assessment has some advantages over ordinary peer assessment. First, students evaluate peers’ work through the web (not in a face-to-face presentation), thereby ensuring anonymity and facilitating a willingness to critique. Second, web-based peer assessment allows teachers to monitor students’ progress at any period of the assessment process. Teachers can always determine how well an assessor or assessee performs and constantly monitors the process whereas this is nearly impossible during ordinary peer assessment when several rounds are involved. Third, web-based peer assessment can decrease photocopying time and expense since assessees do not need to photocopy their assignments for their peer assessors. Some researchers (Downing & Brown, 1997; Davis & Berrow, 1998; Zhao, 1998) explored the feasibility of Internet supported peer assessment. However these studies mainly relied on general-purpose applications or commercial software, such as email, electronic communication applications or ftp. Using applications not specifically designed for peer assessment may either increase management load or cause difficulty in maintaining anonymity among peers. Kwok & Ma (1999) and Rada (1998) are among the few researchers who have implemented web-based peer assessment....
[...]
...Previous studies (Crooks, 1988; Kulik & Kulik, 1988; Bangert-Drowns et al. 1991) indicate that receiving feedback is correlated with effective learning....
[...]
...Previous studies (Crooks, 1988; Kulik & Kulik, 1988; Bangert-Drowns et al. 1991) indicate that receiving feedback is correlated with effective learning. Receiving abundant and immediate peer feedback can prevent some errors and provide hints for making progress. In addition to the positive effects of peer assessment, Lin et al. (2001a) observed that some students had negative feelings about this learning strategy....
[...]

Journal Article•DOI•

The Instructional Effect of Feedback in Test-Like Events

[...]

Robert L. Bangert-Drowns, Chen-Lin C. Kulik, James A. Kulik, MaryTeresa Morgan

01 Jun 1991-Review of Educational Research

TL;DR: The authors reviewed 58 effect sizes from 40 reports and found that feedback effects vary with control for presearch availability, type of feedback, use of pretests, and type of instruction and could be quite large under optimal conditions.

...read moreread less

Abstract: Feedback is an essential construct for many theories of learning and instruction, and an understanding of the conditions for effective feedback should facilitate both theoretical development and instructional practice. In an early review of feedback effects in written instruction, Kulhavy (1977) proposed that feedback’s chief instructional significance is to correct errors. This error-correcting action was thought to be a function of presentation timing, response certainty, and whether students could merely copy answers from feedback without having to generate their own. The present meta-analysis reviewed 58 effect sizes from 40 reports. Feedback effects were found to vary with control for presearch availability, type of feedback, use of pretests, and type of instruction and could be quite large under optimal conditions. Mediated intentional feedback for retrieval and application of specific knowledge appears to stimulate the correction of erroneous responses in situations where its mindful (Salomon & Glo...

...read moreread less

1,248 citations

Journal Article•DOI•

Content analysis of online discussion in an applied educational psychology course

[...]

Noriko Hara¹, Curtis J. Bonk¹, Charoula Angeli²•Institutions (2)

Indiana University¹, University of Pittsburgh²

01 Mar 2000-Instructional Science

TL;DR: The authors analyzed discussion in an online conferencethat supplemented class discussion using an instructional method called the starter-wrappertechnique within a traditional graduate leveleducational psychology course and found that students were using high level cognitive skills such as inferencing and judgment as well as metacognitive strategies related to reflectingon experience and self-awareness.

...read moreread less

Abstract: This study analyzed discussion in an online conferencethat supplemented class discussion using aninstructional method called the starter-wrappertechnique within a traditional graduate leveleducational psychology course. Various quantitativemeasures were recorded to compare instructor andstudent participation rates. In addition, Henri's(1992) model for content analysis of computer-mediatedcommunication was employed to qualitatively analyzethe electronic discourse. Using this model, five keyvariables were examined: (1) student participationrates; (2) electronic interaction patterns; (3) socialcues within student messages; (4) cognitive andmetacognitive components of student messages; and (5)depth of processing -- surface or deep -- within messageposting. Transcript content analyses showed that,while students tended to post just the one requiredcomment per week in the conference, their messageswere lengthy, cognitively deep, embedded with peerreferences, and indicative of a student orientedenvironment. Moreover, students were using high levelcognitive skills such as inferencing and judgment aswell as metacognitive strategies related to reflectingon experience and self-awareness. Weekly conferenceactivity graphs revealed that student electroniccomments became more interactive over time, but werehighly dependent on the directions of discussionstarter. To better understand the impact ofelectronic conferencing discourse, modifications toHenri's model as well as qualitative researchsuggestions were offered.

...read moreread less

993 citations

"Web-based peer assessment : feedbac..." refers methods in this paper

...This deserves a closer examination with qualitative methods, such as content analysis for computer mediated communication (Henri, 1992; Hara et al., 2000)....
[...]