Instructors often incorporate self- and peer evaluations when they use teamwork in their classes, which is common in management education. However, the process is often time consuming and frequentl...

The Comprehensive Assessment of Team Member Effectiveness: Development of a Behaviorally Anchored Rating Scale for Self- and Peer Evaluation

In spite of considerable research about the poor retention rate ofundergraduate engineering students, we still have an inadequateunderstanding of the factors that affect students’ decisions toremain in engineering programs and their ability to perform wellenough to be retained. Although continued study is needed ofexternal factors such as curricular requirements, admissions crite-ria, and test scores, we also need to know much more about therelationships between curricular experiences and students’ learningstyles, habits, and attitudes. The work presented in this paper wasdesigned to enhance educators’ understanding of the factors thatunderlie the concern about student retention in engineering. Byobserving 1,000 engineering students during their first three yearsin college, the research team generated a large database on the stu-dents’ academic and non-academic characteristics as well as theirsuccesses and failures. The traits discovered not only support manyfindings from previous studies but also reveal some new relation-ships that could prove essential to designing an educational envi-ronment that will prepare engineers for success in the future.Keywords: retention, gender, underrepresented minorities

Understanding Our Students: A Longitudinal- Study of Success and Failure in Engineering With Implications for Increased Retention

Background
Empirical evidence suggests that students studying cooperatively exhibit significantly better academic achievement. However, since most prior studies do not carefully monitor the time on task, it is unclear whether the observed learning benefits are due to the intrinsic superiority of cooperative learning or merely a reflection of the increased amount of time students spend on studying.

Purpose (Hypothesis)
This study compares the learning effectiveness of cooperative and individualistic learning. The proposed approach carefully monitors the learning method and the time on task both in regular day-time teaching classes and out-of-class studies.

Design/Method
A series of experiments was performed in which 42 mechanical engineering students were randomly assigned to individualistic or cooperative learning conditions, respectively, and were then formed into heterogeneous groups comprising three team members. The experiments were conducted over an 18-week semester. In conducting the experiments, the students attended both regular classes and out-of-hours homework sessions.

Results
The experimental results showed that given a sufficient period of time for the cooperative learning teams to mature, the students in the cooperative learning condition performed substantially better in both the homework and unit tests than those in the individualistic learning condition.

Conclusions
Since the time on task was carefully monitored, the higher academic performance of the students in the cooperative learning condition suggests that cooperative learning is more effective than individualistic learning.

The Effectiveness of Cooperative Learning

A prerequisite relation describes a basic relation among concepts in cognition, education and other areas. However, as a semantic relation, it has not been well studied in computational linguistics. We investigate the problem of measuring prerequisite relations among concepts and propose a simple link-based metric, namely reference distance (RefD), that effectively models the relation by measuring how differently two concepts refer to each other. Evaluations on two datasets that include seven domains show that our single metric based method outperforms existing supervised learning based methods.

/pdf/measuring-prerequisite-relations-among-concepts-2ss2aglipb.pdf

Measuring Prerequisite Relations Among Concepts

Background



In engineering education, a considerable amount of research effort has been dedicated to study the impacts of student characteristics on their college enrollment, major selection, and college retention. However, there is no standardized categorical classification system of engineering student characteristics in the current literature. Different researchers tend to focus on specific characteristics within the scope of their research interests. This study provides a comprehensive review and analysis of the existing research on the measurement of the characteristics of engineering students.



Purpose



The study addressed the three questions: (1) what engineering student characteristics have been measured; (2) how do engineering student characteristics impact their educational outcomes; and (3) what measurement and analysis methods have been applied in current studies? A standardized classification system for engineering student characteristics involving external, cognitive, affective, and demographic categories is also proposed.



Scope/Method



The study focused on engineering education. Representative research regarding common characteristics of students from majors of science, technology, engineering, and mathematics were also included. The review covers major academic journals, research books, conference proceedings, and government reports in the areas of science and engineering education for the past two decades.



Conclusions



The review analysis indicated that students with certain characteristics are more likely to choose engineering as a profession and that those characteristics are either correlated or causally related with one another. However, many research conclusions based on basic statistical analyses fail to model the interaction effects. More advanced measurement techniques are needed that can model the characteristics interactively and concurrently in a complete framework.

Development of a Classification System for Engineering Student Characteristics Affecting College Enrollment and Retention

This paper presents comparisons of three peer evaluation instruments tested among students in undergraduate engineering classes: a single-item instrument without behavioral anchors, a ten-item instrument, and a single-item behaviorally anchored instrument. Studies using the instruments in undergraduate engineering classes over four years show that the use of behavioral anchors significantly improves the inter-rater reliability of the single-item instrument. The inter-rater reliability (based on four raters) of the behaviorally anchored instrument was 0.78, which was not significantly higher than that of the ten-item instrument (0.74), but it was substantially more parsimonious. The results of this study add to the body of knowledge on evaluating students' performance in teams. This is critical since the ability to function in multidisciplinary teams is a required student learning outcome of engineering programs.

Effects of Behavioral Anchors on Peer Evaluation Reliability

A review of prerequisites often reveals that reasons for requiring a prerequisite may no longer prevail due to curriculum or course changes. Based on a study of a curriculum bottleneck unrelated to required mastery, the prerequisite structure in Clemson University's General Engineering curriculum (the common first-year curriculum for all engineering students) was changed so that Calculus I could be taken in the second semester. Student record analysis shows both the magnitude of the bottleneck prior to the policy change and the effect on student enrollment practices after the policy change. Longitudinal studies show a statistically significant improvement in retention in engineering adding to the body of evidence that indicates that it is important to retention that students start college mathematics at a level for which they are prepared.

Amy G. Yuhasz

Papers

Effects of Behavioral Anchors on Peer Evaluation Reliability

Identifying and Removing a Calculus Prerequisite as a Bottleneck in Clemson's General Engineering Curriculum

The Use Of Sensors In The Engineering Classroom: Design Of An Educational Experiment