Showing papers in "Journal of Engineering Education in 2007"
TL;DR: Previous research on engineering student design processes is extended to compare the design behavior of students and expert engineers to support the argument that problem scoping and information gathering are major differences between advanced engineers and students, and important competencies for engineering students to develop.
Abstract: In this paper we report on an in-depth study of engineering design processes. Specifically, we extend our previous research on engineering student design processes to compare the design behavior of students and expert engineers. Nineteen experts from a variety of engineering disciplines and industries each designed a playground in a lab setting, and gave verbal reports of their thoughts during the design task. Measures of their design processes and solution quality were compared to pre-existing data from 26 freshmen and 24 seniors. The experts spent significantly more time on the task overall and in each stage of engineering design, including significantly more time problem scoping. The experts also gathered significantly more information covering more categories. Results support the argument that problem scoping and information gathering are major differences between advanced engineers and students, and important competencies for engineering students to develop. Timeline representations of the expert designers' processes illustrate characteristic distinctions we found and may help students gain insights into their own design processes.
701 citations
TL;DR: In this paper, a study was conducted on the Felder-Soloman Index of Learning Styles (ILS) to assess reliability, factor structure, and construct validity as well to determine whether changing its dichotomous response scale to a five-option response scale would improve reliability and validity.
Abstract: A study was conducted on the Felder-Soloman Index of Learning Styles© (ILS) to assess reliability, factor structure, and construct validity as well to determine whether changing its dichotomous response scale to a five-option response scale would improve reliability and validity. Data collected in this study had internal consistency reliability ranging from 0.55 to 0.77 across the four learning style scales of the ILS. Factor analysis revealed that multiple factors were present within three of the learning style scales, which correspond to known aspects of the scale definitions. The factor analysis and direct feedback from students on whether they felt their scores accurately represented their learning preferences provide evidence of construct validity for the ILS. Changing the response scale improved reliability, but it did not change the factor structure substantially nor did it affect the strength of the evidence for construct validity based on student feedback.
344 citations
TL;DR: Inez as mentioned in this paper is a female, multi-minority, and from a socio-economically disadvantaged background, and her story provides a poignant example of the impact of five of Conefrey's cultural myths of science.
Abstract: Engineers need a breadth of experience to enrich the gene pool of ideas from which elegant engineering solutions can be drawn, called “individual diversity.” While performing large ethnographic research studies where hundreds of engineering students were interviewed, we interviewed Inez, a student that epitomizes individual diversity. Inez is unlike most engineers: she is female, multi-minority, and from a socio-economically disadvantaged background. Inez’s story is told here using “ethnography of the particular,” where the story of a single individual is explored. Inez has persevered through challenges posed by her lack of familiarity with the culture of engineering, her weak high school preparation, and her feelings of being an outsider in engineering. Inez’s story demonstrates that the playing field in engineering is still not level, particularly for socio-economically disadvantaged students. Her story provides a poignant example of the impact of five of Conefrey’s cultural myths of science.
309 citations
TL;DR: An empirical ethnographic survey of engineers using interviews and field observations in Australia provides evidence that coordinating technical work of other people by gaining their willing cooperation is a major aspect of engineering practice as mentioned in this paper.
Abstract: An empirical ethnographic survey of engineers using interviews and field observations in Australia provides evidence that coordinating technical work of other people by gaining their willing cooperation is a major aspect of engineering practice. Technical coordination in the context of this study means working with and influencing other people so they conscientiously perform necessary work to a mutually agreed schedule. While coordination seems to be non-technical, analysis provides evidence supporting the critical importance of technical expertise. Coordination usually involves one-on-one relationships with superiors, clients, peers, subordinates, and outsiders. Coordinating the work of other people seems to be important from the start of an engineering career.
Engineering education only provides limited informal coordination skill development and current accreditation criteria may not reflect this aspect of engineering. This paper suggests ways in which students can learn coordination, and describes some of the author's experiences in applying this research.
188 citations
TL;DR: In this paper, a large-scale study of 1,000 engineering students during their first three years in college was conducted to understand the factors that affect student retention in engineering programs and their ability to perform well enough to be retained.
Abstract: In spite of considerable research about the poor retention rate ofundergraduate engineering students, we still have an inadequateunderstanding of the factors that affect students’ decisions toremain in engineering programs and their ability to perform wellenough to be retained. Although continued study is needed ofexternal factors such as curricular requirements, admissions crite-ria, and test scores, we also need to know much more about therelationships between curricular experiences and students’ learningstyles, habits, and attitudes. The work presented in this paper wasdesigned to enhance educators’ understanding of the factors thatunderlie the concern about student retention in engineering. Byobserving 1,000 engineering students during their first three yearsin college, the research team generated a large database on the stu-dents’ academic and non-academic characteristics as well as theirsuccesses and failures. The traits discovered not only support manyfindings from previous studies but also reveal some new relation-ships that could prove essential to designing an educational envi-ronment that will prepare engineers for success in the future.Keywords: retention, gender, underrepresented minorities
187 citations
TL;DR: In this article, a combination of publication analysis and faculty interviews was employed to study four NSF-sponsored engineering education coalitions as a case study of the recent history of engineering education.
Abstract: A combination of publication analysis and faculty interviews was employed to study four NSF-sponsored engineering education coalitions as a case study of the recent history of engineering education. Current calls within the engineering education community for increased rigor can be understood in terms of the ways similar disciplines have emerged. In science education, for example, time was needed to develop consensus on important research questions, accepted methods, and standards of rigor. The
166 citations
TL;DR: In this paper, the authors describe conceptual difficulties that may be experienced by engineering faculty as they become engineering education researchers, including framing research questions with broad appeal, grounding research in a theoretical framework, considering operationalization and measurement of constructs, appreciating qualitative or mixed-methods approaches, and pursuing interdisciplinary collaboration.
Abstract: This paper describes conceptual difficulties that may be experienced by engineering faculty as they become engineering education researchers. Observation, survey, and assessment data collected at the 2005 NSF-funded Rigorous Research in Engineering Education workshop were systematically analyzed to uncover the five difficulties encountered by engineering faculty learning to design rigorous education studies: (1) framing research questions with broad appeal, (2) grounding research in a theoretical framework, (3) fully considering operationalization and measurement of constructs, (4) appreciating qualitative or mixed-methods approaches, and (5) pursuing interdisciplinary collaboration. The first four can be understood in terms of disciplinary consensus; they represent explicit steps in education research that are implicit in technical engineering research because there is greater consensus of methods and standards. This work better frames the issue of rigor in engineering education research by clarifying the fundamental differences that prevent application of traditional engineering standards of rigor directly to engineering education research.
165 citations
TL;DR: This article found that women were more likely to be context-oriented than men during problem-scoping, and that context-orientation was positively correlated between the two design tasks, despite differences in data collection and analysis.
Abstract: High-quality engineering design requires an understanding of how the resulting engineered artifact interacts with society, the natural environment, and other aspects of context. This study examines how first-year engineering undergraduates approached two engineering design tasks. We focused on how much students considered contextual factors during problem-scoping, a critical part of the design process. As part of a larger, longitudinal study, we collected data from 160 students at four U.S. institutions. Students varied in their consideration of each design task's context, and women's responses were more likely to be context-oriented than men's. Overall, context-orientation was positively correlated between the two design tasks, despite differences in data collection and analysis. Having found that beginning engineering students, particularly women, are sensitive to important contextual factors, we suggest that efforts to broaden participation in engineering should consider legitimizing and fostering context-oriented approaches to engineering earlier in the curriculum.
130 citations
TL;DR: This article examined demographic, academic, attitudinal, and experiential data from the Cooperative Institutional Research Program (CIRP) for over 12,000 students at two universities to test a methodology for identifying variables showing significant differences between students intending to major in science, technology, engineering, or mathematics (STEM) versus non-STEM subjects.
Abstract: This research examines demographic, academic, attitudinal, and experiential data from the Cooperative Institutional Research Program (CIRP) for over 12,000 students at two universities to test a methodology for identifying variables showing significant differences between students intending to major in science, technology, engineering, or mathematics (STEM) versus non-STEM subjects. The methodology utilizes basic statistical techniques to identify significant differences between STEM and non-STEM students within seven population subgroups based upon school attended, race/ethnicity, and gender. The value of individual variables is assessed by how consistently significant differences are found across the subgroups.
The variables found to be most valuable in identifying STEM students reflect both quantitative and qualitative measures. Quantitative measures of academic ability such as SAT mathematics score, high school grade point average, and to a lesser extent SAT verbal score are all indicators. Qualitative measures including self-ratings of mathematical ability, computer skills, and academic ability are also good indicators.
105 citations
TL;DR: In this paper, an instructional approach aimed at improving the writing skills of a group of undergraduate engineering students was presented. But the approach was only designed for a single writing exercise, which was based on recommendations for best practices from the learning science community.
Abstract: This paper summarizes the design and evaluation of an instructional approach aimed at improving the writing skills of a group of undergraduate engineering students We sought to determine whether student performance in difficult writing skills such as argumentation and synthesis could be improved by integrating a single writing exercise into an upper level engineering course In designing the exercise, we relied heavily on recommendations for best practices from the learning science community, specifically those codified in the National Academy text How People Learn [1] We found reliable improvement in student performance in many of the areas targeted, demonstrating that the approach taken was effective Since we modified the exercise a few times before meeting our objectives for student learning, we could compare the effectiveness of different implementations of our approach Our success and failures provide guidance for others seeking to improve the competence of engineering undergraduates in writing
97 citations
TL;DR: The National Engineering Education Research Colloquies (NEERC) as mentioned in this paper explored how rapid changes in the world require new ways to educate future generations of engineers, and the argument is not simply that we need more engineers than we have had in the past; instead, we need a transformation in how we educate future engineers.
Abstract: The October 2006 issue of the Journal of Engineering Education describes a landmark set of activities called the National Engineering Education Research Colloquies (NEERC) that explored how rapid changes in the world require new ways to educate future generations of engineers. The argument is not simply that we need more engineers than we have had in the past; instead we need a transformation in how we educate future engineers [1]. Many people in the learning sciences have also been exploring the need for educational transformations, and it seems clear that our different research communities have a great deal to learn from one another. Opportunities I have had to work with the VaNTH Bioengineering Center (VaNTH.org; [2]) convince me of the immense value (to me at least) of collaborations such as these. Currently, my colleagues and I in the LIFE Center (Learning in Informal and Formal Environments (LIFE-SLC.org) are finding it useful to juxtapose several research literatures that include: (a) expertise and its development [3–5]; (b) transfer and its implications for assessment [6–7]; (c) change and innovation [8–11]; and (d) design strategies for promoting and managing change [12–14]. The act of juxtaposing these different literatures has generated a number of interesting questions. I discuss five that I hope are useful to raise.
TL;DR: A “lesson study” approach was taken, using constructivist educational theory combined with the variation theory of learning from phenomenography to inform the design of learning activities and to assess their impact.
Abstract: This paper reports the results of a study into the impact of computer simulations on the understanding of fluid mechanics by engineering students. A “lesson study” approach was taken, using constructivist educational theory combined with the variation theory of learning from phenomenography to inform the design of learning activities and to assess their impact. Student difficulties with fluid mechanics concepts were assessed using questions from the Fluid Mechanics Concept Inventory (FMCI). Students had the greatest difficulties with pressure measurement, fluid flow through pipes with changing diameter, and velocity profiles for fluid between flat plates. We developed a set of three simulations to address these difficulties. The impact of the simulations was gauged by a second administration of the FMCI. Most of the students in the sophomore fluid mechanics class participated in the whole of this exercise. Students showed significant improvement in two of the three areas of difficulty. Student feedback on this as an additional learning exercise was very positive.
TL;DR: In this article, the authors suggest new practices for administering and analyzing the results of concept inventories, and the value of administering inventories as pre-tests is examined, as well as the potential for offering misconception diagnosis based on inventories.
Abstract: Concept inventories can be refined and honed into informative assessment tools to serve instruction. The present paper suggests new practices for administering and analyzing the results of concept inventories. Web-based administration enables broader participation across universities and colleges, and ensures the retention of the full set of data necessary to conduct other analyses. Issues related to the provision of meaningful concept level information are addressed, as are the benefits of making direct comparisons with other measures of performance. The value of administering inventories as pre-tests is examined, and the potential for offering misconception diagnosis based on inventories is explored.
TL;DR: In this paper, a new analogical model, which emphasizes flows of personal and interpersonal energy within the educational system, is offered to guide future interventions, and suggestions for applying the model to increase diversity in engineering are proposed.
Abstract: Existing approaches to increasing diversity conceptually pose the problem as a leaky pipeline. Although the pipeline model has supported several types of interventions, the mental model oversimplifies complexities of the underlying processes, focuses interventions at points of unwanted leakage, and suggests that leaks need to be plugged instead of systems renewed. Analysis suggests that even if leak-stopping interventions could be multiplied through significant increases in funding, they would remain insufficient to attain the goal of having student enrollments in engineering substantially reflect the demographics of the general population. Therefore, a new analogical model, which emphasizes flows of personal and interpersonal energy within the educational system, is offered to guide future interventions. Finally, suggestions for applying the model to increase diversity in engineering are proposed.
TL;DR: In this paper, the authors describe their interest in engineering education and engineering in the K-12 system began when they started to collaborate with engineers in a center housed in the College of Engineering on my campus.
Abstract: My interest in engineering education and engineering in the K-12 system began when I started to collaborate with engineers in a center housed in the College of Engineering on my campus. We were writing grants and thinking about ways to get engineering practices into the K-12 classroom.
TL;DR: Students expressed significantly more cognitive activity on computer screens requiring interaction compared to text-based screens and verbalizations revealed that students applied predominantly lower-level cognitive processes when engaging these materials, and they failed to connect the conceptual and procedural knowledge in ways that would lead to deeper understanding.
Abstract: The development of procedural knowledge in students, i.e., the ability to effectively solve domain problems, is the goal of many instructional initiatives in engineering education. The present study examined learning in a rich learning environment in which students read text, listened to narrations, interacted with simulations, and solved problems using instructional software for thermodynamics. Twenty-three engineering and science majors who had not taken a thermodynamics course provided verbal protocol data as they used this software. The data were analyzed for cognitive processes. There were three major findings: (1) students expressed significantly more cognitive activity on computer screens requiring interaction compared to text-based screens; (2) there were striking individual differences in the extent to which students employed the materials; and (3) verbalizations revealed that students applied predominantly lower-level cognitive processes when engaging these materials, and they failed to connect the conceptual and procedural knowledge in ways that would lead to deeper understanding. The results provide a baseline for additional studies of more advanced students in order to gain insight into how students develop skill in engineering.
TL;DR: In the last four years, a series of outreach programs have been offered through the Colorado School of Mines to middle school teachers from eleven school districts in the State of Colorado in the United States to illustrate through hands-on activities the application of mathematics to science and engineering.
Abstract: Over the last four years, a series of outreach programs have been offered through the Colorado School of Mines to middle school teachers from eleven school districts in the State of Colorado in the United States. Each of these programs is designed to illustrate through hands-on activities the application of mathematics to science and engineering. Each also has an academic year follow-up such that a faculty member, an expert teacher, or a graduate student assists the teachers in the classroom. An expected outcome of this effort is the improvement of instruction in mathematics and science in the participating middle schools; an unexpected outcome has been the impact of these projects on the culture of the participating schools, both middle schools and university. Based on our assessment efforts, this article describes the qualitative and quantitative outcomes of this sequence of projects on middle school students, teachers, graduate students, professors, and college curriculum.
TL;DR: This paper analyzed the views and perceptions of engineering undergraduate students on engineering education and found that students strongly emphasized the importance of their own roles in the educational system and the value of instructional technology and real work examples in enhancing the quality of engineering education.
Abstract: The purpose of this study was to understand the views and perceptions of engineering undergraduate students on engineering education. The method of content analysis was used to analyze the language used by engineering undergraduate students, and to extract the underlying common factors or perceived characteristics of “Excellence in Engineering Education.” These common factors were then used to identify and compare the similarities and differences in views between engineering students and perspectives from three types of stakeholders in the field. Forty-seven undergraduate engineering students (17 females and 30 males) participated voluntarily in this study to answer four individual questions and ten group questions. The results showed that students strongly emphasized the importance of their own roles in the educational system and the value of instructional technology and real work examples in enhancing the quality of engineering education. The implications of the research results on excellence in engineering education are discussed.
TL;DR: In this article, the authors extend previous work on undergraduate learning in engineering to provide further validation for an assessment paradigm capable of quantifying engineering students' conceptual and problem-solving knowledge.
Abstract: Conceptual and procedural knowledge are two mutually-supportive factors associated with the development of engineering skill. The present study extends previous work on undergraduate learning in engineering to provide further validation for an assessment paradigm capable of quantifying engineering students' conceptual and problem-solving knowledge. Eight students who were enrolled in an introductory thermodynamics course and four who were enrolled in the course sequel provided verbal protocol data as they used instructional software. They were compared to existing data from a cohort of eleven science and engineering majors who had not taken thermodynamics. The results replicated earlier findings showing more cognitive activity on computer screens requiring overt user interaction compared to text-based screens. The data also indicated that higher- versus lower-performing students, based on course grades, engaged in more higher-order cognitive processing. There was no evidence that students gained deeper cognitive processing as they advanced through the engineering curriculum.
TL;DR: This paper collected narrative accounts of teaching consultations between engineering educators and an instructional consultant and coded the transcripts of these accounts for individual teaching concerns, which were then interpreted from the perspective of existing models and also aggregated into themes.
Abstract: The teaching concerns of engineering educators offer one lens for thinking about how to support engineering educators' efforts to improve their teaching. In this study, we collected narrative accounts of teaching consultations between engineering educators and an instructional consultant. Transcripts of these accounts were coded for individual teaching concerns, which were then interpreted from the perspective of existing models and also aggregated into themes. We discuss our findings by using them to highlight ways in which engineering educators are already thinking effectively, to suggest how the adoption of innovation and professional problem-solving can serve as promising frameworks for thinking about teaching activity, and to suggest that additional research on engineering teaching take advantage of distributed cognition models to truly understand how our students are taught.
TL;DR: The Humanitarian Engineering initiative, sponsored by the William and Flora Hewlett Foundation, at the Colorado School of Mines, is creating a program that will support engineering students in understanding their responsibility for solving community development problems that exist throughout the world as mentioned in this paper.
Abstract: The Humanitarian Engineering initiative, sponsored by the William and Flora Hewlett Foundation, at the Colorado School of Mines, is creating a program that will support engineering students in understanding their responsibility for solving community development problems that exist throughout the world. As part of this effort, data has been collected on faculty and student attitudes using the “Community Service Attitudes Scale,” developed and validated by Shiarella, McCarthy, and Tucker. During the fall 2004, 78 students and 34 faculty members responded to this instrument. Statistically significant differences were found between the attitudes of students and faculty, males and females, and among different age groupings with respect to service activities. A general finding was that faculty displayed better attitudes toward community service than the students.
TL;DR: In this paper, the authors examined the effect of the pace of transitioning from worked examples to independent problem solving for students with three different levels of prior knowledge: immediate transitioning, fast fading, and slow fading.
Abstract: This study examined the effect of the pace of transitioning from worked examples to independent problem solving for students with three different levels of prior knowledge. Three paces of transitioning were examined: immediate transitioning, fast fading, and slow fading. The study was conducted with engineering college freshmen in the engineering knowledge domain of introductory electrical circuit analysis and found a significant interaction between the particpants' prior knowledge and the pace of transitioning to independent problem solving on retention posttest performance. The high prior knowledge participants achieved significantly higher retention scores under the fast and immediate transitioning than under the slow transitioning, whereas the low prior knowledge participants achieved significantly higher retention scores under the slow transitioning. The interaction result for retention indicates that by selectively employing slow fading for low prior knowledge learners and fast fading or immediate transitioning for high prior knowledge learners, significant improvements in learning may be achieved.
TL;DR: This paper found that the School of Industrial Engineering at the University of Oklahoma had organically achieved parity of the sexes at the undergraduate level, and they adopted an ethnographic perspective, interviewing 185 students who represented four fields and four institutions as well as 12 faculty in industrial engineering.
Abstract: Most research about women in engineering focuses on reasons for their under-representation. In contrast, we capitalized on an opportunity to study success: the School of Industrial Engineering at the University of Oklahoma had organically achieved parity of the sexes at the undergraduate level. To investigate this success, we adopted an ethnographic perspective, interviewing 185 students who represented four fields and four institutions as well as 12 faculty in Industrial Engineering at the University of Oklahoma. These data pointed to a combination of aspects of the discipline and the department culture as explanatory variables. Emerging from the data was a third explanatory variable: a high number of students, disproportionately many women, who relocated into Industrial Engineering from another major, underscoring the impact of broad recruiting activities. This paper emphasizes ideas that other departments can consider adapting to their own efforts to increase diversity.
TL;DR: This paper found that student ratings of workload and overall instructor performance in engineering courses were not correlated (e.g., Spearman's rho � 0.7 to 0.068) in data sets from either of two institutions.
Abstract: Many engineering faculty believe that when students perceive a course to have a high workload, students will rate the course and the performance of the course instructor poorly. This belief can be particularly worrying to engineering faculty since engineering courses are often perceived as uniquely demanding. The present investigation demonstrated that student ratings of workload and of overall instructor performance in engineering courses were not correlated (e.g., Spearman’s rho � 0.068) in data sets from either of two institutions. In contrast, a number of evaluation items were strongly correlated (Spearman’s rho � 0.7 to 0.899) with ratings of overall instructor performance across engineering, mathematics and science, and humanities courses. The results of the present study provide motivation for faculty seeking to improve their teaching and course evaluations to focus on teaching methods, organization/preparation, and interactions with students, rather than course workload.
TL;DR: This article reviewed the literature on the current state of the research-teaching nexus and then examined three specific strategies for integrating teaching and scholarship: bringing research into the classroom, involving undergraduates in research projects, and broadening the definition of scholarship beyond frontier disciplinary research.
Abstract: Academicians have been arguing for decades about whether or not faculty research supports undergraduate instruction Those who say it does—a group that includes most administrators and faculty members—cite many ways in which research can enrich teaching, while those on the other side cite numerous studies that have consistently failed to show a measurable linkage between the two activities This article proposes that the two sides are debating different propositions: whether research can support teaching in principle and whether it has been shown to do so in practice The article reviews the literature on the current state of the research-teaching nexus and then examines three specific strategies for integrating teaching and scholarship: bringing research into the classroom, involving undergraduates in research projects, and broadening the definition of scholarship beyond frontier disciplinary research Finally, ways are suggested to better realize the potential synergies between faculty research and undergraduate education
TL;DR: In this article, the authors present the design and preliminary analysis of an experimental forum to facilitate cross-disciplinary discourse within a NSF-sponsored Integrative Graduate Education and Research Traineeship Program (IGERT) at Rutgers.
Abstract: A major challenge for fostering integrative cross-disciplinary collaborations at the graduate level arises from the divergent exposure and training of students from uni-disciplinary graduate programs. In this report, we present the design and preliminary analysis of an experimental forum to facilitate cross-disciplinary discourse within a NSF-sponsored Integrative Graduate Education and Research Traineeship Program (IGERT) at Rutgers. This forum brings together IGERT Graduate Training Fellows and faculty from four diverse graduate programs in the engineering area and four related programs in life sciences and physical sciences for structured seminars and interchanges. Our report offers methodological and analytical tools grounded within a conceptual framework for promoting discourse that integrates content across diverse disciplines as well as across levels of inquiry. Both the theoretical framework and the research tools may be valuable to others seeking to develop integrative training environments for coalescing learning communities between engineers and their collaborators.
TL;DR: In this article, the authors extend work sampling, an economic industry-based alter-native, to observe cognitive and behavioral processes and determine attributes of team-work, establish target time proportions using 100 percent observation, and statistically compare the targets to proportions obtained from work sampling intervals to determine the effective interval.
Abstract: Engineering programs must assess students’ abilities to master “crite-ria 3 a-k.” Skills such as teamwork, problem solving, design, and ethi-cal understanding entail learning various processes; hence, assessingthese outcomes is better accomplished by focusing on the processrather than the result. Methods for observing students’ performance,such as 100 percent behavioral observation, are ideal but expensive.We extend work sampling, an economic industry-based alter-native, to observe cognitive and behavioral processes. Specifically,we describe a work sampling methodology to assess studentsengaged in teamwork. We then determine attributes of team-work, establish target time proportions using 100 percent obser-vation, and statistically compare the targets to proportionsobtained from work sampling intervals to determine the effectiveinterval. The robustness of work sampling is tested in four learn-ing environments. Results indicate that sampling provides a sta-tistically valid alternative for assessing teamwork. However, whenobserving design and ethical understanding processes, additionalresearch is needed to make work sampling viable.Keywords: behavioral observation, teamwork, work sampling
TL;DR: This paper describes how an interdisciplinary team used conceptual graphs to formally specify the model for a good essay response, and then how that model was used as the standard by which the student responses were judged.
Abstract: Automatically grading essay questions can have advantages for instructors in higher education Understanding and specifying how grading is done manually, so that there is potential to do it automatically, is a labor-intensive effort in knowledge elicitation, acquisition, and representation This paper describes how an interdisciplinary team used conceptual graphs to formally specify the model for a good essay response, and then how that expert model was used as the standard by which the student responses were judged The methodology is then described for creating the expert model for student responses These were compared using two different approaches It was found that most students included the most important concepts, but those student answers that were more complete (ie, also including concepts of lesser importance) received higher grades The approaches are then evaluated in terms of reliability and validity, and finally, suggestions are made for future work