to many, however, it constantly appears as an instructional problem. The final two chapters serve as excellent summary statements. Chapter 13 emphasizes the application of previous text material to the classroom situation. Of great significance is the author's discussion of commonly defined independent, dependent, and moderator variables. The final chapter focuses on evaluation in the overall sense, particularly as it relates to programs of study. The principles of measurement and research design are applied to the evaluation process through several ex-

Educational Research: Competencies for Analysis and Application:

creased by 0.47 SDs under active learning (n = 158 studies), and that the odds ratio for failing was 1.95 under traditional lecturing (n = 67 studies). These results indicate that average examination scores improved by about 6% in active learning sections, and that students in classes with traditional lecturing were 1.5 times more likely to fail than were students in classes with active learning. Heterogeneity analyses indicated that both results hold across the STEM disciplines, that active learning increases scores on concept inventories more than on course examinations, and that active learning appears effective across all class sizes—although the greatest effects are in small (n ≤ 50) classes. Trim and fill analyses and fail-safe n calculations suggest that the results are not due to publication bias. The results also appear robust to variation in the methodological rigor of the included studies, based on the quality of controls over student quality and instructor identity. This is the largest and most comprehensive metaanalysis of undergraduate STEM education published to date. The results raise questions about the continued use of traditional lecturing as a control in research studies, and support active learning as the preferred, empirically validated teaching practice in regular classrooms.

/pdf/active-learning-increases-student-performance-in-science-3j0c6wu4k5.pdf

Active learning increases student performance in science, engineering, and mathematics

Students have different levels of motivation, different attitudes about teaching and learning, and different responses to specific classroom environments and instructional practices. The more thoroughly instructors understand the differences, the better chance they have of meeting the diverse learning needs of all of their students. Three categories of diversity that have been shown to have important implications for teaching and learning are differences in students' learning styles (characteristic ways of taking in and processing information), approaches to learning (surface, deep, and strategic), and intellectual development levels (attitudes about the nature of knowledge and how it should be acquired and evaluated). This article reviews models that have been developed for each of these categories, outlines their pedagogical implications, and suggests areas for further study.

Understanding Student Differences

Transforming Qualitative Data: Description, Analysis, and Interpretation

In developing its new engineering accreditation criteria, ABET reaffirmed a set of “hard” engineering skills while introducing a second, equally important, set of six “professional” skills. These latter skills include communication, teamwork, and understanding ethics and professionalism, which we label process skills, and engineering within a global and societal context, lifelong learning, and a knowledge of contemporary issues, which we designate as awareness skills. We review these skills with an emphasis on how they can be taught, or more correctly learned, citing a number of examples of successful and/or promising implementations. We then examine the difficult issue of assessing these skills. We are very positive about a number of creative ways that these skills are being learned, particularly at institutions that are turning to global and/or service learning in combination with engineering design projects to teach and reinforce outcome combinations. We are also encouraged by work directed at assessing these skills, but recognize that there is considerable research that remains to be done.

/pdf/the-abet-professional-skills-can-they-be-taught-can-they-be-5b96nd5h5f.pdf

The ABET “Professional Skills” — Can They Be Taught? Can They Be Assessed?

In Moskal (2000), a framework for developing scoring rubrics was presented and the issues of validity and reliability were given cursory attention. Although many teachers have been exposed to the statistical definitions of the terms "validity" and "reliability" in teacher preparation courses, these courses often do not discuss how these concepts are related to classroom practices (Stiggins, 1999). One purpose of this article is to provide clear definitions of the terms "validity" and "reliability" and illustrate these definitions through examples. A second purpose is to clarify how these issues may be addressed in the development of scoring rubrics. Scoring rubrics are descriptive scoring schemes that are developed by teachers or other evaluators to guide the analysis of the products and/or processes of students' efforts (Brookhart, 1999; Moskal, 2000). The ideas presented here are applicable for anyone using scoring rubrics in the classroom, regardless of the discipline or grade level.

/pdf/scoring-rubric-development-validity-and-reliability-2caq2y410b.pdf

Scoring Rubric Development: Validity and Reliability.

Scoring rubrics have become a common method for evaluating student work in both the K-12 and the college classrooms. The purpose of this paper is to describe the different types of scoring rubrics, explain why scoring rubrics are useful and provide a process for developing scoring rubrics. This paper concludes with a description of resources that contain examples of the different types of scoring rubrics and further guidance in the development process.

/pdf/scoring-rubrics-what-when-and-how-4jp599hy7g.pdf

Scoring Rubrics: What, When and How?

This paper offers and tests an approach to conceptualizing the global competency of engineers. It begins by showing that the often-stated goal of working effectively with different cultures is fundamentally about learning to work effectively with people who define problems differently. The paper offers a minimum learning criterion for global competency and three learning outcomes whose achievement can help engineering students fulfill that criterion. It uses the criterion to establish a typology of established methods to support global learning for engineering students. It introduces the course, Engineering Cultures, as an example of an integrated classroom experience designed to enable larger numbers of engineering students to take the critical first step toward global competency, and it offers a test application of the learning criterion and outcomes by using them to organize summative assessments of student learning in the course.

/pdf/the-globally-competent-engineer-working-effectively-with-44e6i4zjo7.pdf

The Globally Competent Engineer: Working Effectively with People Who Define Problems Differently

This article examines the current state of assessment in engineering education in the United States as reflected in the Journal of Engineering Education. We begin with a brief review of recent developments in the assessment of engineering education and the events that have inspired change. Next, we explore assessment methodologies that have been used repeatedly in the evaluation of engineering courses, curricula, and research investigations as well as some methods that have not been used extensively but are likely to be informative. We conclude with a discussion of the importance of establishing collaborations between researchers in engineering education and educational research. Throughout this paper we highlight examples of sound and rigorous assessments in engineering education.

Assessment in Engineering Education: Evolution, Approaches and Future Collaborations

This paper describes the evaluation of an NSF-sponsored educational research project. The primary focus of this project was to develop and evaluate a course curriculum designed to improve retention and performance for "at risk" introductory computer science majors. The results of this research suggest that the newly developed course and curriculum materials did improve students' performance and retention in computer science and their attitudes towards computer science.

/pdf/evaluating-the-effectiveness-of-a-new-instructional-approach-2hpdl91vd8.pdf

Barbara M. Moskal

Papers

Scoring Rubric Development: Validity and Reliability.

Scoring Rubrics: What, When and How?

The Globally Competent Engineer: Working Effectively with People Who Define Problems Differently

Assessment in Engineering Education: Evolution, Approaches and Future Collaborations

Evaluating the effectiveness of a new instructional approach