The World is Flat: A Brief History of the Twenty-First Century

The purpose of this research review is to open dialog about quantitative, qualitative, and mixed research methods in engineering education research. Our position is that no particular method is privileged over any other. Rather, the choice must be driven by the research questions. For each approach we offer a definition, aims, appropriate research questions, evaluation criteria, and examples from the Journal of Engineering Education. Then, we present empirical results from a prestigious international conference on engineering education research. Participants expressed disappointment in the low representation of qualitative studies; nonetheless, there appeared to be a strong preference for quantitative methods, particularly classroom-based experiments. Given the wide variety of issues still to be explored within engineering education, we expect that quantitative, qualitative, and mixed approaches will be essential in the future. We encourage readers to further investigate alternate research methods by accessing some of our sources and collaborating across education/social science and engineering disciplinary boundaries.

/pdf/quantitative-qualitative-and-mixed-research-methods-in-4e1er7zqph.pdf

Quantitative, Qualitative, and Mixed Research Methods in Engineering Education

Do you spend a lot of time during the design process wondering what users really need? Do you hate those endless meetings where you argue how the interface should work? Have you ever developed something that later had to be completely redesigned? 

Paper Prototyping can help. Written by a usability engineer with a long and successful paper prototyping history, this book is a practical, how-to guide that will prepare you to create and test paper prototypes of all kinds of user interfaces. You'll see how to simulate various kinds of interface elements and interactions. You'll learn about the practical aspects of paper prototyping, such as deciding when the technique is appropriate, scheduling the activities, and handling the skepticism of others in your organization. Numerous case studies and images throughout the book show you real world examples of paper prototyping at work. 

Learn how to use this powerful technique to develop products that are more useful, intuitive, efficient, and pleasing: 

* Save time and money - solve key problems before implementation begins 
* Get user feedback early - use it to focus the development process 
* Communicate better - involve development team members from a variety of disciplines 
* Be more creative - experiment with many ideas before committing to one

*Enables designers to solve design problems before implementation begins 

*Five case studies provide real world examples of paper prototyping at work 

*Delves into the specifics of what types of projects paper prototyping is and isn't good for.

Table of Contents

Foreword by Jakon Nielsen
Acknowledgments
Part I-Introduction to Paper Prototyping
Chapter 1: Introduction
Chapter 2: Case Studies
Chapter 3: Thinking about Prototyping
Chapter 4: Making a Paper Prototype
Part II-Process: Conducting a Usability Study with a Paper Prototype
Chapter 5: Planning a Usability Study with a Paper Prototype
Chapter 6: Task Design
Chapter 7: Preparing the Prototype
Chapter 8: Introduction to Usability Test Facilitation
Chapter 9: Usability Testing with a Paper Prototype
Chapter 10: Observers
Chapter 11: Data: Capturing, Prioritizing, and Communicating
Part III-Deciding Whether to Use Paper
Chapter 12: What Paper Is (and Isn't) Good For
Chapter 13: The Politics of Paper Prototyping
Chapter 14: When to Use Paper
Part IV-Broadening the Focus
Chapter 15: Examples of User-Centered Design
Chapter 16: Final Thoughts
References
Index
Figure Credits
About the Author

Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces

The role of prototypes is well established in the field of HCI and Design. A lack of knowledge, however, about the fundamental nature of prototypes still exists. Researchers have attempted to identify different types of prototypes, such as low- vs. high-fidelity prototypes, but these attempts have centered on evaluation rather than support of design exploration. There have also been efforts to provide new ways of thinking about the activity of using prototypes, such as experience prototyping and paper prototyping, but these efforts do not provide a discourse for understanding fundamental characteristics of prototypes. In this article, we propose an anatomy of prototypes as a framework for prototype conceptualization. We view prototypes not only in their role in evaluation but also in their generative role in enabling designers to reflect on their design activities in exploring a design space. We base this framework on the findings of two case studies that reveal two key dimensions: prototypes as filters and prototypes as manifestations. We explain why these two dimensions are important and how this conceptual framework can benefit our field by establishing more solid and systematic knowledge about prototypes and prototyping.

https://faculty.washington.edu/jtenenbg/publications/limAnatomyOfPrototypes-tochi2008.pdf

The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas

Context: In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers' usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring. Objective: The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14years. Method: A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner. Results: The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners. Conclusions: From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.

Usability evaluation methods for the web: A systematic mapping study

One of the marks of true professionalism is the ability to learn from reflection on practice. Observing the practice of themselves and others provides practitioners with essential knowledge in addition to the core knowledge of the profession. Programs in technical communication, especially graduate programs, should foster the habits and skills of reflecting on practice. This panel discussion will demonstrate ways we have incorporated reflection on learning into our degree programs in technical communication at Mercer University.

Educating reflective practitioners

This paper evaluates the efficacy of using freshman student scores from one non-technical assignment to predict academic success as measured by cumulative grade point average after completion of the first two semesters enrolled at the Mercer University School of Engineering. The predictor assignment is keeping a dialectic course notebook and corresponds to the student's attitude, persistence, and organizational skills rather than math and science preparedness. Statistical analysis, at the 99 percent confidence level, indicated that there was a strong relationship between the student notebook scores and grade point average. Although there was scatter in the data, this one variable does provide insight into student success in the Mercer University Engineering program.

Efficacy of Using a Single, Non-Technical Variable to Predict the Academic Success of Freshmen Engineering Students

Paper prototypes are a low-tech, low-cost, but highly effective, form of usability testing for web site design. Creating a web site is relatively easy, given the technology and tools now available. All too often, web site developers tend to be enamored with the technology and forget the needs of end user. As a result, many web site developers delay usability testing until their web site is completely designed, only to uncover significant interface, content, or structural problems with the site once the customers try to use it. Significant redesign may be impossible due to time, cost, or personnel constraints.This paper discusses the benefits of using paper prototypes to conduct usability testing of a web site for Mercer University's School of Engineering Center for Excellence in Engineering Education (CE3). These benefits include the following: 1. Low cost, both in terms of time and materials required to create the prototype;2. Critical feedback from users, who appear more willing to suggest significant flaws when the site design is obviously very rough;3. Willingness on the part of the designers to change the design, since they have invested minimal time and effort into creating a paper prototype; and ultimately4. Improved usability. .This paper also discusses how to create a paper prototype, how to conduct a usability test with the prototype, and what to do with the test results. This process can be used to enhance the usability of any web site.The design of the CE3 is a project of a team of senior technical communication students in the Department of Technical Communication at Mercer University. This paper also discusses the importance of training technical communicators to be user advocates.

Web site design: a case study in usability testing using paper prototypes

Research Problem: Investigate the match between content published by the Transactions and content sought by its readers. Research Questions: What content does the IEEE Transactions on Professional Communication publish? How does that compare to the content published by other journals in the field? And what content do readers of the Transactions want to read? Literature Review: Researchers in most fields occasionally analyze the entire body of literature within a discipline to assess the current state of the literature, identify leading works, assess the state of the literature, provide a basis for changing the direction of a journal, and assess alignment among parts of the literature. Methodology: To identify what journals published, researchers used the STC Body of Knowledge schema and a list of categories of research methods that classify all peer-reviewed articles published between January 2006 and December 2010 in the Transactions, Journal of Business and Technical Communication, Technical Communication, and Technical Communication Quarterly. To identify reader preferences of the Transactions, researchers surveyed members of the IEEE Professional Communication Society about their preferences for content and types of research. Results and Discussion: In terms of the topics covered, the three most widely covered topics in the Transactions were: (1) Deliverables, (2) Information Design and Development, and (3) Academic Programs. Readers prefer (1) About Technical and Professional Communication, (2) Information Design and Development, and (3) Research Theory, and Practice. The three least-covered topics were (1) Business Knowledge, (2) About Technical Communication, and (3) Technical Communication Standards. Of least interest to participants were: (1) Deliverables, (2) Quality Assurance, (3) Management, and (4) Technical Communication Standards. The Transactions primarily publishes experiments, surveys, and tutorials while readers prefer case studies, literature reviews, and tutorials.

What Does the Transactions Publish? What do Transactions' Readers Want to Read?

Once we have committed to begin delivering instruction or training online, we naturally begin to consider ways to translate what we do face-to-face, with a live class, to what we do in a virtual learning environment. The most significant challenges are not, in fact, choosing the technology or deciding upon content—those decisions will be constrained by a number of boundaries that will make it relatively simple to decide. What is most challenging is developing an authentic interactive learning environment online. To develop such an environment online, our chapter argues for the importance of founding online instruction on sound principles of instructional design, particularly the issue of supporting the learner in creating a virtual collaborative learning experience.

H.M. Grady

Papers

Educating reflective practitioners

Efficacy of Using a Single, Non-Technical Variable to Predict the Academic Success of Freshmen Engineering Students

Web site design: a case study in usability testing using paper prototypes

What Does the Transactions Publish? What do Transactions' Readers Want to Read?

Teaching Well Online with Instructional and Procedural Scaffolding