/pdf/the-development-and-psychometric-properties-of-liwc2015-n5qek2rwxf.pdf

The Development and Psychometric Properties of LIWC2015

This article is a review of experiments comparing the effectiveness of human tutoring, computer tutoring, and no tutoring. “No tutoring” refers to instruction that teaches the same content without tutoring. The computer tutoring systems were divided by their granularity of the user interface interaction into answer-based, step-based, and substep-based tutoring systems. Most intelligent tutoring systems have step-based or substep-based granularities of interaction, whereas most other tutoring systems (often called CAI, CBT, or CAL systems) have answer-based user interfaces. It is widely believed as the granularity of tutoring decreases, the effectiveness increases. In particular, when compared to No tutoring, the effect sizes of answer-based tutoring systems, intelligent tutoring systems, and adult human tutors are believed to be d = 0.3, 1.0, and 2.0 respectively. This review did not confirm these beliefs. Instead, it found that the effect size of human tutoring was much lower: d = 0.79. Moreover, the eff...

The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems

Objectives To update a 2004 systematic review of health care service use and health outcomes related to differences in health literacy level and interventions designed to improve these outcomes for individuals with low health literacy. Disparities in health outcomes and effectiveness of interventions among different sociodemographic groups were also examined. Data sources We searched MEDLINE®, the Cumulative Index to Nursing and Allied Health Literature, the Cochrane Library, PsychINFO, and the Educational Resources Information Center. For health literacy, we searched using a variety of terms, limited to English and studies published from 2003 to May 25, 2010. For numeracy, we searched from 1966 to May 25, 2010. Review methods We used standard Evidence-based Practice Center methods of dual review of abstracts, full-text articles, abstractions, quality ratings, and strength of evidence grading. We resolved disagreements by consensus. We evaluated whether newer literature was available for answering key questions, so we broadened our definition of health literacy to include numeracy and oral (spoken) health literacy. We excluded intervention studies that did not measure health literacy directly and updated our approach to evaluate individual study risk of bias and to grade strength of evidence. Results We included good- and fair-quality studies: 81 studies addressing health outcomes (reported in 95 articles including 86 measuring health literacy and 16 measuring numeracy, of which 7 measure both) and 42 studies (reported in 45 articles) addressing interventions. Differences in health literacy level were consistently associated with increased hospitalizations, greater emergency care use, lower use of mammography, lower receipt of influenza vaccine, poorer ability to demonstrate taking medications appropriately, poorer ability to interpret labels and health messages, and, among seniors, poorer overall health status and higher mortality. Health literacy level potentially mediates disparities between blacks and whites. The strength of evidence of numeracy studies was insufficient to low, limiting conclusions about the influence of numeracy on health care service use or health outcomes. Two studies suggested numeracy may mediate the effect of disparities on health outcomes. We found no evidence concerning oral health literacy and outcomes. Among intervention studies (27 randomized controlled trials [RCTs], 2 cluster RCTs, and 13 quasi-experimental designs), the strength of evidence for specific design features was low or insufficient. However, several specific features seemed to improve comprehension in one or a few studies. The strength of evidence was moderate for the effect of mixed interventions on health care service use; the effect of intensive self-management inventions on behavior; and the effect of disease-management interventions on disease prevalence/severity. The effects of other mixed interventions on other health outcomes, including knowledge, self-efficacy, adherence, and quality of life, and costs were mixed; thus, the strength of evidence was insufficient. Conclusions The field of health literacy has advanced since the 2004 report. Future research priorities include justifying appropriate cutoffs for health literacy levels prior to conducting studies; developing tools that measure additional related skills, particularly oral (spoken) health literacy; and examining mediators and moderators of the effect of health literacy. Priorities in advancing the design features of interventions include testing novel approaches to increase motivation, techniques for delivering information orally or numerically, "work around" interventions such as patient advocates; determining the effective components of already-tested interventions; determining the cost-effectiveness of programs; and determining the effect of policy and practice interventions.

/pdf/health-literacy-interventions-and-outcomes-an-updated-chswt8nilu.pdf

Health literacy interventions and outcomes: an updated systematic review.

/pdf/the-development-and-psychometric-properties-of-liwc2007-3knbmtjyfy.pdf

The development and psychometric properties of LIWC2007

We study the incidence (rate of occurrence), persistence (rate of reoccurrence immediately after occurrence), and impact (effect on behavior) of students' cognitive-affective states during their use of three different computer-based learning environments. Students' cognitive-affective states are studied using different populations (Philippines, USA), different methods (quantitative field observation, self-report), and different types of learning environments (dialogue tutor, problem-solving game, and problem-solving-based Intelligent Tutoring System). By varying the studies along these multiple factors, we can have greater confidence that findings which generalize across studies are robust. The incidence, persistence, and impact of boredom, frustration, confusion, engaged concentration, delight, and surprise were compared. We found that boredom was very persistent across learning environments and was associated with poorer learning and problem behaviors, such as gaming the system. Despite prior hypothesis to the contrary, frustration was less persistent, less associated with poorer learning, and did not appear to be an antecedent to gaming the system. Confusion and engaged concentration were the most common states within all three learning environments. Experiences of delight and surprise were rare. These findings suggest that significant effort should be put into detecting and responding to boredom and confusion, with a particular emphasis on developing pedagogical interventions to disrupt the ''vicious cycles'' which occur when a student becomes bored and remains bored for long periods of time.

/pdf/better-to-be-frustrated-than-bored-the-incidence-persistence-1wr1zi2i9s.pdf

Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments

AutoTutor is a learning environment that tutors students by holding a conversation in natural language. AutoTutor has been developed for Newtonian qualitative physics and computer literacy. Its design was inspired by explanation-based constructivist theories of learning, intelligent tutoring systems that adaptively respond to student knowledge, and empirical research on dialogue patterns in tutorial discourse. AutoTutor presents challenging problems (formulated as questions) from a curriculum script and then engages in mixed initiative dialogue that guides the student in building an answer. It provides the student with positive, neutral, or negative feedback on the student’s typed responses, pumps the student for more information, prompts the student to fill in missing words, gives hints, fills in missing information with assertions, identifies and corrects erroneous ideas, answers the student’s questions, and summarizes answers. AutoTutor has produced learning gains of approximately .70 sigma for deep levels of comprehension.

/pdf/autotutor-a-tutor-with-dialogue-in-natural-language-3iubab04kj.pdf

AutoTutor: a tutor with dialogue in natural language.

Author(s): Graesser, A.C.; Jackson, G.T.; Matthews, E.C.; Mitchell, H.H.; Olney, A.; Ventura, M.; Chipman, P.; Franceschetti, D.; Hu, X.; Louwerse, M.M.; Person, N.K.; Tutoring Research Group

/pdf/why-autotutor-a-test-of-learning-gains-from-a-physics-tutor-174q1mwxtz.pdf

Why/AutoTutor: A Test of Learning Gains from a Physics Tutor with Natural Language Dialog

Researchers of advanced technologies are constantly seeking new ways of measuring and adapting to user performance. Appropriately adapting system feedback requires accurate assessments of user performance. Unfortunately, many assessment algorithms must be trained on and use pre-prepared data sets or corpora to provide a sufficiently accurate portrayal of user knowledge and behavior. However, if the targeted content of the tutoring system changes depending on the situation, the assessment algorithms must be sufficiently independent to apply to untrained content. Such is the case for Interactive Strategy Training for Active Reading and Thinking (iSTART), an intelligent tutoring system that assesses the cognitive complexity of strategy use while a reader self-explains a text. iSTART is designed so that teachers and researchers may add their own (new) texts into the system. The current paper explores student self-explanations from newly added texts (which iSTART had not been trained on) and focuses on evaluating the iSTART assessment algorithm by comparing it to human ratings of the students' self-explanations.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1756-8765.2009.01068.x

Assessing cognitively complex strategy use in an untrained domain.

The difficulties of grading essays with natural language processing tools are addressed. The present project investigated the effectiveness of combining multiple measures of text similarity to grade essays on conceptual physics problems. Latent semantic analysis (LSA) and a new text similarity metric called Union of Word Neighbors (UWN) were used with other measures to predict expert grades. It appears that the best strategy for grading essays is to use student derived ideal answers and statistical models that accommodate inferences. LSA and the UWN gave near equivalent performance in predicting expert grades when student derived ideal answers served as a comparison for student answers. However, if ideal expert answers are used, explicit symbolic models involving word matching are more suitable to predict expert grades. This study identified some computational constraints on models of natural language processing in intelligent tutoring systems.

G. T. Jackson

Papers

AutoTutor: a tutor with dialogue in natural language.

Why/AutoTutor: A Test of Learning Gains from a Physics Tutor with Natural Language Dialog

Assessing cognitively complex strategy use in an untrained domain.

Combining computational models of short essay grading for conceptual physics problems