Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ)

TL;DR: In this paper, the authors report on the construction, reliability, internal validity, and external validity of the Achievement Emotions Questionnaire (AEQ) which is designed to assess various achievement emotions experienced by students in academic settings.

About: This article is published in Contemporary Educational Psychology.The article was published on 2011-01-01 and is currently open access. It has received 1150 citations till now. The article focuses on the topics: Academic achievement & Test anxiety.

Summary

1. Introduction

• [n line with current test anxiety measurement and conceptions of emotion more generally, the AEQ is based on a multi component definition of achievement emotion.
• [n contrast to test anxiety measures, however, the AEQ assesses a broader range of major achievement emotions.
• The 24 scales of the instrument tap into nine different emotions occurring in three different academic achievement settings.
• Next, the authors describe the construction of the instrument.
• The authors then report an empirical analysis targeting item and scale statistics, reliability, internal validity, and external validity of the instrument.

1.1. Conceptual framework : the control value theory of achievement emotions

• As a framework for defining emotions, constructing scales, and validating the instrument, the control value theory of achievement emotions was used (Pekrun, 2006; Pekrun et aI., 2007) .
• The con trol value theory provides an integrative approach for analyzing various emotions experienced in achievement contexts, including academic settings as well as achievement situations in other life domains (e.g., sports, professional activities).
• It expands these views by integrating propositions from different theories and by focusing on both outcome related and activity related achievement emotions.

1.1.1 . Definition and component structures of emotion

• From a measurement perspective, the multi component concep tion of emotions adopted in the control value theory implies that emotions are best modeled as hierarchically organized structures, with the components comprising an emotion being first order factors and the emotion itself being represented by a second order factor.
• Test anxiety would be conceived as being repre sented by one second order factor for the emotion test anxiety, and four primary factors for the affective, cognitive, motivational, and physiological components of test anxiety that are nested within the second order factor (Fig. 1 ; see Hodapp & Benson (1997) for a similar approach).
• Empirically, such hierarchical factor models should prove superior to single factor models postulating just one factor representing the emotion.

1.1.2. Definition of acllievement emotion

• In Pekrun's (2006; Pekrun et aI., 2002) three dimensional tax onomy of achievement emotions, the differentiation of activity versus outcome emotions pertains to the object focus of these emotions.
• In addition, as with emotions more generally, achieve ment emotions can be grouped according to their valence and to the degree of activation implied.
• In terms of valence, positive emotions can be distinguished from negative emotions, such as pleasant enjoyment versus unpleasant anxiety.
• In terms of activa tion, physiologically activating emotions can be differentiated from deactivating emotions, such as activating hope versus deac tivating hopelessness.
• By using the dimensions valence and acti vation, the taxonomy is consistent with circumplex models of affect that arrange affective states in a two dimensional (valence x activation) space (Feldman Barrett & Russell, 1998; Linnenbrink, 2007) .

1.1.3. Situational context and temporal specificity

• In addition, in keeping with emotions more generally, achieve ment emotions can be conceptualized in trait like or state like ways.
• The defining characteristic of the trait versus state distinc tion is the temporal generality of the emotion under consideration.

1.1.4. Antecedents of achievement emotions

• The theory proposes that enjoyment of achievement activities is instigated when these activities are experienced as both controlla ble and valuable.
• A student is expected to enjoy studying when she feels competent to master the learning material and perceives the material as interesting.
• Conversely, boredom is induced when the activity lacks any incentive value.
• A student would feel anxious before an exam if he expects that he could fail and perceives the exam as important.
• Hopelessness is thought to be triggered when achievement seems not controllable at all, implying subjective cer tainty about failure.

1.1.5. Outcomes of achievement emotions

• Positive activating emotions such as enjoyment, hope, and pride are thought to promote both intrinsic and extrinsic motivation, facilitate use of fl exible learning strategies, and support self regu lation, thus positively affecting academic performance under most conditions.
• Conversely, nega tive deactivating emotions, such as hopeless ness and boredom, are posited to uniformly reduce moti vation and the effortful processing of information, implying nega tive effects on performance.
• For positive deactivating and negative activating emotions, such as relief, anger, anxiety, and shame, the relationships are presumed to be more complex.
• Specifically, an gel', anxiety, and shame can undermine intrinsic motivation, but can induce strong extrinsic motivation to invest effort to avoid fail ure, implying that the effects on students' overall motivation to learn and invest effort need not be negative.
• Furthermore, these emotions are expected to promote use of more rigid learning strat egies like rehearsal.

1.2. Construction of the AEQ 1.2.1. Rational empirical strategy of test construction

• Construction of the AEQ was based on the theoretical consider ations outlined earlier and on a series of preliminary empirical studi es.
• These studies included exploratory investigations analyz ing the occurrence and structures of various achievement emotions (Pekrun, 1992c; Pekrun et aI., 2002; Spangler, Pekrun, Kramer, & Hofmann, 2002) and four quantitative studies focusing on scale development (Pekrun, Goetz, Perry, Kramer, & Hochstadt.
• The studies were guided by theory and were used to in form further development of theory based emotion taxonomies which, in turn, were employed to construct the final AEQ scales.
• Thus, the strategy used' involved theory evidence loops integrating both rational and empirical perspectives (for more information, see Pekrun et aI., 2004; Titz, 2001) .

1.2.2. Emotions assessed by the AEQ

• The decision to include scales for nine different emotions (enjoyment, hope, pride, relief, anger, anxiety, hopelessness, shame, and boredom) was based on two criteria.
• Second, the authors chose emotions to represent major emotion categori es as defined by the three dimensional taxonomy outlined earlier.
• Accordingly, the AEQ addresses activity emotions (enjoy ment, boredom, and anger), prospective outcome emotions (hope, anxiety, and hopelessness), and retrospective outcome emotions (pride, relief, and shame).
• In terms of valence, the instrument measures both positive and negative emotions, and in terms of activation, it assesses both activating and deactivating emotions.
• As such, the AEQ makes up the four emotion categories compris ing the valence and activation dimensions: positive activating (enjoyment, hope, pride); positive deactivating ; negative activating (anger, anxiety, shame); and negative deactivating (hopelessness, boredom).

1.2.3. Defining situational context and temporal specificity

• In line with the contextual specificity of achievement emo tions, the authors constructed separate scales for class related, learning related, and test related emotions.
• Regarding temporal specific ity, the original version of the AEQ is intended to measure stu dents' habitual, trait like achievement emotions.
• The instrument can be used to assess all three types of emotions mentioned earlier (trait, course specific, state) by adapting the instructions accordingly (see Pekrun, Goetz, Frenzel, & Perry, 2011 ) .

1.2.4. Item and scale development

• The class related emo tion scales include 80 items and instruct students to report how they feel with regard to class related enjoyment, pride, an ger, anxiety, shame, hopelessness, and boredom.
• The learning re lated emotion scales include 75 items and instruct students to report how they feel with regard to studying in terms of the same eight emotions.
• Within each section, the items are or dered in three blocks assessing emotional experiences before, during, and after an encounter with the specified academic con text.
• These blocks focus on activity emotions , prospec tive outcome emotions , and retrospective outcome emotions related to the setting addressed.
• Sequencing items this way is in line with principles of situation reaction inventories and is intended to help respondents access their emotional memories (Endler & Okada, 1975) .

1.3. Prior research using the AEQ

• The findings of these studies suggest that the AEQ scales can be used to analyze various achievement emotions.
• Since none of studies included more than a subset of scales, they did not provide a systematic account of the psychometric quality of the instrument and of the full range of emotions addressed by the AEQ.
• Therefore, the authors lack knowledge about the overall reliability and validity of the instrument.
• In particular, there is a research def icit regarding the internal component structures and interrelations of diverse achievement emotions as assessed by the AEQ, and about the relationships of the full set of emotions with important antecedents and outcomes such as students' control value apprais als, learning, and academic performance.

2.1. Partidpants and procedure

• The sample consisted of 389 stud ents (234 female; age: M = 20.63 years; SD = 3.48) in several undergraduate psychology courses at a large, Midwestern Canadian university who partici pated in return for extra course credit.
• The distribution of students across genders and faculties is typical for students participating in undergraduate psychology courses at Canadian universities (Perry, Stupnisky, Hall, Chipperfield, & Wei ner, 2010) .
• Participants completed the measures in one session.

2.2. 1. Achievement emotions

• To assess students' achievement emotions, the complete AEQ as described earlier was used .
• Stu dents were instructed to report how they felt, typically, when attending class, studying, or taking test and exams in their univer sity courses.

2.2.2. Perceived co ntrol and value

• The items of the Perceived Academic Control Scale relate to influenci ng academic performance (e.g., "I have a great deal of con trol over my academic performance in my courses";.
• The Self Efficacy for Learning and Performance Scale consists of five items assessing students' confidence about being able to master academic tasks and get good grades (e.g., "I'm confident I can do an excell ent job on the assignments and tests in courses at unive rsity").
• Partici pants responded by using 1 (strongly disagree ) to 5 (strongly agree) scales, and the scores w ere summed to form the two co ntrol in dexes (a = .83 and .82 for academic control and self efficacy, res pectively).

2.2.4. Learning strategies

• As indicators of flexible versus rigid learning strategies, stu dents' use of elaboration and rehearsal strategies was assessed.
• Both strategies were measured with t he respective scales of the MSLQ (Pi ntrich et aI. , 1991) .
• The elaboration and rehearsal scales consisted of six and four items, respectively (e.g., "When reading for my classes, I tIy to relate the material to what I already know" ;.
• "When studying for my classes, I practice saying the material to myself over and over").
• Participants responded by u sing 1 (strongly disagree ) to 5 (strongly agree) scales, and the scores were summed to form the elaboration and rehearsal indexes (as = .73 and .59 for elaboration and rehearsal, respectively).

2.2.5. Self regulation versus external regulation of learning

• A four item versio n of Goetz' (2004) Perceived External Regulation of Learning Scale was used to measure exter nal regulation (e.g., "The way I study largely depends on the pro fessor's recommendations";.
• "When studying, I entirely rely on the readings I am given").
• Participants responded by using 1 (strongly disagree ) to 5 (strongly agree) scales, and the scores were summed to form the self regulation and external regulation in dexes (as = .72 and .57, respectively).

3.1. Item and scale statistics

• Table 1 shows response distributions, item total correlations, and reliabili ties of the AEQ scales.
• The findings indicate that there was sufficient variation of scores on all scales.
• Most of the distribu tions were relatively symmetrical, the exception being the hope less ness scales which were positively skewed.
• Given the extreme nature and re latively rare occurrence of this emotion in achieve ment settings (Pekrun, 1992c ; Titz, 2001) , such skewness seems adequate and should not be reduced by normalizing distributions.
• In sum, these findings indi cate that the AEQscales show sufficient variation, and that re li abil ities range from good to excellent.

3.2. Gender differences

• In analyzing the scale statistics separately by gender, the authors found that means were significantly different for five emotions.
• In interpreting these gender differences, it should b e noted that the effect sizes of the differences were relatively small (a ll ds < .40), and that none of the differences generalized across settings, with the single exception of anxiety which differed significantly for set tings of both learning and tak ing tests.
• The differences in self re ported anxiety are consistent with a vast literature s howing that, on an average, female students report higher achievement anxiety than male students (H embree, 1988; Zeidner, 1998 ) .
• Given the consistency of this finding reported in the literature, the present findings attest to the convergent val idity of the AEQ anxiety scales in terms of replicating these differences.

3.3. Internal validity: component structures of emotions

• These findings indicate that scale construction was successful in terms of internal, structural validity.
• As such, the findin gs show that test anxiety is not the only emotion for which internal compo nent structures should be taken into account.
• For most of the scales, the component factors and hierarchical models which dif ferentiated between emotion components fit better than one fac tor models, thus corroborating their propositions on the internal structures of achievement emotions.

3.4.1. Correlational analysis

• Hopelessness, which showed no more than moderately negative relationships.
• Indeed, the strongest relationships were found for neighboring, like valenced emotions such as enjoyment and hope, or anxiety, shame, and hopelessness.
• In interpreting these correia tions, it is important to note that the present study used the AEQ to assess students' habitual, trait like emotions.
• Furthermore, as expected, the correlations also indicate that the emotions were separable across the three settings examined (Ta ble 3).
• The strongest correia tions across class, studying, and taking tests were found for stu dents' hopelessness and shame.

3.4.2. Structural equation modeling of latent relationships

• Linkages with students' appraisals, leaming, and pelformance 3.5.1, also known as 5. External validity.
• Relationships with control and value appraisals Table 5 shows the correlations of students' control value appraisals and the AEQ achievement emotions.
• As predicted by Pekrun's (2006) control value theory described earlier, there were clear linkages between appraisals and emotions.
• Regarding value, it should be noted that this variable was opera tionalized as positive task value in the present study.
• The con trol value theory proposes that the negative value of failure contributes to students' negative outcome emotions such as anxi ety, shame, and hopelessness, but this proposition was not tested in the present study.

3.5.2. Relationships with leaming and performance

• These findings demonstrate the external validity of the AEQ scales and show that students' emotions have substantial linkages with their engagement and performance.
• Many of these relation ships proved to be rather strong, with correlations in the.30 .50 range.
• Interestingly, these relationships were relatively weak for test anxiety, as compared with other achievement emotions.
• Whereas the correlation between the AEQ test hopeless ness scale and students' GPA was r = -.34, the correlation for test anxiety was r = -. 14 in the present research a low correlation which is quite typical for the range of correlations produced by test anxiety studies (Hembree, 1988) .
• The findings of the present study thus reinforce the premise that research on students' affect is well advised to move on from test anxiety to include a broader range of emotions experienced in academic settings.

4. Conclusions

• Finally, the present findings have a number of important implications for educational practice.
• To date, the instrument has mainly been employed for research purposes, but it also may be well suited to serve practical pur poses for assessment in counseling and evaluation.
• Given the overall length of the instrument, this may require further re search to tailor the scales to the specific purposes within gi ven diagnostic settings.
• Also, research would be needed to norm the scales for practical application.
• By implication, educators are well advised to heed stu dents' emotions including the well researched emotion test anx iety, but also including a broad variety of emotions beyond anxiety.