Showing papers in "Evaluation & Research in Education in 2006"

Prevalence of Mixed-methods Sampling Designs in Social Science Research

Abstract: The purpose of this mixed-methods study was to document the prevalence of sampling designs utilised in mixed-methods research and to examine the interpretive consistency between interpretations made in mixed-methods studies and the sampling design used. Classification of studies was based on a two-dimensional mixed-methods sampling model. This model provides a typology in which sampling designs can be classified according to the time orientation of the components (i.e. concurrent versus sequential) and the relationship of the qualitative and quantitative samples (i.e. identical versus parallel versus nested versus multilevel). A quantitative analysis of the 42 mixed-methods studies that were published in the four leading school psychology journals revealed that a sequential design using multilevel samples was the most frequent sampling design, being used in 40.5% (n=17) of the studies. More studies utilised a sampling design that was sequential (66.6%; n=28) than concurrent (33.4%; n=14). Also, multilevel...

Educational Research and Evidence-based Policy: The Mixed-method Approach of the EPPE Project

Abstract: In a review of the challenges to progress in providing social research evidence that might usefully inform policy, Oakley (2004) argues strongly that the ‘paradigm divide’ between qualitative and quantitative research communities continues to constitute a major problem. Oakley refers to a number of recent critiques of what is seen as ‘misplaced positivism’ in educational research and Hammersley (1997, 2005) has suggested that educational research findings are now routinely being applied uncritically to inform educational policy and practice. Research in early childhood education has not been immune to these general criticisms and specific concerns have also been expressed regarding alleged technicism, and the reification of dominant conceptions of early educational quality. In this paper we test these theoretical claims against the realities of conducting one recent and influential early years study: the Effective Provision of Preschool Education (EPPE) Project. The paper provides an account of the EPPE r...

The Role of Reading Comprehension in Responses to Positively and Negatively Worded Items on Rating Scales

Abstract: Should instruments, such as Likert-type scales, contain both positively worded and negatively worded items within the same scale (i.e. mixed format)? Recent evidence suggests that the use of scales with a mixed format can adversely affect the psychometric properties of scales. In particular, the mean item response to the positively worded items has been found to be statistically and practically significantly different than the corresponding mean item response to the negatively worded items with the same scale. Because positively and negatively worded items may evoke different responses for some individuals, it is likely that reading ability plays an important role in the quality of responses to scales with a mixed format. Indeed, Marsh (1984) found a relationship between inappropriate responses to negatively worded items and reading ability among pre-adolescents. Likewise, Williams and Swanson (2001) found a similar relationship for adults. Yet, no other studies have investigated this relationship. Thus, ...

Motivation in Second Language Acquisition in Chinese Research Students in the UK

Abstract: The present study sets out to examine motivation to learn English by Chinese research students in an informal setting. Data were collected, using semi-structured interviews, from four research students at two points in time during their first year in the UK. The main findings are: they believed that learning English was important; their main goal orientations were instrumental and extrinsic; they set learning goals and persisted to attain them; they valued their current learning environment in general and saw it as supportive of their goals; they held both positive and negative attitudes towards the British, which had differential effects on their motivation; their self-perceived support seemed to have a positive impact on their motivation and the development of self-confidence; they tended to attribute their success to stable causes such as the environment and failure to unstable but controllable causes such as effort. It is concluded that qualitative data of this kind may complement insights from quanti...

Methodological Synergy in a National Project: The VITAE Story

Abstract: VITAE was a four-year research project designed to explore the work and lives of a purposive sample of 300 Key Stage 1, 2 and 3 (English and maths) teachers at different phases of their careers in 100 primary and secondary schools in different socioeconomic contexts, drawn from seven local authorities in England. Its focus was upon identifying variations in different aspects of teachers' lives and work and examining possible connections between these and their effects on pupils as perceived by the teachers themselves and as measured by value-added national test scores. An integrated mixed-method approach was developed in addressing the research questions. The results showed that there were associations between teachers' work, lives and identities, that teachers' perceived and relative (valueadded) effectiveness varied within each of six professional life phases, and that this variation depended upon their capacity to manage a number of moderating and mediating factors. Statistically significant relationships were found between teacher commitment, resilience and the value-added pupil test scores. The findings from the study shed new light upon the meanings and measurement of teacher effectiveness and the complex nature and trajectories of teachers' work, lives and effectiveness in different school contexts.

Combining Numbers with Narratives

Abstract: (2006). Combining Numbers with Narratives. Evaluation & Research in Education: Vol. 19, No. 2, pp. 59-62.

An Evaluative Study on Building the Bridge Between the Training Room and the Workplace

Abstract: A study is reported of the use of a personal journal as a tool for transferring learning from the training room to the workplace. The targeted learning event was an English for Communication training programme. This programme was an intensive four-day course that grouped together participants from the non-UK sites of the aircraft manufacturer Airbus. The methodology centred on measuring the extent to which the learners' autonomy increased over time. The global focus of the analysis was a qualitative evaluation of learner perception of autonomy and of the relative value of the evaluated transfer task. The results from one specific session of the course were compared to three other sessions that did not use the personal journal.

‘It's Pretty Cool To Be Clever’: The Marginal Relevance of Gender to Educational Practices and Attainment at A/AS Level?

Abstract: Gender differences in educational attainment have generated considerable debate. Unfortunately, this debate has been constructed in terms of either the measurement of change in the size of the gender gap over time or attempts to theorise such change. This scholarly divide reflects the commitment of researchers to particular methodological paradigms. Therefore, this paper draws on part of a longitudinal, mixedmethods study conducted in three Sixth Forms in England to examine both the extent and causes of gender differences in attainment. Initially, a questionnaire was completed by 330 students in the Sixth Forms and, subsequently, in-depth interviews, nonparticipant observation and a vignette were used to explore students' educational experiences. The public examination grades students held on entering and leaving the Sixth Form were analysed to establish gender differences in attainment. At the start of students' studies, the size of the gender gap in attainment was found to be relatively modest and this ...