Showing papers in "Evaluation in 1995"
TL;DR: In this article, the authors define a set of policy measures that constitute the "New Public Management" (NPM), and evidence is cited to support the contention that the NPM has been widely adopted.
Abstract: The article first defines a set of policy measures that constitute the 'New Public Management' (NPM). Evidence is cited to support the contention that the NPM has been widely adopted, with local va...
470 citations
TL;DR: In fact, what experience and history teach us is this, that nations and governments have never learned anything from history as mentioned in this paper, and this is the case even in the case of science.
Abstract: But what experience and history teach us is this, that nations and governments have never learned anything from history. (G. W. F. Hegel, 1837, cited in Feyerabend 1978)Contemporary literature on p...
79 citations
TL;DR: In this article, the authors argue that evaluators can make major contributions to strategic decision-making by using procedures which clarify the nature of programs in their trial stage and outline the implications of implementing the program for the organization as a whole.
Abstract: There is an emerging need for evaluators to assist with strategic decision making in organizations committed to ongoing learning and renewal. While some evaluators have developed rapid-response techniques to assist decision-makers fine tune their programs, new evaluation techniques are required to serve the needs of learning organizations which are committed to systems thinking as a way of improving their effectiveness. This article argues that evaluators can make major contributions to strategic decision- making by using procedures which clarify the nature of programs in their trial stage and outline the implications of implementing the program for the organization as a whole. A case study of the use of these procedures in an educational setting is outlined for the purposes of illustration.
71 citations
TL;DR: This paper describes how trailing research was applied to evaluate a Norwegian public program supporting business development and found that trailing research had no trouble producing credible results for stakeholders involved in the program and external stakeholders who shared the learning process had few problems in granting it credibility.
Abstract: Trailing research is a model for evaluation aiming at enhancing use. The model integrates formative and summative evaluation in a planned learning process coupled with producing knowledge for the scientific community. Formative and summative activities are built into an action research model to enhance the immediate use of evaluation findings. The underlying idea of trailing research is to combine pragmatic evaluation activity with reflection processes. In this paper we describe how trailing research was applied to evaluate a Norwegian public program supporting business development. A major challenge in evaluations is to produce credible results. Trailing research had no trouble producing credible results for stakeholders involved in the program. A small number of external stakeholders not engaged in the evaluation activity were more unwilling to accept its results or grant it credibility. External stakeholders who shared the learning process, however, had few problems in granting it credibility.
58 citations
TL;DR: In this paper, an evaluation strategy informed by feminist and critical thinking is discussed as a means for restructuring the evaluation function so as to make it more consistent with the critical interest of empowerment-oriented social programs and the implications this has for evaluation research.
Abstract: This article addresses the evaluative concerns of feminists and other critical thinkers who are politically committed to an agenda for social change. Attempts to move beyond traditional concepts of social programming, particularly those which are empowerment oriented, run the risk of being at odds with available evaluation strategies. The argument being presented is that the incompatibility between empowerment-oriented interventions and traditional evaluation practice is more an issue of discourse than it is of methodology or technique. A Habermasian perspective is applied to identify the critical interest of empowerment-oriented social programs and the implications this has for evaluation research. An evaluation strategy informed by feminist and critical thinking is discussed as a means for restructuring the evaluation function so as to make it more consistent with this critical interest.
50 citations
TL;DR: In this article, a taxonomy of policy responses is devised and some conclusions drawn on the usefulness of concept mapping as an evaluation technique to assess community relations programmes in Northern Ireland, based on which a policy evaluation in such a scenario is described.
Abstract: Ideal experimental design conditions rarely exist in public policy evaluation. It is not uncommon for evaluators to be faced with a situation where a collection of programmes is being implemented in pursuance of a public policy, couched in nebulous terms, which is more politically defensible than pragmatic. The individual programmes, which collectively equate to 'the policy', may have had staggered starting times with no baseline measurements and only a tenuous sense of the causal link between policy activities and impact. This paper describes a policy evaluation in such a scenario. Concept mapping (multidimensional scaling and hierarchical clustering) is used as an evaluation technique to assess community relations programmes in Northern Ireland. From this a taxonomy of policy responses is devised and some conclusions drawn on the usefulness of concept mapping as an evaluation technique.
23 citations
TL;DR: In France, the institutionalization of evaluation through its adoption by state and then local bureaucracies, a process heavily encouraged by the European Commission, has led to a new relationship between knowledge and power as discussed by the authors.
Abstract: Evaluation of public policies in France has been late arriving but its subsequent development has been markedly sustained. Although the methods used are often either rudimentary or excessively rigid, evaluation is now an integral part of public action. Having first presented the approaches behind this trend, this article then follows the institutionalization of evaluation through its adoption by state and then local bureaucracies, a process heavily encouraged by the European Commission. Two distinctive features, both involving social scientists, mark the French evaluation scene and influence the article's final section. First, although some continue to actually carry out evaluations, today a growing number of academics are involved as key members of committees advising actors responsible for the evaluation of public policies. Second, these committees serve as interfaces between evaluators and actors. Together these developments could be seen as prefiguring a new relationship between knowledge and power.
22 citations
TL;DR: In this article, the authors draw on their experience of managing and evaluating a major international education project in Belize, Central America (as well as other international projects) to draw attention to potential conflicts that arise when western assumptions are presumed in evaluating major educational reform programs in small states.
Abstract: Drawing on their experience of managing and evaluating a major international education project in Belize, Central America (as well as other international projects) the authors seek to draw attention to potential conflicts that arise when western assumptions are presumed in evaluating major educational reform programs in small states. In particular, they draw attention to the political dimensions that underlie such projects; to the issues of State vs Church; to cultural differences and to the issue of absorptive capacity. In doing so, they question whether characteristics which have been attributed uniquely to small states are, in fact, not more generally to be found in all societies where resources are scarce. They conclude with a checklist of approaches that seek to address these issues.
20 citations
TL;DR: The European Union's Framework Programmes for research and technological development have been subject to a variety of evaluative activities over the past 10 years as discussed by the authors, covering issues such as the interaction between peer review panels and supporting studies by evaluation specialists, difficulties of establishing comparability across diverse national R&D systems, and the problems involved in addressing the Programmes' higher-level goals such as enhancing the competitiveness of European industry.
Abstract: The European Union's Framework Programmes for research and technological development have been subject to a variety of evaluative activities over the past 10 years. These have included evaluations by panels, studies of impacts upon individual Member States, horizontal or issue-based evaluations and high-level reviews by, or on behalf of the principal stakeholders. This article reviews this experience, covering issues such as the interaction between peer review panels and supporting studies by evaluation specialists, the difficulties of establishing comparability across diverse national R&D systems, and the problems involved in addressing the Programmes' higher-level goals such as enhancing the competitiveness of European industry. The different perspectives taken by the main stakeholders, the Commission, the Member States and the European Parliament, are noted. In the light of recent proposals to develop the evaluation of the Framework Programme, conclusions are drawn on a series of topics. In particular,...
18 citations
TL;DR: The following address is by Eleanor Chelimsky, who at the time it was given was Assistant Comptroller General for Program Evaluation and Methodology in the United States General Accounting Office.
Abstract: Speeches and Addresses is an occasional feature, not in article format, in which presentations at conferences and other public gatherings are seen as likely to be of interest to a wider audience. Sometimes these speeches and addresses will undoubtedly be contentious, in which case responses and counter-arguments are to be expected and are welcome. Contributions to this section are intended to make accessible relevant material for academic, policy-making and practitioner audiences.The following address is by Eleanor Chelimsky, who at the time it was given was Assistant Comptroller General for Program Evaluation and Methodology in the United States General Accounting Office. This was a 'Keynote Address' at the National Conference on Research Synthesis sponsored by the Russell Sage Foundation, Washington DC, in June 1994.
10 citations
TL;DR: The European evaluation community has a long history of debates about positivist and constructivist paradigms, about the skills of evaluators-technical and reflexive, about methodology, utilization, evaluation design and a host of other theoretical, policyoriented and practitioner issues as discussed by the authors.
Abstract: European evaluation community. These developments have been accelerated by the existence and policies of the European Union but have also built on much older foundations. Well-established evaluation networks in the UK, Scandinavia and Spain, for example, have been in active dialogue with colleagues in North America for many years (a dialogue the journal aims to sustain and enhance). But many of these established links were between particular evaluation networks: in education, in research and development, and in healthcare, or perhaps among those engaged in environmental protection or third world development. What we are now seeing is a more general blurring of the boundaries between different evaluation traditions which have often existed in relative isolation and even ignorance of each other. These evaluation traditions have sometimes become identified with distinctive ’quasi disciplines’, such as program evaluation, technology assessment, auditing and value-added studies, quality assurance and control processes, software validation and testing. Yet on closer inspection these apparently quite differentiated traditions have much in common. Debates about positivist and constructivist paradigms, about the skills of evaluators-technical and reflexive, about methodology, utilization, evaluation design and a host of other theoretical, policyoriented and practitioner issues-find resonance across the board. There are also parallel value debates within many evaluation traditions. For example, these may be between ’top-down’ managerial and accountability driven concerns and
TL;DR: The authors adopt a variety of formats for reviews, intended to foster a pluralistic and trans-diseiplinary approach, including within the coverage not only books but also grey material of contemporary relevance to the diverse evaluation community.
Abstract: The Reviews section will feature in alternate issues of the journal. We will be taking a pluralistic and transdiseiplinary approach, including within the coverage not only books but also ’grey’ material of contemporary relevance to the diverse evaluation community. ’Classic’ texts which have had a seminal influence on the field may also be revisited for their contemporary significance. We propose adopting a variety of formats for reviews, intended to foster
TL;DR: The evaluation community is open to contributions from members of the evaluation community worldwide as discussed by the authors, and they particularly welcome news about evaluation societies, the events that they are undertaking and other international evaluation activities.
Abstract: This section, which will appear in each issue of the journal, is open to contributions from members of the evaluation community worldwide. We particularly welcome news about evaluation societies, the events that they are undertaking and other international evaluation activities.Please send contributions c/o The Editorial Office.
TL;DR: In this article, the authors present frameworks designed to frame qualitative evaluation data in a manner that addresses both macro perspectives of communities and micro concerns of individuals; illuminate educational obstacles and conflicts surrounding the conditions and consequences of reformed social policies; portray the social, cultural and psychological concerns of participants in their acceptance or rejection of changed policies; and develop clarity in repr...
Abstract: Transformed societal relationships in nations emerging from former Soviet influence present unique evaluation challenges. Altered links between governance and instruction create a climate in which the instruments for evaluation must differ not only in their scope but also in their intent. A central question in evaluating educational reform is how classroom, local school and community data can be made accessible to policy-makers who may not have the time, the training or the inclination to search through what has been traditionally regarded as ethnographic data. This article shares frameworks designed to: frame qualitative evaluation data in a manner that addresses both macro perspectives of communities and micro concerns of individuals; illuminate educational obstacles and conflicts surrounding the conditions and consequences of reformed social policies; portray the social, cultural and psychological concerns of participants in their acceptance or rejection of changed policies; and develop clarity in repr...
TL;DR: In this paper, the application of discourse analysis in the empirical context of learning technology innovation is examined, and various theoretical and methodological principles, as well as the technical procedures which have been followed in relation to the operationalization of an evaluation design using discourse analysis.
Abstract: This article examines the application of discourse analysis in the empirical context of learning technology innovation. It outlines the various theoretical and methodological principles, as well as the technical procedures which have been followed in relation to the operationalization of an evaluation design using discourse analysis. The study itself is concerned with stakeholders' subjective and experiential knowledge structures which underlie a culture of technology innovation. The article examines how technology innovation is discursively constituted through the creation of shared meanings and images among stakeholders.