Growing demand for health care, rising costs, constrained resources, and evidence of variations in clinical practice have triggered interest in measuring and improving the quality of health care delivery. For a valuable quality assessment, relevant data on outcome must be obtained in a standardized and reproducible manner to allow comparison among different centers, between different therapies and within a center over time.1–3 Objective and reliable outcome data are increasingly requested by patients and payers (government or private insurance) to assess quality and costs of health care. Moreover, health policy makers point out that the availability of comparative data on individual hospital's and physician's performance represents a powerful market force, which may contribute to limit the costs of health care while improving quality.4

Conclusive assessments of surgical procedures remain limited by the lack of consensus on how to define complications and to stratify them by severity.1,5–8 In 1992, we proposed general principles to classify complications of surgery based on a therapy-oriented, 4-level severity grading.1 Subsequently, the severity grading was refined and applied to compare the results of laparoscopic versus open cholecystectomy9 and liver transplantation.10 This classification has also been used by others11–13 and was recently suggested to serve as the basis to assess the outcome of living related liver transplantation in the United States (J. Trotter, personal communication). However, the classification system has not yet been widely used in the surgical literature.

The strength of the previous classification relied on the principle of grading complications based on the therapy used to treat the complication. This approach allows identification of most complications and prevents down-rating of major negative outcomes. This is particularly important in retrospective analyses. However, we felt that modifications were necessary, particularly in grading life-threatening complications and long-term disability due to a complication. We also felt that the duration of the hospital stay can no longer be used as a criterion to grade complications. Although definitions of negative outcomes rely to a large extend on subjective “value” appraisals, the grading system must be tested in a large cohort of patients. Finally, a classification is useful only if widely accepted and applied throughout different countries and surgical cultures. Such a validation was not done with the previous classification.

Therefore, the aim of the current study was 3-fold: first, to propose an improved classification of surgical complications based on our experience gained with the previous classification1; second, to test this classification in a large cohort of patients who underwent general surgery; and third, to assess the reproducibility and acceptability of the classification through an international survey.

/pdf/classification-of-surgical-complications-a-new-proposal-with-h643vehuw5.pdf

Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey.

Postoperative pancreatic fistula: an international study group (ISGPF) definition.

Background. Postoperative pancreatic fistula (POPF) is still regarded as a major complication. The incidence of POPF varies greatly in different reports, depending on the definition applied at each surgical center. Our aim was to agree upon an objective and internationally accepted definition to allow comparison of different surgical experiences. Methods. An international panel of pancreatic surgeons, working in well-known, high-volume centers, reviewed the literature on the topic and worked together to develop a simple, objective, reliable, and easyto-apply definition of POPF, graded primarily on clinical impact. Results. A POPF represents a failure of healing/sealing of a pancreatic-enteric anastomosis or a parenchymal leak not directly related to an anastomosis. An all-inclusive definition is a drain output of any measurable volume of fluid on or after postoperative day 3 with an amylase content greater than 3 times the serum amylase activity. Three different grades of POPF (grades A, B, C) are defined according to the clinical impact on the patient’s hospital course. Conclusions. The present definition and clinical grading of POPF should allow realistic comparisons of surgical experiences in the future when new techniques, new operations, or new pharmacologic agents that may impact surgical treatment of pancreatic disorders are addressed. (Surgery 2005;138:8-13.)

Special article Postoperative pancreatic fistula: An international study group (ISGPF) definition

This new edition of Ann Bowling's well-known and highly respected text has been thoroughly revised and updated to reflect key methodological developments in health research. It is a comprehensive, easy to read, guide to the range of methods used to study and evaluate health and health services. It describes the concepts and methods used by the main disciplines involved in health research, including: demography, epidemiology, health economics, psychology and sociology.The research methods described cover the assessment of health needs, morbidity and mortality trends and rates, costing health services, sampling for survey research, cross-sectional and longitudinal survey design, experimental methods and techniques of group assignment, questionnaire design, interviewing techniques, coding and analysis of quantitative data, methods and analysis of qualitative observational studies, and types of unstructured interviewing. With new material on topics such as cluster randomization, utility analyses, patients' preferences, and perception of risk, the text is aimed at students and researchers of health and health services. It has also been designed for health professionals and policy makers who have responsibility for applying research findings in practice, and who need to know how to judge the value of that research.

Research Methods in Health. Investigating health and health services

Background and Purpose— The modified Rankin scale (mRS), a clinician-reported measure of global disability, is widely applied for evaluating stroke patient outcomes and as an end point in randomized clinical trials. Extensive evidence on the validity of the mRS exists across a large but fragmented literature. As new treatments for acute ischemic stroke are submitted for agency approval, an appreciation of the mRS’s attributes, specifically its relationship to other stroke evaluation scales, would be valuable for decision-makers to properly assess the impact of a new drug on treatment paradigms. The purpose of this report is to assemble and systematically assess the properties of the mRS to provide decision-makers with pertinent evaluative information. Methods— A Medline search was conducted to identify reports in the peer-reviewed medical literature (1957–2006) that provide information on the structure, validation, scoring, and psychometric properties of the mRS and its use in clinical trials. The selecti...

Outcomes validity and reliability of the modified Rankin scale: implications for stroke clinical trials: a literature review and synthesis.

Background: Anastomotic leak after gastrointestinal surgery is an important postoperative event that leads to significant morbidity and mortality. Postoperative leak rates are frequently used as an indicator of the quality of surgical care provided. Comparison of rates between and within institutions depends on the use of standard definitions and methods of measurement of anastomotic leak. The aim of this study was to review the definition and measurement of anastomotic leak after oesophagogastric, hepatopancreaticobiliary and lower gastrointestinal surgery. Methods: A systematic review was undertaken of the published literature. Searches were carried out on five bibliographical databases (Medline, Embase, The Cochrane Library, Cumulative Index for Nursing and Allied Health Literature and HealthSTAR) for English language articles published between 1993 and 1999. Articles were critically appraised by two independent reviewers and data on definition and measurement of anastomotic leak were extracted. Results: Ninety-seven studies were reviewed and a total of 56 separate definitions of anastomotic leak were identified at three sites: upper gastrointestinal (13 definitions), hepatopancreaticobiliary (14) and lower gastrointestinal (29). The majority of studies used a combination of clinical features and radiological investigations to define and detect anastomotic leak. Conclusion: There is no universally accepted definition of anastomotic leak at any site. The definitions and values used to measure anastomotic failure vary extensively and preclude accurate comparison of rates between studies and institutions.

Systematic review of the definition and measurement of anastomotic leak after gastrointestinal surgery.

Background Limited resources coupled with unlimited demand for healthcare mean that decisions have to be made regarding the allocation of scarce resources across competing interventions. Policy documents have advocated the importance of public views as one such criterion. In principle, the elicitation of public values represents a big step forward. However, for the exercise to be worthwhile, useful information must be obtained that is scientifically defensible, whilst decision-makers must be able and willing to use it. Aims and objectives The aim was to identify techniques that could be reasonably used to elicit public views on the provision of healthcare. Hence, the objectives were: (1) to identify research methods with the potential to take account of public views on the delivery of healthcare; (2) to identify criteria for assessing these methods; (3) to assess the methods identified according to the predefined criteria; (4) to assess the importance of public views vis-a-vis other criteria for setting priorities, as judged by a sample of decision-makers; (5) to make recommendations regarding the use of methods and future research. Methods A systematic literature review was carried out to identify methods for eliciting public views. Criteria currently used to evaluate such methods were identified. The methods identified were then evaluated according to predefined criteria. A questionnaire-based survey assessed the relative importance of public views vis-a-vis five other criteria for setting priorities: potential health gain; evidence of clinical effectiveness; budgetary impact; equity of access and health status inequalities; and quality of service. Two techniques were used: choice-based conjoint analysis and allocation of points technique. The questionnaire was sent to 143 participants. A subsample was followed up with a telephone interview. Results The methods identified were classified as quantitative or qualitative. RESULTS - QUANTITATIVE TECHNIQUES: Quantitative techniques, classified as ranking, rating or choice-based approaches, were evaluated according to eight criteria: validity; reproducibility; internal consistency; acceptability to respondents; cost (financial and administrative); theoretical basis; whether the technique offered a constrained choice; and whether the technique provided a strength of preference measure. Regarding ranking exercises, simple ranking exercises have proved popular, but their results are of limited use. The qualitative discriminant process has not been used to date in healthcare, but may be useful. Conjoint analysis ranking exercises did well against the above criteria. A number of rating scales were identified. The visual analogue scale has proved popular within the quality-adjusted life-year paradigm, but lacks constrained choice and may not measure strength of preference. However, conjoint analysis rating scales performed well. Methods identified for eliciting attitudes include Likert scales, the semantic differential technique, and the Guttman scale. These methods provide useful information, but do not consider strength of preference or the importance of different components within a total score. Satisfaction surveys have been frequently used to elicit public opinion. Researchers should ensure that they construct sensitive techniques, despite their limited use, or else use generic techniques where validity has already been established. Service quality (SERVQUAL) appears to be a potentially useful technique and its application should be researched. Three choice-based techniques with a limited application in healthcare are measure of value, the analytical hierarchical process and the allocation of points technique, while those more widely used, and which did well against the predefined criteria, include standard gamble, time trade-off, discrete choice conjoint analysis and willingness to pay. Little methodological work is currently available on the person trade-off. RESULTS - QUALITATIVE TECHNIQUES: Qualitative techniques were classified as either individual or group-based approaches. Individual approaches included one-to-one interviews, dyadic interviews, case study analyses, the Delphi technique and complaints procedures. Group-based methods included focus groups, concept mapping, citizens' juries, consensus panels, public meetings and nominal group techniques. Six assessment criteria were identified: validity; reliability; generalisability; objectivity; acceptability to respondents; and cost. Whilst all the methods have distinct strengths and weaknesses, there is a lot of ambiguity in the literature. Whether to use individual or group methods depends on the specific topic being discussed and the people being asked, but for both it is crucial that the interviewer/moderator remains as objective as possible. The most popular and widely used such methods were one-to-one interviews and focus groups. (ABSTRACT TRUNCATED)

Eliciting public preferences for healthcare: a systematic review of techniques.

Background Surgical adverse events contribute significantly to postoperative morbidity, yet the measurement and monitoring of events is often imprecise and of uncertain validity. Given the trend of decreasing length of hospital stay and the increase in use of innovative surgical techniques--particularly minimally invasive and endoscopic procedures--accurate measurement and monitoring of adverse events is crucial. Objectives The aim of this methodological review was to identify a selection of common and potentially avoidable surgical adverse events and to assess whether they could be reliably and validly measured, to review methods for monitoring their occurrence and to identify examples of effective monitoring systems for selected events. This review is a comprehensive attempt to examine the quality of the definition, measurement, reporting and monitoring of selected events that are known to cause significant postoperative morbidity and mortality. METHODS - SELECTION OF SURGICAL ADVERSE EVENTS: Four adverse events were selected on the basis of their frequency of occurrence and likelihood of evidence of measurement and monitoring: (1) surgical wound infection; (2) anastomotic leak; (3) deep vein thrombosis (DVT); (4) surgical mortality. Surgical wound infection and DVT are common events that cause significant postoperative morbidity. Anastomotic leak is a less common event, but risk of fatality is associated with delay in recognition, detection and investigation. Surgical mortality was selected because of the effort known to have been invested in developing systems for monitoring surgical death, both in the UK and internationally. Systems for monitoring surgical wound infection were also included in the review. METHODS - LITERATURE SEARCH: Thirty separate, systematic literature searches of core health and biomedical bibliographic databases (MEDLINE, EMBASE, CINAHL, HealthSTAR and the Cochrane Library) were conducted. The reference lists of retrieved articles were reviewed to locate additional articles. A matrix was developed whereby different literature and study designs were reviewed for each of the surgical adverse events. Each article eligible for inclusion was independently reviewed by two assessors. METHODS - CRITICAL APPRAISAL: Studies were appraised according to predetermined assessment criteria. Definitions and grading scales were assessed for: content, criterion and construct validity; repeatability; reproducibility; and practicality (surgical wound infection and anastomotic leak). Monitoring systems for surgical wound infection and surgical mortality were assessed on the following criteria: (1) coverage of the system; (2) whether or not denominator data were collected; (3) whether standard and agreed definitions were used; (4) inclusion of risk adjustment; (5) issues related to data collection; (6) postdischarge surveillance; (7) output in terms of feedback and wider dissemination. RESULTS - SURGICAL WOUND INFECTION: A total of 41 different definitions and 13 grading scales of surgical wound infection were identified from 82 studies. Definitions of surgical wound infection varied from presence of pus to complex definitions such as those proposed by the Centres for Disease Control in the USA. A small body of literature has been published on the content, criterion and construct validity of different definitions, and comparisons have been made against wound assessment scales and multidimensional indices. There are examples of comprehensive hospital-based monitoring systems of surgical wound infection, mainly under the auspices of nosocomial surveillance. To date, however, there is little evidence of systematic measurement and monitoring of surgical wound infection after hospital discharge. RESULTS - ANASTOMOTIC LEAK: Over 40 definitions of anastomotic leak were extracted from 107 studies of upper gastrointestinal, hepatopancreaticobiliary and lower gastrointestinal surgery. No formal evaluations were found that assessed the validity or reliability of definitions or severity scales of anastomotic leak. One definition was proposed during a national consensus workshop, but no evidence of its use was found in the surgical literature. The lack of a single definition or gold standard hampers comparison of postoperative anastomotic leak rates between studies and institutions. RESULTS - DEEP VEIN THROMBOSIS: Although a critical review of the DVT literature could not be completed within the realms of this review, it was evident that a number of new techniques for the detection and diagnosis of DVT have emerged in the last 20 years. The group recommends a separate review be undertaken of the different diagnostic tests to detect DVT. RESULTS - SURGICAL MORTALITY MONITORING SYSTEMS: The definition of surgical mortality is relatively consistent between monitoring systems, but duration of follow-up of death postdischarge varies considerably. The majority of systems report in-hospital mortality rates; only some have the potential to link deaths to national death registers. Risk assessment is an important factor and there should be a distinction between recording pre-intervention factors and postoperative complications. A variety of risk scoring systems was identified in the review. Factors associated with accurate and complete data collection include the employment of local, dedicated personnel, simple and structured prompts to ensure that clinical input is complete, and accurate and automated data capture and transfer. Conclusions The use of standardised, valid and reliable definitions is fundamental to the accurate measurement and monitoring of surgical adverse events. This review found inconsistency in the quality of reporting of postoperative adverse events, limiting accurate comparison of rates over time and between institutions. The duration of follow-up for individual events will vary according to their natural history and epidemiology. Although risk-adjusted aggregated rates can act as screening or warning systems for adverse events, attribution of whether events are avoidable or preventable will invariably require further investigation at the level of the individual, unit or department. CONCLUSIONS - RECOMMENDATIONS FOR RESEARCH: (1) A single, standard definition of surgical wound infection is needed so that comparisons over time and between departments and institutions are valid, accurate and useful. Surgeons and other healthcare professionals should consider adopting the 1992 Centers for Disease Control (CDC) definition for superficial incisional, deep incisional and organ/space surgical site infection for hospital monitoring programmes and surgical audits. There is a need for further methodological research into the performance of the CDC definition in the UK setting. (2) There is a need to formally assess the reliability of self-diagnosis of surgical wound infection by patients. (3) There is a need to assess formally the reliability of case ascertainment by infection control staff. (4) Work is needed to create and agree a standard, valid and reliable definition of anastomotic leak which is acceptable to surgeons. (5) A systematic review is needed of the different diagnostic tests for the diagnosis of DVT. (6) The following variables should be considered in any future DVT review: anatomical region (lower limb, upper limb, pelvis); patient presentation (symptomatic, asymptomatic); outcome of diagnostic test (successfully completed, inconclusive, technically inadequate, negative); length of follow-up; cost of test; whether or not serial screening was conducted; and recording of laboratory cut-off values for fibrinogen equivalent units. (7) A critical review is needed of the surgical risk scoring used in monitoring systems. (8) In the absence of automated linkage there is a need to explore the benefits and costs of monitoring in primary care. (9) The growing potential for automated linkage of data from different sources (including primary care, the private sector and death registers) needs to be explored as a means of improving the ascertainment of surgical complications, including death. This linkage needs to be within the terms of data protection, privacy and human rights legislation. (10) A review is needed of the extent of the use and efficiency of routine hospital data versus special collections or voluntary reporting.

/pdf/the-measurement-and-monitoring-of-surgical-adverse-events-5dmk6zpn4d.pdf

Elizabeth M. Russell

Papers

Systematic review of the definition and measurement of anastomotic leak after gastrointestinal surgery.

Eliciting public preferences for healthcare: a systematic review of techniques.

The measurement and monitoring of surgical adverse events.

A population based survey of women's experience of the menopause

The quality of measurement of surgical wound infection as the basis for monitoring: a systematic review