/pdf/research-design-qualitative-quantitative-and-mixed-methods-1m92m1x5gg.pdf

Research Design: Qualitative, Quantitative, and Mixed Methods Approaches

The goal of treatments for multiple sclerosis (MS) and other chronic neurological conditions that produce morbidity but have limited effect on mortality is to reduce disease impact on patients’ lives and to assure that interventions result in more good than harm. A measure of our success in reaching these goals can be made only with direct information from patients about how they experience the illness and the effects of treatments. Patient-derived data are increasingly accepted as an essential assessment domain in clinical research and treatment. Research shows that measures of patient perception and clinician-derived data are not redundant. ± 3 Observation of patient functioning in the clinic is not sufficient in itself since it has been determined that patient functioning in the artificial setting of the treatment center is not always duplicated at home. The approach to patient assessment that is considered relevant in monitoring the consequences of diseases and their treatments is known as `health-related quality of life’. Quality of life is one domain of health as defined by the World Health Organization (WHO). Health-related quality of life (HRQoL) is a discrete component of general quality of life. While general quality of life can be affected by many factors beyond the scope of health care including economic instability, civil unrest or poor environment, these have only an indirect relationship with HRQoL and are not included in its definition. While such factors as equal opportunity and social security are important to community health, these extend beyond the more immediate goal of treating the sick. This thinking led to the HRQoL definition: `Quality of life’ in clinical medicine represents the functional effect of an illness and its consequent therapy upon a patient, as perceived by the patient’. Although there is variation in terminology, this construct includes four broad domains: physical and occupational function, psychological function, social interaction and somatic sensation. Several operational characteristics of HRQoL assessment help to further define the construct. First and foremost, HRQoL is subjective. As Schipper and his colleagues explain `. . . in clinical medicine the ultimate observer of the experiment is not a dispassionate third party but a most intimately involved patient’. They note that since the goal of treatment is to minimize the manifest consequences of disease, HRQoL represents `the final common pathway of all the physiological, psychological and social inputs into the therapeutic process’. The second characteristic of HRQoL is that it is multifactorial. Having defined HRQoL as the integration of four domains, it is important to assure that patients’ daily experiences in each domain are explored in the questionnaire, albeit in a manner that is parsimonious and minimizes respondent burden. The third characteristic is self-administration: because HRQoL is subjective, there is concern that external administration would in some way influence patient report. The final characteristic is that HRQoL is time variable; it fluctuates. Progress in HRQoL assessment in research or clinical monitoring is evolutionary and, arguably, comparable to the increasing use of MRI and immunological markers in quantifying MS severity. Data derived from HRQoL measures are used for three general purposes. The first, most commonly used in research, is to classify or group patients by levels of disease severity. The second is to monitor for change in status that indicates a need for modifying the treatment plan or to predict the health of subjects at a future point in time. The third use is as an outcome variable in clinical research. HRQoL assessment is increasingly accepted as an outcome measure in MS research for disease-modifying treatments ± 11 and symptom management. ± 14 HRQoL assessment is less commonly used to monitor patient well-being as part of the usual process of care. This can be done routinely, in much the same way that upper and lower extremity monitoring is done using the NineHole Peg Test or Timed Twenty-five Foot Walk. However, challenges to using HRQoL monitoring in clinical practice are the limited availability of instruments proved to be sensitive to change in individual patients over time, easily interpreted by clinicians, accepted by patients as relevant, and that require limited time to complete. Quality of life researchers are actively engaged in improving the science of HRQoL interpretation at the individual level; it remains an emerging field. In this issue, Shawaryn describes a measure of illness intrusiveness that has good potential for use as an HRQoL screening instrument in the clinical setting. Her findings clearly demonstrate that the physical and cognitive aspects of MS affect HRQoL, each in different ways. Her findings hold for HRQoL as measured directly by the Multiple Sclerosis Quality of Life Inventory and indirectly, by the Illness Intrusiveness Rating Scale. This later measure has a number of benefits for clinical use including brevity, face validity, clinical relevance and patient acceptability. Even though there are not yet standard methods for determining when a change in this or other patient self-reported measures indicates a need to review or revise a patient’s treatment plan, these measures have an important use for physicians providing care to patients with MS and other chronic conditions. Routine use of such measures allows physicians to review, quickly and systematically, patient functioning and concerns and to help patients to become active participants in their care.

Health-related quality of life.

Changes in health policy have underlined the importance of evidence-based clinical practice and rigorous evaluation of patient-based outcomes. As patient-based outcome measurement is particularly important in treatment trials of multiple sclerosis, a number of disease-specific instruments have been developed recently. One limitation of these instruments is that none was developed using the standard psychometric approach of reducing a large item pool generated from people with multiple sclerosis. Consequently, an outcome measure for clinical trials of multiple sclerosis that is disease specific and combines patient perspective with rigorous psychometric methods will complement existing instruments. The aim of this study was to develop such a measure. Standard psychometric methods were used. A pool of 129 questionnaire items was generated from interviews with 30 people with multiple sclerosis, expert opinion and literature review. The questionnaire was administered by postal survey to 1530 people selected randomly from the Multiple Sclerosis Society membership database. Redundant items and those with limited measurement properties were removed. The remaining items (n = 41) were grouped into scales using factor analysis, and then refined to form the Multiple Sclerosis Impact Scale (MSIS-29), an instrument measuring the physical (20 items) and psychological (nine items) impact of multiple sclerosis. Five psychometric properties of the MSIS-29 (data quality, scaling assumptions, acceptability, reliability and validity) were examined in a separate postal survey of 1250 Multiple Sclerosis Society members. A preliminary responsiveness study of the MSIS-29 was undertaken in 55 people admitted for rehabilitation and intravenous steroid treatment of relapses. The MSIS-29 satisfied all psychometric criteria. Data quality was excellent, missing data were low (maximum 3.9%), item test-re-test reliability was high (r = 0.65-0.90) and scale scores could be generated for >98% of respondents. Item descriptive statistics, item convergent and discriminant validity, and factor analysis indicated that it was legitimate to generate scores for MSIS-29 scales by summing items. MSIS-29 scales showed good variability, small floor and ceiling effects, high internal consistency (Cronbach's alpha <or=0.91) and high test-re-test reliability (intraclass correlation <or=0.87). Correlations with other measures and the analysis of group differences provided evidence that the MSIS-29 measures the physical and psychological impact of multiple sclerosis. Effect sizes (physical scale = 0.82, psychological scale = 0.66) demonstrated preliminary evidence of good responsiveness. These results indicate the MSIS-29 is a clinically useful and scientifically sound patient-based outcome measure of the impact of multiple sclerosis suitable for clinical trials and epidemiological studies.

/pdf/the-multiple-sclerosis-impact-scale-msis-29-a-new-patient-2ox3xdoly7.pdf

The Multiple Sclerosis Impact Scale (MSIS-29)A new patient-based outcome measure

Objective: To improve, with the aid of psychometric analysis, the Balance Evaluation Systems Test (BESTest), a tool designed to analyse several postural control systems that may contribute to poor functional balance in adults. Methods: Performance of the BESTest was examined in a convenience sample of 115 consecutive adult patients with diverse neurological diagnoses and disease severity, referred to rehabilitation for balance disorders. Factor (both explorative and confirmatory) and Rasch analysis were used to process the data in order to produce a new, reduced and coherent balance measurement tool. Results: Factor analysis selected 24 out of the 36 original BESTest items likely to represent the unidimensional construct of “dynamic balance”. Rasch analysis was then used to: (i) improve the rating categories, and (ii) delete 10 items (misfitting or showing local dependency). The model consisting of the remaining 14 tasks was verified with confirmatory factor analysis to meet the stringent requirements of modern measurement. Conclusion: The new 14-item scale (dubbed mini-BESTest) focuses on dynamic balance, can be conducted in 10–15 min, and contains items belonging evenly to 4 of the 6 sections from the original BESTest. Further studies are needed to confirm the usefulness of the mini-BESTest in clinical settings.

/pdf/using-psychometric-techniques-to-improve-the-balance-2jwd5s1bcl.pdf

Using psychometric techniques to improve the balance evaluation systems test: the mini-bestest

Objective: To develop a patient-based measure of walking ability in MS. Methods: Twelve items describing the impact of MS on walking (12-Item MS Walking Scale [MSWS-12]) were generated from 30 patient interviews, expert opinion, and literature review. Preliminary psychometric evaluation (data quality, scaling assumptions, acceptability, reliability, validity) was undertaken in the data generated by 602 people from the MS Society membership database. Further psychometric evaluation (including comprehensive validity assessment, responsiveness, and relative efficiency) was conducted in two hospital-based samples: people with primary progressive MS (PPMS; n = 78) and people with relapses admitted for IV steroid treatment (n = 54). Results: In all samples, missing data were low (≤3.8%), item test–retest reproducibility was high (≥0.78), scaling assumptions were satisfied, and reliability was high (≥0.94). Correlations between the MSWS-12 and other scales were consistent with a priori hypotheses. The MSWS-12 (relative efficiency = 1.0) was more responsive than the Functional Assessment of Multiple Sclerosis mobility scale (0.72), the 36-Item Short Form Health Survey physical functioning scale (0.33), the Expanded Disability Status Scale (0.03), the 25-ft Timed Walk Test (0.44), and Guy’s Neurologic Disability Scale lower limb disability item (0.10). Conclusions: The MSWS-12 satisfies standard criteria as a reliable and valid patient-based measure of the impact of MS on walking. In these samples, the MSWS-12 was more responsive than other walking-based scales.

Measuring the impact of MS on walking ability: the 12-Item MS Walking Scale (MSWS-12).

BACKGROUND The importance of evaluating disability outcome measures is well recognised. The Functional Independence Measure (FIM) was developed to be a more comprehensive and “sensitive” measure of disability than the Barthel Index (BI). Although the FIM is widely used and has been shown to be reliable and valid, there is limited information about its responsiveness, particularly in comparison with the BI. This study compares the appropriateness and responsiveness of these two disability measures in patients with multiple sclerosis and stroke. METHODS Patients with multiple sclerosis (n=201) and poststroke (n=82) patients undergoing inpatient neurorehabilitation were studied. Admission and discharge scores were generated for the BI and the three scales of the FIM (total, motor, and cognitive). Appropriateness of the measures to the study samples was determined by examining score distributions, floor and ceiling effects. Responsiveness was determined using an effect size calculation. RESULTS The BI, FIM total, and FIM motor scales show good variability and have small floor and ceiling effects in the study samples. The FIM cognitive scale showed a notable ceiling effect in patients with multiple sclerosis. Comparable effect sizes were found for the BI, and two FIM scales (total and motor) in both patients with multiple sclerosis and stroke patients. CONCLUSION All measures were appropriate to the study sample. The FIM cognitive scale, however, has limited usefulness as an outcome measure in progressive multiple sclerosis. The BI, FIM total, and FIM motor scales show similar responsiveness, suggesting that both the FIM total and FIM motor scales have no advantage over the BI in evaluating change.

https://jnnp.bmj.com/content/jnnp/66/4/480.full.pdf

Measuring change in disability after inpatient rehabilitation: comparison of the responsiveness of the Barthel Index and the Functional Independence Measure

Objective: To compare the 10-item Barthel Index (BI), 18-item Functional Independence Measure (FIM), and 30-item Functional Independence Measure + Functional Assessment Measure (FIM+FAM) as measures of disability outcomes for neurologic rehabilitation.

Methods: A total of 149 inpatients from two rehabilitation units in South England specializing in neurologic disorders were studied. Traditional psychometric methods were used to evaluate and compare acceptability (score distributions), reliability (internal consistency, intrarater reproducibility), validity (concurrent, convergent and discriminant construct), and responsiveness (standardized response mean).

Results: All three rating scales satisfied recommended criteria for reliable and valid measurement of disability, and are acceptable and responsive in this study sample. The FIM and FIM+FAM total scales are psychometrically similar measures of global disability. The BI, FIM, and FIM+FAM motor scales are psychometrically similar measures of physical disability. The FIM and FIM+FAM cognitive scales are psychometrically similar measures of physical disability.

Conclusions: In the sample studied, the BI, FIM, FIM+FAM have similar measurement properties, when examined using traditional psychometric analyses. Although instruments with more items and item response categories generate more qualitative information about an outcome, they may not improve its measurement. Results highlight the importance of using recognized techniques of scale construction to develop health outcome measures.

Evidence-based measurement: which disability scale for neurologic rehabilitation?

One of the primary aims of rehabilitation for patients with multiple sclerosis (MS) is to reduce their levels of disability and handicap, yet little systematic research into the outcomes of this intervention has been undertaken. This stratified, randomized, wait-list controlled study evaluated the effectiveness of a short period of multidisciplinary inpatient rehabilitation in people with MS. Sixty-six patients in the progressive phase of the disease were assessed at 0 and 6 weeks with validated measures of impairment (Expanded Disability Status Scale and Functional Systems), disability (Functional Independence Measure), and handicap (London Handicap Scale). Both groups were comparable in terms of age, sex, disease duration and severity, disability, and handicap. At the end of 6 weeks, although the level of impairment in both groups remained the same, those who participated in a short period of inpatient rehabilitation (average of 25 days) significantly improved their level of disability and handicap compared with those in the wait-list control group. Despite unchanging impairment, inpatient rehabilitation resulted in reduced disability and handicap in patients with progressive MS.

The impact of inpatient rehabilitation on progressive multiple sclerosis

Objective: To consider clinical issues surrounding goal setting in neurological rehabilitation, and to identify priorities for future research.Participants: Twenty-four rehabilitation professionals were invited to attend because they had taught or published on the topic of goal setting. In addition two patient groups were represented.Evidence: (1) The results of a systematic literature review, (2) presentations given during the two-day conference by investigators working within the field of goal setting, (3) questions and statements from conference attendees during open discussion, (4) a report initially formulated by a panel composed of four of the conference attendees, and then circulated to all attendees for comment, (5) views of the conference attendees gathered using a modified Delphi technique.Consensus: There were significant areas of consensus about goal setting. The Delphi studies highlighted and confirmed these areas of general agreement with consensus that goal setting is a core component of th...

Areas of consensus and controversy about goal setting in rehabilitation: a conference report

Background Falls are a significant issue in people with multiple sclerosis (MS), with research demonstrating fall rates of more than 50%.

Purpose The purpose of this study was to evaluate the risk factors associated with falling in people with MS.

Data Sources Mixed search methods were used, including computer-based and manual searches. Additionally, hand searches of reference lists and conference abstracts were performed. All literature published from the source's earliest date to January 2012 was included; only full-text English-language sources (or those where a translation was available) were included.

Study Selection Eligibility criteria specified articles evaluating any aspect of fall risk in adults with a confirmed MS diagnosis, where the incidence of falling as determined by prospective or retrospective participant report was included.

Data Extraction Data were extracted independently by 2 reviewers using a written protocol and standardized extraction documentation. Detailed assessment of each article was independently undertaken by both reviewers, including assessment of study quality using an adaptation of the Newcastle Ottawa Scale plus extraction of key data (participant characteristics, fall incidence, and outcomes).

Data Synthesis The final review comprised 8 articles with a total of 1,929 participants; 1,037 (53.75%) were classified as fallers. Eighteen different risk factors were assessed within the included studies. Meta-analysis demonstrated an increase in fall risk associated with impairments of balance and cognition, progressive MS, and use of a mobility aid. Narrative review of the qualitative articles and those factors where meta-analysis was not possible also was undertaken.

Limitations Variation in assessment, analysis, and reporting methods allowed meta-analysis for only 4 factors.

Conclusion There is limited evidence of the factors associated with fall risk in people with MS. Further methodologically robust studies are needed.

/pdf/identification-of-risk-factors-for-falls-in-multiple-4b4qbmc1es.pdf

Jennifer Freeman

Papers

Measuring change in disability after inpatient rehabilitation: comparison of the responsiveness of the Barthel Index and the Functional Independence Measure

Evidence-based measurement: which disability scale for neurologic rehabilitation?

The impact of inpatient rehabilitation on progressive multiple sclerosis

Areas of consensus and controversy about goal setting in rehabilitation: a conference report

Identification of Risk Factors for Falls in Multiple Sclerosis: A Systematic Review and Meta-Analysis