American Academy of Orthotists & Prosthetists

Link to Figures
Show Figures

Article Text
Figures
References

Send Link
Send HTML Article

RESEARCH FORUM--Conference Report: An Agenda for Medical Rehabilitation Outcomes Research

Marcus J. Fuhrer, PHD

ABSTRACT

Introduction

Providing timely and effective medical rehabilitation services for people disabled by disease, injury or congenital impairments can contribute materially to their subsequent health, independence and personal productivity. In the aggregate, therefore, medical rehabilitation services are a powerful means of combating disability which, according to some sources, is the principal public health-care problem in the United States (1).

The explosive growth of medical rehabilitation in the last 15 years or so has not been accompanied by a comparable increase in systematic evidence for the effectiveness of its practices. Most practices were developed during the course of service provision rather than as part of a formal research and development process. Few practices have been evaluated either for their efficacy in carefully controlled circumstances or for their effectiveness in typical clinical situations. This paucity of information is unacceptable in view of the mounting demands for the accountability of all health-care providers, and commitments by the disciplines that provide medical rehabilitation services to ground their practices firmly in clinical science.

Conference Background

Outcomes research is a priority for the National Center for Medical Rehabilitation Research (NCMRR). Established legislatively in the fall of 1990, NCMRR has the mission of fostering the development of scientific knowledge needed to enhance the health, productivity, independence and quality of life of people with disabilities. Research aimed at assessing the efficacy and effectiveness of medical rehabilitation practices is one of seven emphasis areas described in the Research Plan for the National Center for Medical Rehabilitation Research (2). The other emphasis areas are improving functional mobility; promoting behavioral adaptation to functional losses; developing improved assistive technology; understanding whole body system responses to physical impairments and functional changes; developing more precise methods of measuring impairments, disabilities, and societal and functional limitations; and training research scientists in the medical rehabilitation field.

A conference was conducted Aug. 29-31, 1994, to develop recommendations regarding needed initiatives in medical rehabilitation outcomes research. Organized by NCMRR and cosponsored with the Agency for Health Care Policy and Research (AHCPR), the conference was titled "An Agenda for Medical Rehabilitation Outcomes Research." The invited presenters had made important contributions to medical rehabilitation outcomes research and represented the spectrum of disciplines that collaborate in the conduct of medical rehabilitation services and research. Other participants represented federal agencies interested in medical rehabilitation services or research, professional organizations whose memberships provide rehabilitation services, voluntary health organizations that are advocates for the recipients of medical rehabilitation services or the National Advisory Board on Medical Rehabilitation Research.

To facilitate communication, presenters were asked to use the language of the International Classification of Impairments, Disabilities and Handicaps (ICIDH) (3). Presentations were grouped under four rubrics that concern issues applicable to many disabling conditions. They were:

• philosophic issues in medical rehabilitation outcomes research
• strategy and design issues in medical rehabilitation outcomes studies
• measurement of disability and handicap in medical rehabilitation outcomes studies
• quality of life and health status measurement in medical rehabilitation outcomes studies

These rubrics were the basis of forming four workgroups that were charged with developing recommendations for future research in their respective areas. Five additional workgroups generated recommendations relevant to various groupings of specific disabling conditions, including nervous system, cardiovascular, musculoskeletal, early development and aging-related conditions. The workgroups' recommendations were presented in a plenary session for comment by the attendees.

Recommendations

Several disclaimers are in order. 1) The order of the recommendations that follow does not indicate judgments about their priority. 2) The recommendations by no means imply that relevant research is wholly unavailable. 3) Participants in each of the workgroups appreciated that many aspects of their discussions and recommendations overlapped the four topical areas. 4) Participants recognized that to address the topics adequately, some considerations need to be raised that are not integral to the subject of outcomes research.

Workgroup 1: Philosophic Issues in Medical Rehabilitation Outcomes Research

Chairperson: John Whyte, MD, PHD, Moss Rehabilitation Research Institute
Rapporteur: D. Jesse Peters, MD, PHD, University of Washington School of Medicine

Seven major concerns were considered by this group. Three of these were assigned particularly high priority: values, cross-domain issues and epistemology. The recommendations dealing with these concerns were designed with several objectives in mind:

• Identify the social values that are implicit in rehabilitation outcomes.
• Understand the impact of provider values on access to services, timeliness of treatment and quality of care.
• Delineate the assumptions underlying different research approaches so that appropriate methods are chosen for problems of interest in the domains of pathology, impairment, disability, handicap, quality of life and other areas of importance.
• Develop a cadre of researchers who communicate and collaborate across domains.

Values. The workgroup recognized that outcomes are unavoidably value ladened and may differ in importance depending on who is judging them. Whether that individual is a consumer, payer or provider of services can make an enormous difference in the value attributed to particular outcomes. The challenge is to identify and, more dauntingly, integrate judgments about outcomes from the diverse perspectives of service recipients and their families, practitioners, payers, policy makers and researchers. This challenge is magnified if one considers possible variations in preferred outcomes among service recipients from different cultural, ethnic or socioeconomic backgrounds.

Recommendations to clarify some of the issues associated with values included:

• initiating a research program on the outcomes valued by different subcultures and other groupings of constituencies. At a minimum, consumers, providers and payers should be distinguished. Exploratory research to identify the values of these groups and survey research to determine variations in values both within and among these groups are two proposed initiatives.
• designing flexible approaches for weighing outcomes to make allowance for the values and preferences of individuals. This action would be a step toward addressing the person behind the disability (i.e., the issue of personhood in disablement).
• developing methods to increase consensus among various constituencies and cultural groups on valued rehabilitation outcomes and, at a minimum, promoting dialogue through mechanisms such as a consensus conference
• developing methods for obtaining valid proxy judgments about valued rehabilitation outcomes for people who cannot represent themselves
• supporting research about the impact of provider values on the provision and outcomes of medical rehabilitation services

Cross-domain issues. The rehabilitation outcomes domains of pathology, impairment, disability and handicap are often approached separately and in isolation by researchers. Within each domain, different terminologies and methodologies are used, and the emphasis is on the deficits and not the strengths of people with disabilities.

The workgroup recommended several undertakings to address these issues, including:

• research training grants to develop a cadre of researchers who can work effectively in interdisciplinary research centers
• consensus conferences to produce agreement on boundaries among domains, terminology and how to take into account people's abilities rather than only their deficits
• developing conceptual models of the disablement process and the empirical testing of such models, making provisions to gather data across multiple domains
• developing measurement and analytic tools appropriate to multidomain analysis, using collaborative and interdisciplinary methods

It also was recommended that applicants for research funding to agencies like NCMRR should be responsible for indicating how the issues they are addressing fit into the macro model of disablement.

Epistemology (philosophical assumptions). The workgroup began its deliberations on this issue by acknowledging that different implicit assumptions of science underlie different approaches to research. These assumptions are rarely made explicit yet they can be influential in shaping topics of study, framing research questions and specifying funding priorities. The result is that the choice of research foci may be unduly influenced by such implicit assumptions rather than by considering the work's clinical or theoretical significance.

Recommendations relevant to the issue of epistemology included:

• reviewing existing literature relevant to rehabilitation research to identify underlying assumptions about the research methods being used and, through studies, developing guidelines to help link phenomena of interest in rehabilitation outcomes with appropriate research methods
• supporting collaboration with disciplines traditionally considered to be outside of rehabilitation (e.g., anthropology, sociology, education, learning theory and phenomonological psychology) to explore how the characteristic approaches of these disciplines may complement more traditional approaches to medical rehabilitation research.

Other observations of this group were:

• Operational health policies may result in different and sometimes discriminatory eligibility criteria and access to resources. Outcomes research may influence such policies, exemplifying the linkage among outcomes research, health policy and ethical issues.
• Identifying comorbidities, designating confounding factors and finding ways to deal with longitudinal and developmental issues in research designs are methodological challenges that are not handled well in most available studies of medical rehabilitation outcomes.
• Program models are needed that promote interdisciplinary familiarity and interaction in medical rehabilitation education, training and research.

Workgroup 2: Strategy and Design Issues in Medical Rehabilitation Outcomes Studies

Chairperson: Kenneth Ottenbacher, PHD, State University of New York at Buffalo
Rapporteur: Judith Barr, PHD, Northeastern UniversityMeasures. This workgroup emphasized the importance of obtaining valid, reliable and sensitive outcomes measures for comparing the effectiveness of alternative rehabilitation interventions, systems of care, methods of financing and consumers satisfaction and preferences. An important preliminary step is to identify outcomes measures that possess sufficient sensitivity to detect clinically significant differences, if present, among alternative interventions, settings of care, methods of financing and types of practitioners. Considering consumers' satisfaction and preferences in developing sets of outcome measures was emphasized.

The resulting recommendation was to encourage continuing development of appropriate outcomes measures to assess the effectiveness of medical rehabilitation programs and practices while also examining the interaction of different methods of intervention, means of organizing services, financing methods and value systems.

Community-based studies aimed at hypothesis testing. Predicting the likely outcomes of rehabilitation services depends partly upon accurate knowledge of the populations for whom those services are provided. That knowledge, combined with theoretical understanding of the interventions, provides the groundwork for experimental studies to test hypotheses about the effects of interventions. This viewpoint led to the recommendation that studies addressing key methodological issues in medical rehabilitation outcomes research should be supported. Examples are studies to determine appropriate study designs for population-based studies, multicenter trials, registries and investigations to replicate key conclusions of clinical trials that are based on relatively small samples.

Controlled studies. This workgroup observed that medical rehabilitation researchers have produced many descriptive studies of outcomes but very few controlled trials. Additionally, very few studies have documented the economic implications of medical rehabilitation practices. Much greater effort is needed to provide evidence for the effectiveness or efficacy of defined rehabilitative interventions by comparing them to practical service alternatives. The recommendations to address these concerns were:

• Sponsor randomized controlled trials or rigorous quasi-experimental studies of defined rehabilitation programs and selected practices within programs.
• Encourage the use of economic analyses such as cost-effectiveness and cost-benefit analysis.
• Exploit clinical databases to analyze variations in outcomes for similar patient groups who received different treatments.

Better descriptions of interventions and independent variables. The group's report stated that medical rehabilitation interventions and the service delivery systems of which they are a part are poorly and inconsistently described in reports of outcome studies. Well-documented descriptions of services also are missing for most ongoing clinical databases. Work needs to be done, therefore, to develop theory-based models and classification systems of rehabilitation interventions and practice models that are uniformly applicable to a variety of services and are related to the outcomes goals of medical rehabilitation. In this regard, it was recommended that

• uniform systems to classify interventions be developed and pilot-tested
• procedures for taking into account the severity of patients' conditions be developed and used in individual trials and multicenter databases
• theory-based models of rehabilitation service delivery and practices be developed and empirically tested

Registries and clinical databases. This workgroup addressed the question of how existing databases and registries can be used to the greatest advantage to obtain knowledge of medical rehabilitation outcomes. Attention was directed to the need for better data elements, including more details about impairments, broader coverage of outcomes, more refined information about the severity of conditions, and more complete characterizations of interventions and service delivery systems. The resulting recommendations were:

• support the development of effective methodologies for using clinical rehabilitation databases to study practice variations, develop hypotheses for subsequent Studies and contribute to outcomes management procedures.
• encourage the development of uniform database elements and standards

Workgroup 3: Measurement of Disability and Handicap in Medical Rehabilitation Outcomes Studies

Chairperson: Byron Hamilton, MD, PHD, VA Medical Center, Durham, N. C.
Rapporteur: Diana Rintala, PHD, Baylor College of Medicine

Conceptualization and terminology. The group discussed defining and measuring disability using dimensions other than independence/dependence (e.g., self-assessed difficulty in performing an activity). Imprecision between the boundaries of impairment and disability and disability and handicap were discussed.

The desirability of making "functional limitations" a separate domain that would be interposed between impairments and disabilities was considered. For example, the functional limitation of "not able to grasp and release" would be relevant on the one hand to the impairment of "weakness" and, on the other hand, to the disability of "not being able to eat by oneself."

Another topic was how much various measures of disability tend to converge, both in terms of their design and the findings they yield. Consequently, several existing scales may be amenable to useful transformations. For example, since similar logit values exist for like items on the Patient Evaluation Conference System (PECS) and the Functional Independence Measure (FIM), the findings from both scales often result in the same conclusions. This observation underscores the possibility of developing a comprehensive assessment tool that contains existing scale items in a usable format.

Applications. The workgroup pointed to the unavailability of a uniformly useful measure of instrumental activities of daily living (IADLs). Developing a measure that can be used for people with various disabling conditions was viewed as being a worthwhile objective. Developing a supplement to scales like the FIM to reflect the cost of resources for supervising cognitively impaired individuals also was deemed beneficial. Seeking shorter and less cumbersome instruments for clinical and research applications was emphasized as well.

It was acknowledged that measurement instruments should be adapted for use cross-culturally. Translating instruments into other languages is an obvious step in that regard. The instruments then could be validated in these cultures and modified if necessary. Study findings might show that different norms exist for different cultures.

The workgroup saw merit in particular functional assessment instruments being used across different fields of medicine. The measurement of disability associated with arthritis may be facilitated by using instruments like the FIM, which is widely used in neurological conditions. Conversely, rehabilitation specialists could benefit from considering measures used in other fields (e.g., for assessing social disabilities). The need for valid measures of disability and handicap in multiple-system injuries or diseases also was considered. The reliability of measures of disability and handicap that are based on telephone contacts with respondents was discussed. Finally, it was noted that it would be useful if detailed information on all functional assessment instruments were available in a central location, perhaps at the National Library of Medicine.

Three categories of recommendations were formulated: conceptual advances, measurement development and applications of measures:

Conceptual Advances

• Seek new dimensions for defining disability.
• Identify and standardize common characteristics of disability and handicap scales.
• Clarify the interface between impairment and disability, and between disability and handicap.

Measurement Development

• Develop uniform measures of IADLs.
• Seek more sensitive measures of demands associated with supervising individuals because of their cognitive impairments or safety concerns.
• Improve the practicality of functional assessment instruments.
• Pursue the cross-cultural validation of measures of disability.
• Encourage uniformity in the use of functional assessment instruments across medical disciplines.
• Develop valid measures of disability and handicap in multiple system injuries or diseases such as AIDS.
• Determine the reliability of functional assessment instruments when administered using the telephone.
• Develop measures of the social and physical environments that contribute to handicap.

Applications of Measures

• Develop an understanding of the relationships among pathology, impairment, disability and handicap. This goal may be approached first by using existing techniques, such as structured equation modeling, and then by using computer simulations to confirm the resulting conceptualizations.

Workgroup 4: Measurement of quality of Life and Health Status in Medical Rehabilitation Outcomes Studies

Chairperson: Denise Tate, PHD, University of Michigan
Rapporteur: James S. Krause, PHD, Shepard Spinal Center

Clarification of concepts and development of measures. This workgroup commented on the difficulty of its charge because of the diversity and number of quality-of-life measures and because of the multitude of populations and settings in which these measures have been applied. Consequently, an urgent need was identified to consolidate information regarding quality-of-life measures. Continuing effort also was called for to clarify the concept of quality of life and its principal components. Following a great deal of additional discussion, three primary recommendations were made:

• Review, integrate and consolidate the vast literature on quality of life as it applies to the field of medical rehabilitation. Following that effort, conduct a consensus conference to depict the state of the art in quality of life and health outcomes measurement as applied to the evaluation of medical rehabilitation services.
• Support development of a classification system for quality of life and health outcomes. This classification scheme may be developed by a theoretical analysis based on existing literature or by a large-scale empirical study of quality of life and health status using data reduction techniques such as cluster- or factor-analysis. Special attention needs to be directed to components that generalize across disabling conditions as well as components that are relevant to particular disabling conditions.
• Encourage continuing development of instruments to measure quality of life and health outcomes for different rehabilitation populations. These measures may adapt existing instruments or create new ones. In either case, they should be consistent with the classification system discussed above and with the highest standards of instrument development.

Conclusion

If the design and conduct of definitive medical rehabilitation outcomes studies were relatively straightforward tasks, much more evidence would exist about the effectiveness and efficacy of medical rehabilitation practices. A host of difficulties confront investigators wishing to undertake such studies. Withholding services to constitute control groups is generally unacceptable; individual practices are embedded in larger arrays of services that are difficult to specify with precision; pathobiological processes underlying impairments and disabilities may be changing in ways that are not affected directly by rehabilitation intervention; and it is frequently impossible to determine if service objectives were achieved until a relatively long time after services are completed. Nevertheless, it can be expected that the amount and quality of outcomes data will continue to increase.

Nothing less is acceptable to (1) service consumers who want to know about the implications of treatment options; (2) service purchasers who will continue to insist upon evidence for the cost effectiveness of their expenditures; (3) service providers who need well-substantiated guidelines for patient management decisions; and (4) researchers who recognize that authoritative studies of the effectiveness of contemporary practices and of the efficacy of newly developed ones are pivotal for advancing the field of medical rehabilitation.

Acknowledgment

Appreciation is expressed to Dr. Patricia J. Villani for her assistance in organizing the recommendations of the workgroups and in reviewing this manuscript.

MARCUS J. FUHRER, PHD, is director of the National Center for Medical Rehabilitation Research, National Institute of Child Health and Human Development, National Institutes of Health, NCMRR/NICHD, Executive Building/Room 2A03, 6100 Executive Blvd., MSC 7510, Bethesda, MD 20892.

References:

1. Pope AM, Tarloff AR [Eds]. Disability in America. Washington, D.C., National Academy Press, 1991.
2. Research plan for the National Center for Medical Rehabilitation Research. Bethesda, Md., National Institute of Child Health and Human Development, National Institutes of Health, 1993.
3. World Health Organization: International classification of impairments, disabilities and handicaps: a manual of classification relating to the consequences of disease. Geneva, World Health Organization, 1980.