Findings From the Recent Academy State-of-the-Science Conference on Lower Limb Prosthetic Outcomes

This State of the Science Conference was convened to examine the body of scientific knowledge related to Lower Limb Prosthetic (LLP) Outcome Measures. The meeting was held September 7-9, 2005 in Chicago, Illinois. Funding for this conference was provided by the American Academy of Orthotists and Prosthetists through a grant from the US Department of Education. Experts in the field were invited to offer their insight and expertise in summarizing the evidence and identifying key areas for future research. It was the goal of the state-of-thescience conference to examine, debate, and answer these questions:

1. What validated instruments are available in English to measure global lower limb prosthetic outcomes?

2. What do these instruments attempt to measure?

3. What are the relative strengths and weaknesses of current instruments?

4. What are the barriers to clinical application of these tools?

5. In light of the literature review and the panel’s discussion, what are the primary future research priorities?

To address questions one and two, a comprehensive written review of the literature was completed by Elizabeth Condie, PT, FCSP, and colleagues. Copies of the relevant articles selected for review were made available to the participants prior to the meeting. In preparation for the conference, participants were asked to review the literature prior to the meeting. Selected participants were asked to present their experience with specific tools (question three) while other participants were asked to provide certain perspectives on the clinical application of outcome measurement tools (question four).

Participants included those with clinical and research backgrounds in prosthetics and in areas related to health outcomes and epidemiology. They included: Laura A Miller, CP, PhD, Chair (Rehabilitation Institute of Chicago), Julia McCay, CP, Vice-Chair (Scheck & Siress O&P), David Boone, CP,LP,MPH,PhD (Cyma Corporation), Elizabeth Condie, PT, FCSP (University of Strathclyde), Deanna Fish, CPO (Linkia), Robert Gailey, PhD, PT (University of Miami School of Medicine), Steven Gard, PhD (Northwestern University ), Christiane Gauthier- Gagnon, MS (University of Montreal), Brian Hafner, PhD (Prosthetics Research Study), Allen W. Heinemann, PhD (Northwestern University and Rehabilitation Institute of Chicago), Todd Kuiken, MD, PhD (Rehabilitation Institute of Chicago and Northwestern University), John Michael, CPO, FAAOP (CPO Services, Inc), Paul Pasquina, MD (Walter Reed Army Medical Center), Michael Pinzur, MD (Loyola University Medical Center), Kathryn Roach, PhD, PT (University of Miami), Douglas G Smith, MD (University of Washington, Prosthetics Research Study, and The Amputee Coalition of America), and Jack Uellendahl, CPO (Hanger P&O).

One of the largest accomplishments of the State-of-the-Science meeting was the comprehensive literature review performed. The literature review included recent publications only (within 10 years), in English, and related to lower limb prosthetics. It was found that a multitude of measures are currently in use in prosthetic rehabilitation, however, there is not an agreement on a “gold standard”. Reliability, validity and scaling data as reported in the included 28 studies were evaluated. Some recurring methodological flaws were noted. These included inconsistent use of numbers and percentages, insufficient information on patient populations, no analysis of difference between response and non-response groups, and conclusions that tend to overstate findings. No single measure emerged as being universally appropriate. The overall agreement of the group is that there may not be an over all “gold standard” because outcome measurement tools should be selected based on the variable to be measured and the specific population group.

Many of the published studies are complex and difficult to read, which may limit the transfer of research findings into clinical practice. Terminology was indicated as a barrier to efficient application of outcome measurement tools because it can be inconsistent and confusing. To ensure that all participants were using the same terminology in discussing LLP outcomes, two individuals presented on terminology. Christiane Gauthier-Gagnon, MS, recommended the WHO standards of terminology for disability. It was conveyed that it is important to understand the influence of personal factors and environmental factors related to body function and structure, activities, and participation when describing and constructing measurement tools. Measurement tools can evaluate either the actual level of performance or perceived performance and can also be generic or population specific.

Kathryn Roach, PhD, PT, continued the discussion on terminology related to health outcomes by noting that various measurements of outcomes can be recorded. The quality of the information provided by an outcome measure depends in part on psychometric properties such as the level of measurement (e.g. type of data such as categories and scores), reliability (differences between scores are due to real differences), validity (measuring the intended point of interest), and responsiveness to change (can a measure respond to an intervention?). The group collectively agreed that this terminology is often misused and suggested two references (2,3) for those interested in conducting outcome measurements in the field of LLP.

Some of the participants spoke about tools they developed or used extensively, including the AMP, PPA-LCI, PEQ, OPUS, and clinical gait analysis. One primary conclusion that emerged from the discussion regarding the strengths and weaknesses of outcome measures in LLP was that although there are many tools that have been validated in one situation, this does not mean they can be used in any situation. The tool must be validated for the specific application (question to be answered and population to be studied). An article on how to choose an outcome measurement tool using an evidence-based approach (not limited to the area of LLP) was recommended during the discussion (4). Prior to developing new prosthetic outcome tools, the group recommended first investigating current tools, both in our field and in others, and how they could be applied to LLP in a validated and reliable manner, with a clear definition of the outcome to be evaluated. As an example, there has been substantial investigation into how falls impact the geriatric population, but these results have not been confirmed in the lower limb amputee population.

Other participants presented issues related to clinical barriers in applying outcome measurement tools. Regarding this topic, the group then agreed the principle clinical barriers to implementing outcome measures on a routine basis are: the demand on clinicians’ time (having the time to apply the tool), recruiting an adequate population for studies (especially long-term studies where resources are tight and patient attrition due to disease or death is likely), potential differences between the patient’s and clinician’s goals (such as restoring walking function versus cosmetic appearance), the technology required to perform the test (equipment, software, etc); finances (funding opportunities), concerns about poorly constructed research studies, and concern about justifying the effort required (how do we show it is financially worthwhile to spend money and time on outcome measures unless we can convince clinicians that they can get a return on that investment from improved patient care or consistent insurer payment?).

The participants agreed upon five Research Priorities and their order of importance. The primary future research questions were then categorized into these five priority areas:

1. Health Care Economic Impact

2. Rehabilitation Guidance Tools

3. Activity Limitation/Performance Measures

4. Pain and Comfort

5. Quality of Life

Health Care Economic Impact was agreed to be the most important category, with Rehabilitation Guidance Tools second. The primary research questions relative to each of the priority categories were determined and are listed below:

1. Health Care Economic Impact

• How does patient management and prosthesis prescription affect health care costs over time?

2. Rehabilitation Guidance Tools

• What existing tools can be used to predict outcomes? (e.g. AMP and others)

• What are the clinical indicators that can maximize prosthetic rehabilitation?

• Can existing tools be used to measure pre-morbid function?

3. Activity Limitation/Performance Measures

• What measurements are responsive to changes in __________?

• Improvements

• Deficits

• Physical therapy interactions

• Componentry

• Time

• Health status

• Population groups

4. Pain and Comfort

• How can socket comfort and performance be evaluated and measured?

• How does prosthetic use correlate to secondary conditions or pain, including but not limited to:

• Back pain?

• Degenerative joint disease?

• Diabetic foot complications?

• Residual limb pain?

• Phantom pain?

• Hip pain?

• Osteoporosis?

5. Quality of Life (QQL)

• How do QOL measures correlate to performance measures?

• What constructs in existing QOL tools are addressed in various prosthetic populations?

The group stressed that overlap across these priority categories is to be expected, and in many cases, encouraged. For example, pain and comfort will inevitably affect an amputee’s ability to wear a prosthesis and go to work, which will have a direct impact on the economics of the situation. Therefore, tools that quantify socket fit and comfort may show an improved economic impact to the payer. Also, if the person has less pain, it is likely that they will be more active, have improved performance, and report an improved quality of life.

It was strongly recommended that existing outcome tools in other fields that might be applicable to prosthetics be identified and investigated. For instance, studies have been done in other fields on such relevant topics as falls, health care economics, over-use injury, and chronic disability. Due to the importance given by the group to Health Care Economic Impact, it was recommended that this be the topic for a future State-of-the-Science meeting (5,6).

References

1. Journal of Prosthetics and Orthotics, State-of-the-Science Number 6, Proceedings, 2006

2. Finch E, Brooks D, Stratford P, Mayo N. Physical rehabilitation outcome measures. Baltimore: Lippencott Williams and Wilkins, 2002

3. McDowell I, Newell C. A guide to rating scales and questionnaires. Oxford: Oxford University Press; 1996.

4. Jerosh-Herold C. An Evidence-based approach to choosing outcome measures: a checklist for the critical appraisal of validity, reliability, and responsiveness studies. Brit J Occupat Ther. 2005:68(8):347-353.

5. Gold M, Siegel J, Russell L, Weinstein M, editors. Cost-Effectiveness in Health and Medicine. Oxford, Oxford University Press: 1996.

6. U.S. Department of Health and Human Services. Healthy People 2010. 2nd ed. With understanding and improving health and objects for improving health. 2 vols. Washington DC; US Government Printing Office; 2000.