View Options - Click to expand
Print Options - Click to expand
E-Mail Options - Click to expand

Measurement of Functional Outcome with Individuals Who Use Upper Extremity Prosthetic Devices: Current and Future Directions

F. Virginia Wright, BSc(PT), MSc, PhD

ABSTRACT

Since the mid-1990s, there has been considerable activity internationally on development and validation of pediatric prosthetic functional status measures for use with children with an upper extremity limb loss. In contrast, there are few prosthesis-specific functional status measures available specifically for use with adults with upper limb loss. The recent emphasis on outcome measurement across rehabilitation reflects the recognition that understanding of outcome results is an essential part of an effective health care system. From an outcomes perspective, the prosthetic community needs to systematically and prospectively gather information using validated functional status measures for both adult and pediatric groups.

The purpose of this article is to provide measurement information to help clinicians and researchers in the identification and selection of "activity" and "participation" outcome measures that relate to upper extremity function and use of a prosthetic device. This review investigates: intended use of the measure, condition-specific (disease-specific) versus generic measures, age/life-stage considerations, observational measures versus client/parent-report questionnaires, psychometric properties of the measure, evaluation of change, use of individualized measures, choosing a measure or a core set of measures, and aspects of clinical utility.

Three key recommendations were made by the author for future work: 1) Given the relative lack of prosthetic-specific measures available for adults, it is important to begin their development and validation. It may be possible to use existing pediatric prosthetic measures as a starting structural and content framework; 2) The building of a core set of outcome measures is recommended to provide a comprehensive picture of different aspects of hand function and prosthetic use. This core set should include an individualized measure of outcome; and 3) Because the numbers of upper limb amputees in any one center are often limited, formation of networks across rehabilitation facilities and use of the similar core sets of measures would facilitate capture of outcome data within large cooperative databases. There is a tremendous opportunity for the prosthetic service providers and researchers to make use of the recently validated functional measures and contribute ground-breaking knowledge on upper extremity outcomes. (J Prosthet Orthot. 2006;18:46–56.)

Rehabilitation research and clinical practice are undergoing a shift in their underlying philosophy, moving from a traditional assessment and intervention focus at the level of "body structure and functions" to an emphasis on goals and outcomes at the levels of "activity" (carrying out tasks) and "participation" (involvement in life situations.1 The health framework known as the International Classification of Function (ICF) delineates both the complexity of relationships among these three areas of outcome and the potential impact of the environment and personal factors on outcomes (Figure 1 ). The ICF model is intended to serve as a common language across health disciplines2 and has been recommended as an organizing framework for outcome measure selection and goal setting to facilitate work on a broad and meaningful spectrum of outcomes.3–6

The recent emphasis on outcome measurement across rehabilitation reflects the recognition that understanding of outcome results is an essential ingredient in the development and maintenance of an effective health care system. For assistive technology devices (ATD) in general, the value of the devices has been assumed rather than demonstrated, despite the absolute need for financial accountability to a variety of stakeholders in relation to outcomes.7 In prosthetics, as in the broader field of ATD, evaluation of functional outcomes by validated measures lags far behind the developing technologies.7 Discussions about the evaluation of prosthetic devices are still surprisingly scarce in the rehabilitation literature. In their 2005 overview of measurement of rehabilitation outcomes for use with assistive technology, Rust and Smith8 made no mention of prosthetic measures, upper or lower extremity, in the list of 100 widely used instruments. Prosthetic device evaluations have tended to rely on surveys that evaluate frequency of device use and abandonment, or satisfaction.9 The results from these surveys are used as proxies for functional outcomes, e.g., if the client is satisfied and the device is worn extensively, the functional capabilities/ outcomes are assumed to be high.

ATD outcomes research has been defined as the systematic study of the effects produced by ATDs in the lives of users and their environments.10 From an outcomes perspective, the prosthetic community needs to gather information on "activity" and "participation" prospectively using validated functional status measures that are recognized internationally for prosthetic use. As part of the continuing work on developing the better prosthetic devices for adults and children, it is critical that service providers develop a comprehensive understanding of the functional strengths and limitations of the current devices, as well as about patterns of use and rejection.

OVERVIEW

The purpose of this report is to provide measurement information that can be used by clinicians and researchers in the identification and selection of "activity" and "participation" outcome measures that relate to upper extremity function and use of a prosthetic device. An overview of relevant pediatric and adult functional outcome measures within a measurement framework is presented. The reader is referred to the specific publications referenced for details on each measure's psychometric properties. Other domains, such as satisfaction, social significance, and subjective well-being, as outlined in the taxonomic framework for ATDs by Jutai et al.10 are mentioned briefly to allow the reader to situate the functional outcomes piece within this larger framework.

The literature search for published functional outcome measures was conducted for the years 1987 through November 2005 using MEDLINE and CINAHL (OVID) electronic databases. The following key words were used: amputee, prosthetics, upper extremity, lower extremity, adult, pediatric, activity of daily living, functional abilities, physical therapy, occupational therapy, and outcome measures. In addition, names and acronyms of the key assessments identified were used as search words.

GUIDE TO SELECTION OF OUTCOME MEASURES

INTENDED USE OF THE MEASURE

For the purposes of outcome assessment, it is important to identify a measure that has demonstrated evaluative properties, meaning that it is responsive to meaningful changes in status.11 Thus, one of the first factors that needs to be considered in measurement selection is whether the identified tool is evaluative, discriminative, or predictive in its purpose.12 Evaluative measures are typically made up of multiple items that span the broad continuum of ability and skill levels, have potential for detecting change in the targeted population, and are rated on a graded response option scale.12

As the concept of responsiveness to change came into prominence only in the mid-1990s, the more recent hand function measures are the ones that have been developed with outcome evaluation as a primary goal, e.g., the Assessment for Capacity of Myoelectric Control (ACMC),13 the Assisting Hand Assessment,14 the Prosthetic Upper Extremity Functional Status Index (PUFI),15,16 the Unilateral Below Elbow Test (UBET),17 and the Disabilities of the Arm Shoulder and Hand Scale (DASH).18 Given the establishment of key aspects of the reliability and validity of these tools, work is under way (often by the original developers of the measure) to formally determine their responsiveness to change. Although it is possible that some of the older discriminative measures, such as the University of New Brunswick Test (UNB Test),19 have evaluative properties, they were not designed to assess change, so their ability to detect change should not be assumed.11

DISEASE-SPECIFIC VERSUS GENERIC MEASURES

The advantages and disadvantages of using a conditionspecific versus generic measure have been debated in other fields of rehabilitation and should be weighed whenever selecting a measure. As Stanton et al.20 summarized in their outcome measures paper, "The literature suggests that disease- specific measures may be useful when used alone or in conjunction with a general health instrument. It is apparent that disease-specific instruments, although supplying much information about the disease process and outcomes, do not yield the breadth of information about overall health status that is possible with a general measure" (p. 33). However, if the clinician or researcher's goal is to understand the contribution of the prosthetic device to function, a conditionspecific measure should be used.8 An outcome measure that is targeted for prosthetic use also will help service providers identify prosthetic issues that might be responsive to interventions such as prosthetic device adaptations or training. A well-designed evaluative measure that has a prosthetic focus should have the advantage of containing items that are likely more in tune with the issues and priorities of prosthetic device wearers.

Condition-specific functional status measures that have been designed specifically for upper extremity prosthetics include the ACMC,13 Child Amputee Prosthetics Project- Functional Status Inventory series (CAPP-FSI, CAPP-FSIP, and CAPP-FSIT),21–23 PUFI,16 UNB Test,19 and UBET.17 The focus of each of these measures is described in Table 1 . Notably all but the ACMC are intended solely for use with pediatric clients. This intensive pediatric measurement work has occurred in response to increasing requests from funders for empirical evidence on outcomes related to age of first prescription and subsequent use and wear of an upper extremity prosthetic device. Of particular interest are the decisions about earliest age of provision of a device and the best type of prosthesis with respect to a child's developmental level. For example, from a developmental outcomes and functional use perspective, are there advantages to fitting very young children with an active device?13

Hand/arm function measures such as the Assessment of Motor and Process Skills (AMPS),24 Hand Function Sort Questionnaire,25 Michigan Hand Outcomes Questionnaire,26 and the DASH questionnaire18 have been designed for use in adult rehabilitation across a variety of hand disabilities. Areas rated by these scales include degree of difficulty/ability to perform, satisfaction with performance, and quality/effectiveness of performance. Although these measures are capable of giving feedback on the ability to perform two-handed tasks, the details about use of the prosthetic device (active, passive, capability) are not provided. Use of the DASH has been reported with adults who have an upper extremity amputation.27

There also are detailed hand function observational assessments that focus on the speed (dexterity) component of function. These typically have been designed for use across hand disabilities. These include the well-known Jebsen Standardized Test of Hand Function28 and Purdue Pegboard Test29 as well as the more recently developed Southampton Hand Assessment Protocol (SHAP).30 The SHAP has been recommended by its authors for use in prosthetics because it provides a picture of effectiveness of the device through the timing of functional tasks as performed by the prosthetic hand.30

Alternatively, one could consider the use of a number of condition-specific hand function outcome measures that pertain to individuals with neurologic diagnoses and unilateral hand impairment, e.g., cerebral palsy (CP) or stroke. From a pediatric perspective, the ABILHAND-Kids questionnaire31 for parents of children with CP may have potential in pediatric prosthetics because it consists of bimanual activities and uses a degree of difficulty scale, i.e., the focus is not on the rating of movement quality. The AHA14 is a pediatric observational measure that was designed for use in hemiplegic CP and unilateral brachial plexus injury, and its scores are based on the perceived effectiveness of the impaired hand in bimanual activity. It is being adapted for use with children with upper limb reduction deficiencies by a prosthetic group in The Netherlands.32 Other disease-specific neurologic measures such as the Quality of Upper Extremity Skills Test (QUEST),33 Melbourne Assessment,34 and Arm Motor Ability Test (AMAT)35 do not appear to be well-suited to evaluation of prosthetic function because they focus on quality of movement in the impaired hand from a neurologic standpoint and emphasize unilateral tasks.

Evaluation of upper extremity abilities also can be approached from the perspective of assessing function overall by means of a generic functional measure, such as the Activity Scale for Kids (ASK),36 AMPS,37,38 Pediatric Evaluation of Disability Inventory (PEDI),39 Pediatric Orthopaedic Society of North America Questionnaire (known as both the PODCI and POSNA),40 School Function Assessment,41 or Functional Independence Measure (FIM)/WeeFIM® Instrument.42 These measures do not give feedback on use and control of the prosthesis. Instead, they are broad in content and often include participation, social function, and health-related, quality-of-life items. They profile the individual's ability to perform higher level tasks and give a picture of overall functional capability in daily life situations. However, it is not possible to determine whether individuals integrate the use of their prosthetic device into their performance of the stated activities. With these tests, use of a device typically is rated as being less independent,8 e.g., on the FIM, the individual who is scored as using his or her prosthesis for a task rather than doing it without a device received a modified independence score for that item. Perhaps this is one reason there is little evidence from the published literature that generic measures of function are being used. One of the exceptions is a recent prosthetic impact study in pediatrics43 that employed the PODCI and Pediatric Quality of Life (PedsQL)44 in addition to a prosthetic measure, the UBET.

AGE/LIFE-STAGE CONSIDERATIONS

Different functional content areas are needed to handle the differences in activity and participation demands specific to adults and children. Within pediatrics, there are age breakdowns that can be done to tailor content to infants and preschool children, school-age children, and adolescents. Similarly with adults, one might want to consider the differences in activity demands and priorities between a younger adult and a retired individual.

MEASURES FOR PEDIATRIC CLIENTS

Since the mid-1990s, there has been considerable activity internationally in the development and validation of pediatric prosthetic functional status measures (observational or selfreport) for use with individuals with an upper extremity limb loss (Table 1 ). These validated tools have replaced the studyspecific functional ability measures profiled in earlier articles. 37,45,46 One of the biggest challenges in this development work is dealing with the differences in age demands within the pediatric grouping. The UNB Test19 and UBET17 have tackled this by having age subgroup tests. The PUFI15,16 has a version for children younger than 6 years and a second version for children 6 years and older. The CAP-FSI series has toddler (CAPP-FSIT)23 and preschool versions (CAP-FSIP),22 as well as the original school-age–child version.21

MEASURES FOR ADULT CLIENTS

Although there are a number of functional status measures designed specifically for lower extremity amputees,47–53 development of upper extremity prosthetic-specific measures for adults is much less evident than in pediatrics. In a report on development of the functional status measures in the Orthotics and Prosthetics Users' (OPUS) Survey group,49 the authors noted that their deferral of the design of an upper extremity measure was based on the restricted number of respondents with upper limb amputations. Similarly, in application of the Trinity Amputation and Prosthesis Experience Scales (TAPES) to upper extremity prosthetics, Desmond and MacLachlan54 also suggested that the small number of upper extremity amputees contributed to lack of measures specifically for them. Most often, there has been crossover use of a pediatric observational measure such as the UNB Test into the adult group55 or development of a questionnaire specifically for the purpose of a functional outcome or satisfaction study.56–58 These study-specific measures do not appear to have undergone validation, and they have not been used by other researchers.

The most notable new prosthetic measure for adults who use a myoelectric prosthesis is the ACMC, an observational tool.13 It is also suitable for use in pediatric age groups because of its naturalistic test protocol. Specifically, the task for evaluation is selected according to the client's developmental level and interests (e.g., playing a game, preparing a meal), and a standard 37-item, skill-based rating scheme is applied by the assessor to the task.

Although there is some indication in the lower limb prosthetic literature59–62 of use of generic functional tools, such as the Sickness Impact Profile or FIM Instrument, there was no evidence from this author's literature search of similar use in the adult upper limb prosthetic literature.

The work on the TAPES54,63 demonstrates the interest in looking concurrently at function, health-related quality of life, and satisfaction with the device and clinic services for the adult prosthetic groups. It seems likely that the expansion of the number of prosthetic-specific, adult-based tools with an upper extremity focus may occur as a result of the impact of the large number of traumatic amputees from the armed forces personnel serving overseas. It is expected that funders will require evidence on the effectiveness of the new and costly devices that are being designed to meet the high-level functional needs of these individuals.

OBSERVATIONAL MEASURES VERSUS CLIENT/PARENT-REPORT QUESTIONNAIRES

Many of the prosthetic-specific measures used with pediatric and adult clients are observational in format and directly evaluate how the prosthetic hand functions in bimanual tasks. These measures serve as a critical component of the problem-identification process that a prosthetic team employs to make decisions about fitting, revisions, and training. Some of the observational tests take place in structured situations (i.e., using standard equipment and tasks),30,37,44 whereas others involve seminaturalistic testing, in which a series of tasks are performed with specified equipment in the course of play or life skills activity.17,19 Alternatively, there are observational assessments that evaluate prosthetic use during performance of the client's choice of a complex upper extremity task.13,38 In the seminaturalistic and client-choice tests, the client is not aware of which parts of the activity are being scored. The recording of scores is done by the clinician after the activity so that scoring does not influence the client's performance.

Client- or parent-report measures permit a different look at the individual's functional status, in this case focusing on reported use and value of the prosthetic device in real world situations. The development of self-report measures came into prominence in the 1990s in tandem with the clientcentered care movement in rehabilitation. There is strong evidence from work with self-report measures in other fields of rehabilitation64 that these tools have the potential to be reliable, valid, and responsive to change. When results from these questionnaires are integrated with those from observational measures, they are expected to enhance the information provided because they give the added perspective of the impact of personal and environmental factors on function.1

PSYCHOMETRIC PROPERTIES OF THE MEASURE

When a measure is chosen, it is important to consider the characteristics of the client group on whom the psychometric properties such as reliability, validity, and responsiveness were established. For outcome tools, the properties of greatest interest are test-retest reliability, inter-rater reliability if the measure is observational, construct validity (i.e., extent of correlation with other known measures of similar constructs), and responsiveness to change. The reader is referred to the article by Lexell and Downham65 for a detailed review of evaluation of reliability evaluations as they pertain to evaluation of change.

Prosthetic outcome measures that have been developed since the mid-1990s have generally been introduced to the rehabilitation community via one or more publications on psychometric properties. It is difficult to find comprehensive published evidence of validation for older tools that were published before the emergence of the emphasis on psychometric testing. Instead, their first use was likely profiled within a clinical report. Often these tools are so well known and accepted from a clinical standpoint that their properties are not called into question. The longstanding measure best known in the upper extremity prosthetic world is the UNB Test. Although there is published information in the UNB Test manual19 regarding its inter-rater reliability, evidence about its test-retest reliability is absent. As discussed at the Myoelectric Controls (MEC) 2005 Outcomes Measures Workshop, because the UNB Test is undergoing content revisions and age expansion by the UNB group, there will be a new opportunity and reason to reassess its psychometric properties. Psychometric retesting is needed when a measure undergoes content or format changes, regardless of its former psychometric status.

One barrier to consideration of these properties for generic tools such as the AMPS and the DASH is that the validation work likely was established with nonprosthetic populations. It cannot be assumed that a functional measure that works well with adults with arthritis or a hand injury will have the same psychometric properties when applied to adults with a prosthetic device, given the differences in underlying pathology and performance challenges.

EVALUATION OF CHANGE

When a measure is designed to evaluate outcome, the intention is that it will be able to detect clinically important changes that have occurred in response to an intervention (e.g., new prosthetic device, device revisions, training) or other factors. One key consideration in the interpretation of change is the standard error of measurement (SEM) of the tool as determined by its test-retest reliability. Change–positive or negative–needs to exceed the 95% confidence interval associated with the SEM before one can be certain that actual change has occurred.66 Thus, it is important when choosing a measure and deciding on the magnitude of change that will be set as a programmatic outcome target that the tool's measurement error from test-retest work with similar populations be considered whenever available.67 The other consideration in this determination is the amount of difference that clients and clinicians deem to be clinically important. Although minimal clinically important differences have yet to be established for prosthetic measures, there are models from work in adult rheumatology that could be used to guide the prosthetic community in this endeavor.68

The current development of prosthetic client-report questionnaires is occurring at a time when the Rasch measurement model is gaining prominence. Application of the Rasch model is intended to have a positive effect on a measure's ability to detect change.14 Rasch measurement converts an ordinal rating scale into one that is interval such that a particular change score magnitude has the same meaning across all points of the scale. Thus, a high-functioning client who is near the top of the scale would be given considerable credit for skill gains of a single very difficult item, whereas clients in the middle of the scale would be expected to make improvements on several less difficult items to be considered to have a similar degree of change.69 In contrast, on an ordinal scale, such as are typical for functional measures, there may be more of an inclination to have a ceiling effect for higher functioning clients while those in the middle of the scale benefit from an inflation of scores. Three hand function measures, one for prosthetics13 and two that are generic,14,38 have undergone Rasch analysis early in their development, and it is expected that several of the existing measures will undergo the same process as the number of users grows and sample sizes are sufficient to support the complex analysis.

USE OF INDIVIDUALIZED MEASURES

A number of individualized measures may be suitable for use in prosthetics.70 The two that feature most prominently in rehabilitation literature are Goal Attainment Scaling (GAS)71,72 and the Canadian Occupational Performance Measure (COPM).70,73,74 There is evidence that individualized measures have greater sensitivity to change in the short-term than do fixed-item measures.67,70 There is also an indication that outcomes are enhanced when a client takes part in the goal-setting process.74

A single example of GAS use in prosthetics occurred in Rushton and Miller's study75 of adults with lower-extremity amputations in which the investigators concluded that GAS was more responsive to short-term change than either the Locomotor Capabilities Index of the Prosthetic Profile of the Amputee48 or the Barthel Index.76 Rushton and Miller75 recommended that GAS be used in tandem with fixed-item functional status measures to provide a broader picture of patients' capabilities and outcomes. This leads to the idea of partnering measures, i.e., GAS or the COPM as a companion to fixed-item measures as a way of exploring areas of priority and helping in the interpretation of change scores from fixed-item tools. The use of individualized measures has yet to be addressed in the pediatric prosthetic literature, despite the strong recognition of the potential for use of these measures in other areas of pediatric rehabilitation.71

One drawback to the use of an individualized measure by itself is that it does not allow comparison of one client's abilities to another because there is not a common set of tasks evaluated. This could have serious ramifications in prosthetic evaluations because it becomes impossible to gain an overall group perspective on the actual skills accomplished when a prosthesis is used. Group information is critical to the understanding of comparative benefits of different types of prostheses and of performance capability with and without a prosthetic device.

CHOOSING A MEASURE OR A CORE SET OF MEASURES

One of the most important considerations in the choice of a measure relates to the outcome question(s) asked and the definitions associated with the indicators of outcome. Although it may initially seem that a group of potentially competing prosthetic hand function measures are available, there are notable differences in the measurement intents of these tools (Table 1 ). For example, a measure might look at different aspects of control of the hand (grip, different hand actions, hand function in different arm positions with or without looking).13,15,46 Alternatively, a tool might focus on the speed and competence aspects of prosthetic use (quality/ training issues).19,37,45 Another possible emphasis is on a child's comparative abilities with and without his or her prosthesis15,16 or on the child's preferred method of performance of tasks.16

As discussed, it is likely desirable to use more than one measure to obtain a comprehensive picture of different aspects of hand function and prosthetic use. This could mean the incorporation of two or more prosthetic measures or use of a prosthetic tool along with a generic functional/health status measure.11 The idea of developing a core set of measures or toolkit of measures77 for particular client groups fits well within the ICF framework way of looking at outcomes.6 For example, to represent outcomes across ICF components, the clinician could choose an observational measure that is reflective of the prosthetic control (body functions and structures) and basic functional skill (activity), e.g., the UNB Test, SHAP or ACMC. Another measure chosen might be more reflective of higher level functional skills performed in the context of daily life (activity and possibly participation), e.g., a self-report measure such as the CAPP-FSI, PUFI, or health-related measures such as the POSNA40 or TAPES. Integration of an individualized goal measure such as GAS or the COPM into this outcome set would provide an indication of priority activities for improvement and likely give a more sensitive evaluation of change in the short term.

An outcome study by Burger et al.78 provides an example of the pairing of a prosthetic questionnaire and observational assessment. The CAPP-FSI served as an initial quick evaluation of the child's overall reported functional performance in daily life, while the UNB Test provided the necessary details on method of performance. Combinations of observational (UBET) and self-report measures (PUFI, PODCI, and Pediatric Quality of Life) were employed in the below-elbow deficiency study by James and Bagley.43 Lambregts et al.79 used the PUFI, CAPP-FSI, SHAP, and UBET to provide comprehensive information on children's capability (UBET and SHAP) and on their actual performance in real life (CAPP-FSI and PUFI). The distinction between "can do" and "does do" is an important one in rehabilitation,36 helping the service provider to better understand what goals and interventions need to be put in place to maximize the individual's function.

It is important not to assume that changes in one area (e.g., control of prosthesis) are necessarily strongly associated with changes in another (e.g., functional use of the prosthesis in daily activities) even if measures are strongly associated at baseline.11,80 As shown in the ICF framework, a multiplicity of other personal and environmental factors have an impact on changes that occur. Consequently, it may be wise to choose a measure in two or more outcome areas to ensure that the various aspects of change are detected.

ASPECTS OF CLINICAL UTILITY

For the most part, the prosthesis-specific measures described in this article are freely available through their authors, sometimes in exchange for sharing of data within an international database (e.g., the PUFI), and there is a move to have PDF versions available for free access on a test developer's facility website. Generic tools such as the PEDI, SF-36, and COPM are purchased measures from a publisher. A few observational measures (e.g., AMPS, AHA) require purchase and formal clinician training.

One key to successful application of outcome measures is that the information derived needs to have relevance for both the service provider and client to support both the time required to administer the tool and the respondent burden. The different administration demands of various scales need to be considered when choosing, e.g., level of assessor training to complete reliably, equipment, scoring method, time to administer, and overall demands on the client and assessor. When a questionnaire is to be completed by a school-aged child or teen, the measure needs to be visually appealing and easy to complete. From the clinician's perspective, questionnaires should be easy to administer and score and should provide summaries that can be shared promptly with the client and team.

When reviewing a measure, clinicians should not be deterred by existence of a few items that do not fit the needs of a particular client or dismiss a measure because it is missing an activity area. Well-developed outcome measures are designed to be comprehensive, rather than exhaustive, in content. No single measure, no matter how strong its psychometric properties, will fit all of the content needs of the clinician for every client. This leads to an important caveat in relation to clinicians' tendency to adapt a scale to make it better suited to their individual clientele. To maintain the psychometric strengths of the scale, it is important that chosen tools are used in their entirety. Thus, it is not suitable to use parts of measures unless subscales have been demonstrated to be independent (e.g., the PEDI is such a measure in which Self-care and Mobility scale scores can be considered separately).39

The issue of respondent burden also needs to be considered when several outcome measures are integrated into a core set. It is crucial to have a clear focus on how the data are going to be used from each of the measures chosen.11 The content of selected measures should be considered carefully at the selection stage so that duplication of content between measures is minimized.

Challenges in implementing a systematic outcome measure process are well documented in rehabilitation. For example, limitations to the uptake of standardized hand assessments have been described in rheumatology,81 in which fewer than 10% of responding occupational therapists used standardized observational assessments, preferring instead to use in-house tools and citing time as a major barrier to getting assessments done.

Use of a computer-based, direct-access questionnaire can facilitate client completion and scoring accuracy and provide instant reports that the clinician can share with the client. The PUFI-PC version 3.0 that has been developed by the author is an example of client-access scoring software. Given its success from a clinical utility standpoint, plans are under way to develop a web version to permit completion of the questionnaire at home just before the child's clinic appointment.

Software versions of questionnaires permit the storage of data in electronic databases, and this facilitates group evaluation. Use of scoring software helps address issues regarding adaptation to other languages because a variety of language versions can be built into the software. There is evidence that the electronic version results are comparable to paper-report versions,82 although it is essential to formally test the assumptions about test-retest reliability as they are developed. The main barrier to development of software versions is the cost in adapting paper measures to electronic delivery.

SUMMARY

It is evident from the presentations at the MEC 2005 Outcome Measures Workshop that there are a growing number of observational and client-report measures that might be chosen for a prosthetic client. This expanded choice is evident in pediatrics in particular. However, the continuing emergence and refinement of new functional status measures results in a new challenge for clinicians. That is, the greater the choice, the more difficult it becomes for the clinician to choose the best measure(s)83 for a given clinical situation. To fully understand the strengths, potential, and limitations of a core set of measures in a given population, it is important to pilot-test their use. Work by Demers et al.,77 in which they developed and pilot-tested a toolkit of outcome measures for use in geriatric rehabilitation, may provide a useful model for clinicians, program managers, and researchers to follow as they determine the core set of best outcomes for use in prosthetics.

It is important to recognize that development and refinement of a psychometrically strong outcome measure can be a 10- to 15-year process.69,84 Given the advent of a number of new pediatric functional status measures since the late 1990s, it is hoped there will be a rapid increase in the uptake of these measures and in the number of outcome papers related to pediatric prosthetics. There is little rationale at this time to develop additional pediatric measures. Rather, it is important to promote international use of the existing ones, making refinements to these as the experience base grows. In contrast, given the relative lack of prosthetic-specific measures available for adults, thought needs to given as to what is needed both from a content and age-breakdown perspective. It may be possible to use an existing pediatric measure, such as the UNB Test, PUFI, or CAPP-FSI, as a starting framework, e.g., use the rating scale concepts and formats and then conduct formal item generation to determine the most responsive set of activities for adult respondents.

In conclusion, this is both an exciting and challenging time for outcome measurement in prosthetics given the intense pressures within the health care system to demonstrate the comparative effectiveness and necessity of rehabilitation interventions. There is a tremendous opportunity for the prosthetic service providers and researchers to make use of the recently validated functional measures and contribute ground-breaking knowledge on outcomes. All members of the prosthetic team should be included in the determination of the most suitable core set of outcome measures for their client group. Because the numbers of upper limb amputees in any one center often are relatively small, formation of networks across rehabilitation facilities and use of the similar core sets of measures could facilitate capture of outcome data within large cooperative databases. These databases, used in conjunction with large prospective trials and meaningful outcome questions, are a central foundation for the next decade of evidence-based knowledge in the field of upper extremity prosthetics.7

ACKNOWLEDGMENTS

The author thanks the PUFI study team (Sheila Hubbard, Rose Nishiyama, Ka Lun Tam and Susan Cohen) at Bloorview Kids Rehab for the insights provided in the development of a pediatric prosthetic measure. The author also thanks the Bloorview Research Institute for supporting her time in writing this article.

Correspondence to:F. Virginia Wright, BSc(PT), MSc, PhD, Scientist, Outcomes and Measurement Theme, Bloorview Research Institute, 150 Kilgour Road, Toronto, Ontario, Canada, M4G 1R8; e-mail: .


F. VIRGINIA WRIGHT, BSc(PT), MSc, PhD, is affiliated with Bloorview Research Institute, Toronto, Ontario, Canada.

References:

  1. World Health Organization (WHO). ICIDH-2: International Classification of Functioning, Disability, and Health (ICIDH). Geneva, Switzerland: World Health Organization; 2001.
  2. Stucki G, Ewert T, Cieza A. Value and application of the ICF in rehabilitation medicine. Disabil Rehabil 2003;25:628–634.
  3. Baxter P. ICF: health vs disease. Dev Med Child Neurol 2004; 46:291–297.
  4. Hermann KM, Reese CS. Relationships among selected measures of impairment, functional limitation, and disability in patients with cervical spine disorders. Phys Ther 2001;81: 903–914.
  5. Rentsch HP, Bucher P, Nyffeler ID, et al. The implementation of the ‘International Classification of Functioning, Disability and Health' (ICF) in a daily practice of neurorehabilitation: an interdisciplinary project at the Kantonsspital of Lucerne, Switzerland. Disabil Rehabil 2003;25:411–421.
  6. Stucki G. International classification of functioning, disability, and health (ICF): a promising framework and classification for rehabilitation medicine. Am J Phys Med Rehabil 2005;84: 733–740.
  7. Fuhrer MJ. Assistive technology outcomes research: challenges met and yet unmet. Am J Phys Med Rehabil 2001;80:528–535.
  8. Rust KL, Smith RO. Assistive technology in the measurement of rehabilitation and health outcomes. A review and analysis of instruments. Am J Phys Med Rehabil 2005;84:780–793.
  9. Atkins DJ, Heard DCY, Donovan WH. Epidemiologic overview of individuals with upper-limb loss and research priorities. J Prosthet Orthot 1996;8:2–11.
  10. Jutai J, Fuhrer MJ, Demers L, et al. Toward a taxonomy of assistive technology assessment. Am J Phys Med Rehabil 2005; 84:294–302.
  11. Finch E, Brooks D, Stratford PW, Mayo NE. Physical Rehabilitation Outcome Measures, 2nd ed. Hamilton, Ontario, Canada: Canadian Physiotherapy Association and BC Decker Inc; 2002.
  12. May L. The challenge of measuring change: responsiveness of outcome measurements. Can J Rehabil 1995;10:15–24.
  13. Hermansson LM, Fisher AG, Bernspang B, Eliasson A-C. Assessment of capacity for myoelectric control: a new Rasch-built measure of prosthetic hand control. J Rehabil Med 2005;37: 166–171.
  14. Krumlinde-Sundholm L, Eliasson A-C. Development of the assisting hand assessment: a Rasch-built measure intended for children with unilateral upper limb impairments. Scand J Occup Ther 2003;10:16–26.
  15. Wright FV, Hubbard S, Jutai J, Naumann S. Development and reliability testing of a new functional status questionnaire for children who use upper extremity prostheses. J Hand Ther 2001;14:91–104.
  16. Wright FV, Hubbard S, Jutai J, Naumann S. Evaluation of the validity of the Prosthetic Upper Extremity Functional Index (PUFI). Arch Phys Med Rehabil 2003;84:518–527.
  17. Bagley A. The unilateral below elbow test (abstract). Dev Med Child Neurol 2004;46:33–44.
  18. Hudak PL, Amadio PC, Bombardier C, and the Upper Extremity Collaborative Group. Development of an upper extremity outcome measure: The DASH (disabilities of the arm, shoulder and hand). Am J Industr Med 1996;29:602–608.
  19. Sanderson ER, Scott RN. UNB test of prosthetic function: a test for unilateral amputees (test manual). Fredericton, New Brunswick, Canada, University of New Brunswick; 1985.
  20. Stanton MP, Gresham GE, Dittmar SS. Relationship among measures of disease, general health, and functional status. In: Dittmar, SS, Gresham GE, eds. Functional Assessment and Outcome Measures for the Rehabilitation Professional. Gaithersburg, MD: Aspen Publishers, 1997:31–36.
  21. Pruitt SD, Varni J, Setoguchi Y. Functional status in children with limb deficiency: development and initial validation of an outcome measure. Arch Phys Med Rehabil 1996;77:1233–1238.
  22. Pruitt SD, Varni JW, Seid M, Setoguchi Y. Functional status in limb deficiency: development of an outcome measure for preschool children. Arch Phys Med Rehabil 1998;79:405–411.
  23. Pruitt SD, Seid M, Varni JW, Setoguchi Y. Toddlers with limb deficiency: conceptual basis and initial application of a functional status outcome measure. Arch Phys Med Rehabil 1999; 80:819–824.
  24. Bray K, Fisher AG, Duran L. The validity of adding new tasks to the Assessment of Motor and Process Skills. Am J Occup Ther 2001;55:409–415.
  25. Matheson LN, Kaskutas VK, Nada D. Development and construct validation of the Hand Function Sort. J Occup Rehabil 2001;11:75–86.
  26. Chung KC, Pillsbury MS, Walters MR, Hayward RA. Reliability and validity testing of the Michigan Hand Outcomes Questionnaire. J Hand Surg 1998;23A:575–587.
  27. Davidson J. A comparison of upper limb amputees and patients with upper limb injuries using the Disability of Arm, Shoulder and Hand (DASH). Disabil Rehabil 2004;26:917–923.
  28. Jebsen RH, Taylor N, Trieschmann RD, et al. An objective and standardized test of function. Arch Phys Med Rehabil 1969;50: 311–319.
  29. Tiffin J, Asher E. The Purdue Pegboard: norms and studies of reliability and validity. J Appl Psych 1948;32:234–247.
  30. Light CM, Chappell PH, Kyberd PJ. Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: normative data, reliability, and validity. Arch Phys Med Rehabil 2002;83:776–783.
  31. Arnould C, Penta M, Renders A, Thonnard JL. ABILHAND-Kids: a measure of manual ability in children with cerebral palsy. Neurology 2004;63:1045–1052.
  32. Buffart LM, Pesch-Battenburg JMFB, Heijningen VG, et al. Functional status of children with a congenital upper limb reduction deficiency. Myoelectric Controls (MEC) Symposium, Fredericton, New Brunswick, Canada, August 17–19, 2005.
  33. DeMatteo C, Law M, Russell D, et al. The reliability and validity of the Quality of Upper Extremity Skills Test. Phys Occup Ther Pediatr 1993;13:1–18.
  34. Bourke-Taylor H. Melbourne Assessment of Unilateral Upper Limb Function: construct validity and correlation with the Pediatric Evaluation of Disability Inventory. Dev Med Child Neurol 2003;45:92–96.
  35. Kopp B, Kunkel A, Flor H, et al. The Arm Motor Ability Test: reliability, validity and sensitivity to change of an instrument for assessing disabilities in activities of daily living. Am J Phys Med Rehabil 1997;78:615–620.
  36. Young NL, Williams JI, Yoshida KK, et al. The context of measuring disability: does it matter whether capability or performance is measured? J Clin Epidemiol 1996;49:1097–1101.
  37. Thornby MA, Krebs DE. Bimanual skill development in pediatric below-elbow amputation: a multi-centre, cross-sectional study. Arch Phys Med Rehabil 1992;73:697–702.
  38. Fisher AG. Assessment of Motor and Process Skills. Fort Collins: Three Star Press; 1999.
  39. Haley S, Coster WJ, Ludlow LH, et al. Pediatric Evaluation of Disability Inventory (PEDI). Version 1.0. Development, Standardization and Administration Manual. Boston: New England Medical Center Hospitals, Inc; 1992.
  40. Daltroy LH, Liang MH, Fossel AH, et al. The POSNA pediatric musculoskeletal functional health questionnaire: report of reliability, validity, and sensitivity to change. J Pediatr Orthop 1998;18:561–571.
  41. Coster W, Deeney T, Haltiwanger J, Haley S. School Function Assessment. San Antonio: The Psychological Corporation, Harcourt Brace and Company (Therapy Skill Builders); 1998.
  42. Msall M, DiGaudio K, Duffy L, et al. WeeFIM: normative sample of an instrument for tracking functional independence in children. Clin Pediatr 1994;33:431–438.
  43. James MA, Bagley A. Impact of prostheses on outcomes for children with unilateral congenital below elbow deficiency. Paper presented at the American Academy of Orthopaedic Surgeons 2005 Annual Meeting, Washington, DC, February 23–27, 2005.
  44. Varni JW, Seid M, Kurtin PS. Reliability and validity of the Pediatric Quality of Life Inventory Version 4.0 Generic Core Scales in healthy and patient populations. Med Care 2001;39: 800–812.
  45. Edelstein JE, Berger N. Performance comparison among children fitted with myoelectric and body-powered hands. Arch Phys Med Rehabil 1993;74:376–380.
  46. Weaver SA, Lange LA, Vogts VM. Comparison of myoelectric and conventional prostheses for adolescent amputees. Am J Occup Ther 1988;42:87–91.
  47. Legro MW, Rieher GD, Smith DG, et al. Prosthesis evaluation questionnaire for persons with lower limb amputations: assessing prosthesis-related quality of life. Arch Phys Med Rehabil 1998;79:931–938.
  48. Gauthier-Gagnon C, Grise M-C. Prosthetic profile of the amputee questionnaire: validity and reliability. Arch Phys Med Rehabil 1993;75:1309–1314.
  49. Heinemann AW, Bode RK, O'reilly C. Development and measurement properties of the Orthotics and Prosthetics Users' Survey (OPUS): a comprehensive set of clinical outcomes instruments. Prosthet Orthot Int 2003;27:191–206.
  50. Hagberg K, Branemark O, Hagg O. Questionnaire for persons with a transfemoral amputation (Q-TFA): initial validity and reliability of a new measure. J Rehabil Res Dev 2004;41: 695–705.
  51. Hart DL. Orthotics and Prosthetics National Outcomes Tool (OPOT): initial reliability and validity assessment for lower extremity prosthetics. J Prosthet Orthot 1999;11:101–111.
  52. Devlin M, Pauley T, Head K, Garfinkel S. Houghton Scale of prosthetic use in people with lower extremity amputations: reliability, validity, and responsiveness to change. Arch Phys Med Rehabil 2004;85:1339–1344.
  53. Selles RW, Janssens PJ, Jongenengel CD, Bussmann JB. A randomized controlled trial comparing functional outcome and cost efficiency of a total surface-bearing socket versus a conventional patellar tendon-bearing socket in transtibial amputees. Arch Phys Med Rehabil 2005;86:154–161.
  54. Desmond DM, MacLachlan M. Factor structure of the Trinity Amputation and Prosthesis Experience Scales (TAPES) with individuals with acquired upper limb amputations. Am J Phys Med Rehabil 2005;84:506–513.
  55. Lake C. Effects of prosthetic training on upper-extremity prosthesis use. J Prosthet Orthot 1997;9:3–9.
  56. Datta D, Selvarajah K, Davey N. Functional outcome of patients with proximal upper limb deficiency-acquired and congenital. Clin Rehabil 2004;18:172–177.
  57. Davidson J. A survey of the satisfaction of upper limb amputees with their prostheses, their lifestyles and their abilities. J Hand Ther 2002;15:62–70.
  58. Fraser CM. An evaluation of the use made of cosmetic and functional prostheses by unilateral amputees. Prosthet Orthot Int 1998;22:216–223.
  59. Muecke L, Shekar S, Dwyer D, et al. Functional screening of lower-limb amputees: a role in predicting rehabilitation outcomes. Arch Phys Med Rehabil 1992;73:851–858.
  60. Schoppen T, Boonstra A, Groothof JW, et al. Physical, mental, and social predictors of functional outcome in unilateral lowerlimb amputees. Arch Phys Med Rehabil 2003;84:803–811.
  61. Streppel KRM, de Vries J, van Harten WH. Functional status and prosthesis use in amputees, measured with the Prosthetic Pro- file of the Amputee (PPA) and the short version of the Sickness Impact Profile (SIP68). Int J Rehabil Res 2001;24:251–256.
  62. Whyte A, Carroll LJ. The relationship between catastrophizing and disability in amputees experiencing phantom pain. Disabil Rehabil 2004;26:649–654.
  63. Gallagher P, MacLachlan M. The Trinity Amputation and Prosthesis Experience Scales (TAPES) and quality of life in people with lower-limb amputation. Arch Phys Med Rehabil 2004;85: 730–736.
  64. Dittmar S. Selection and administration of functional assessment and rehabilitation outcome measures. In: Dittmar S, Gresham GE, eds. Functional Assessment and Outcome Measures for the Rehabilitation Professional. Gaithersburg, MD: Aspen Publishers, 1997: 11–15.
  65. Lexell JE, Downham DY. How to assess the reliability of measurements in rehabilitation. Am J Phys Med Rehabil 2005;84: 719–723.
  66. Stratford P, Spadoni G, Kennedy D, et al. Seven points to consider when investigating a measure's ability to detect change. Physiother Can 2002;54:16–24.
  67. Stratford PW, Spadoni GF. Assessing improvement in patients who report small limitations in functional status on conditionspecific measures. Physiother Can 2005;57:234–241.
  68. Wells G, Beaton D, Shea B, et al. Minimal clinically important differences: review of methods. J Rheumatol 2001;28:406–412.
  69. Russell DJ, Avery LM, Rosenbaum PL, et al. Improved scaling of the Gross Motor Function Measure for children with cerebral palsy: evidence of reliability and validity. Phys Ther 2000;80: 873–885.
  70. Carswell A, McColl MA, Baptiste S, et al. The Canadian Occupational Performance Measure: a research and clinical literature review. Can J Occup Ther 2004;1:210–222.
  71. King GA, McDougall J, Palisano RJ, et al. Goal Attainment Scaling: its use in evaluating pediatric therapy programs. Phys Occup Ther Pediatr 1999;15:31–52.
  72. Kiresuk TJ, Smith A, Cardillo JE. Goal Attainment Scaling: Applications, Theory and Measurement. Hillsdale, NJ: Lawrence Earlbaum Associates; 1994.
  73. Law M, Baptiste S, Carswell A, et al. Canadian Occupational Performance Measure. Canadian Association of Occupational Therapy Publications. 3rd ed. Ottawa, Canada; 1998.
  74. Wressle E, Eeg-Olofsson AM, Marcusson J, Henricksson C. Improved client participation in the rehabilitation process using a client-centred goal formulation structure. J Rehabil Med 2002; 34:5–11.
  75. Rushton PW, Miller WC. Goal attainment scaling in the rehabilitation of patients with lower-extremity amputations: a pilot study. Arch Phys Med Rehabil 2002;83:771–775.
  76. Mahoney FI, Barthel D. Functional evaluation: the Barthel Index. Maryland State Med J 1965;14:56–61.
  77. Demers L, Desrosiers J, Ska B, et al. Assembling a toolkit to measure geriatric rehabilitation outcomes. Am J Phys Med Rehabil 2005;84:460–472.
  78. Burger H, Brezovar D, Marincek C. Comparison of clinical test and questionnaires for the evaluation of upper limb prosthetic use in children. Disabil Rehabil 2004;26:911–916.
  79. Lambregts SAM, Doornbosch F, Roebroeck ME, et al. Functional outcome of adolescents and young adults with congenital upper limb reduction deficiency. Myoelectric Controls (MEC) Symposium, Fredericton, New Brunswick, Canada, August 17–19, 2005.
  80. Wright V. How do changes in impairment, activity and participation relate to each other? Evaluation using the model of Botulinum Toxin Type A injections in children with cerebral palsy. [Thesis] Hamilton, Canada: McMaster University, 2005.
  81. Blenkiron EL. Uptake of standardised hand assessments in rheumatology: why is it so low? Brit J Occup Ther 2005;68: 148–157.
  82. Athale N, Sturley A, Skoczen S, et al. A web-compatible instrument for measuring self-reported disease activity in arthritis. J Rheumatol 2004;31:223–228.
  83. Schwartz B. The Paradox of Choice: Why More is Less. Paperback ed. New York: Harper Collins Publishers Inc; 2005.
  84. Russell D, Rosenbaum P, Cadman D, et al. The Gross Motor Function Measure: a means to evaluate the effects of physical therapy. Dev Med Child Neurol 1989;31:341–352.