Outcomes Measurement: A Primer for Orthotic and Prosthetic Care

Lance O. Hoxie

All science is measurement.
-Helmholtz (1)

ABSTRACT

Outcomes measurement has become the essential component of any continuous quality improvement process. Increasingly, consumers and payers of healthcare services demand objective data, based on outcomes, to evaluate the quality of care among their providers.

This article presents a basic overview of outcomes measurement and its potential application to orthotic and prosthetic practices. It suggests a systematic approach to continuous improvement, with outcomes measurement as the principal focus. It also identifies potential areas of O&P care and management that could be monitored, with objective data and evaluation, to establish demonstrable outcomes of quality care.

Historical Overview

Fundamental to any component of patient care is measurement of performance; i.e., performance not only in the context of what and how care is delivered, but by the outcome of such care as well.

Until recently, the prevailing quality paradigm incorporated the beliefs that health-care organizations were collections of unrelated units; that higher quality meant higher costs; that quality was defined as meeting minimum requirements; and that the mechanism of evaluation ignored variations in care.

In this paradigm, organizational leadership was fueled by individual results. Unfortunately, the paradigm caused internal conflicts within organizations by creating competition for resources based upon individuals' perceived contributions to quality. Organizations were hierarchical and incorporated short-term thinking and objectives. This thinking resulted in an inward focus on quality improvement.

In essence, the health-care organization viewed itself as the customer. It failed to recognize the need to meet external expectations of the real customers-patients and payers.

The current environment, however, has witnessed an increased public demand for accountability in health care. Simply stated, patients, payers and others have begun to realize cost is not the sole determinant of quality. Rather, the ultimate outcome of care, based on the processes of care delivery, is the important factor in determining the quality of care. Thus, cost and quality have been linked in the minds of consumers and has become known as cost effectiveness.

The new paradigm stresses the importance of an organization as an integrated system of units that have a mutual and collaborative impact on quality. It demands the organization understands variation in the delivery of care. It also understands that an emphasis on quality creates a chain reaction that leads to lower costs.

Within this context, organizational leadership becomes obsessed with quality within every component of the organization. It stresses unity of purpose; it evaluates and finds fault with the system of care rather than penalizing the performance of individuals. It ultimately creates a sense of teamwork rather than competition among individuals. As a result, the organization develops an "outward focus"-the staff, patients and community become the customers.

W. Edwards Deming, a leading statistical quality control expert who is viewed by Japanese industrialists as the "founding father" of quality improvement, identified the "quality chain reaction" (see Figure 1 ) that serves as the visual personification of this paradigm. The model legitimizes the need for organizations to pursue performance improvement in that it directly affects the tangible goals of increased market share and organizational sustainability.

Yet one of the greatest challenges that faces health-care organizations, including those engaged in orthotics and prosthetics, is successfully evolving from the old to the new paradigm. Organizations must learn to apply concepts and methods of performance measurement that use outcomes as the driving concern for evaluating and improving clinical or administrative functions and procedures. The difficulty involved in this process is illustrated by the various methods of quality improvement that range from the most basic to the most sophisticated approaches:

Chart audit: an early attempt to define quality by reviewing individual patient records to determine if the care for specific patients was appropriate and effective (achieved maximum quality).
Unfortunately, this process rarely identified patterns or trends of care across a population of patients.
Medical/care audits: evaluations that were devoted largely to a specific condition or treatment, the results of which may have led to changes in treatment or delivery of care to achieve perceived quality.

This method, while more advanced than simple chart audits, often was based on perceived problems; there were no investigations as to whether problems actually existed. For the most part, this resulted in the expenditure of significant resources with little evidence of "pay-off."
Time-limited, focused studies: evaluations of a treatment protocol used across a patient population over a specified period of time. Until recently, this methodology was the predominant quality improvement mechanism. The focus of assessment was on those functional activities or care processes that affected overall quality.

This methodology paid moderate attention to outcomes. However, little data were used to determine if those outcomes were appropriate to the population. In addition, corrective actions typically were imposed on the human side of the equation; human error was cited as the cause of diminished quality. Given that such evaluations were time-limited, the methodology frequently failed to continuously monitor corrective actions to determine if improvements were made or sustained.
Continuous improvement: as promulgated by health-care standards adopted by such groups as the Joint Commission on Accreditation of Healthcare Organizations, organizations have adopted performance improvement initiatives that identify important aspects of care; establish indicators of optimal performance; apply thresholds of acceptability; implement actions to improve performance; and conduct follow-up oversight to determine if improvement has been achieved.

As initially administered, many organizations tended to use this approach as a more sophisticated evaluation of processes. Again, emphasis on outcomes typically was limited due to the perceived lack of valid data that established optimal patient/condition outcomes with various processes.
Outcomes measurement: rather than emphasizing a particular evaluation approach, outcomes measurement uses the best features of the aforementioned methods: (1) acquiring patient/care data through chart review; (2) identifying optimal treatment/management paradigms affecting patient outcomes through periodic audits; (3) verifying results through focused studies; and (4) establishing monitoring techniques, indicators of performance and thresholds of acceptability as embraced by continuous improvement techniques.

Most importantly, outcomes measurement stresses the value of evaluating treatment and management paradigms on the basis of optimal outcomes by using treatment protocols or management processes that are efficient, efficacious and appropriate. This approach also drives an organization to understand variation in care across patient populations. Thus, it embraces both the desire to assure optimal quality for a specific patient but also encourages improvement in processes that benefit all patients.

Controlling Variation to Achieve Optimal Outcome

The focus of any performance improvement program that stresses achievement of optimal outcomes requires efforts to identify and control variation in the processes of care. Deming identifies two types of variation:

Common causes. These are the variations in a process common to every occurrence of that process. These causes behave in different ways at different times and produce a variation in process outcomes that exhibits no particular pattern but is predictable within definable limits. All processes have common cause variation.

Processes in which only common cause variation is present are said to be "in statistical control." For example, the use of certain components in orthoses or prostheses cause varying stress tolerances depending on the materials used.

In designing and fabricating devices that will produce optimal outcome- restoring or improving function-it is important to monitor the stress performance of such components. Should the components fail to meet either maximum or minimum tolerances, the process of fabrication or materials used for the components should be changed or modified.
Special causes. These are the variations that affect a small group of occurrences and usually are abrupt at onset. Sometimes referred to as "sentinel events," these special causes can come and go sporadically, or they can cause sustained shifts over longer periods of time before changing once again.

Processes that contain special causes of variation are said to be "out of statistical control." Using the previous example, a sentinel event would be the complete failure of a component that could result in patient injury or destruction of the orthosis or prosthesis.

The outcomes-based performance improvement process monitors both types of causes through statistical evaluations. Figure 2 portrays both types of variation. Using time and frequency as the axes, a specific process can be monitored. By averaging the frequency of events plus or minus x standard deviations, upper and lower control limits (or maximum and minimum tolerances) are established.

The intent of the monitoring function is to identify those points along a process continuum in which both common and special cause variations can regularly be observed. In this way, actions can be taken to both eliminate future sentinel events and minimize common variations, thus elevating the efficiency of the process and achieving optimal outcome.

This approach establishes a surveillance system that prevents adverse outcomes instead of a system that devotes resources to correcting adverse events after they occur.

Units of Measure

The initial steps in performance improvement are establishing units of measure and identifying (statistically) optimal outcomes for the processes to be measured. As with many segments of the health-care industry, the availability of outcomes data is generally quite limited. For the most part, little data have been published other than traditional morbidity and mortality statistics for medical/surgical settings.

A similar paucity of outcomes data exists for the care of orthotic and prosthetic patients. However, possible sources for acquiring population-based orthotic and prosthetic outcomes data can include manufacturers' data, academic research, scientific studies or peer data from similar practices.

In the absence of published data, the practice may need to establish baseline outcomes data by "benchmarking" performance of specific processes, patient outcomes or devices. In so doing, outcomes goals are initially established based on available information. As data are generated and evaluated, the outcomes goals are refined to reflect that experience.

In initiating outcomes goals, indicators of performance should be identified for which statistical outcomes are established. Table 1 sets forth possible indicators for use in O&P care. They span both administrative and clinical outcomes.

Regardless of the indicator used, outcomes measurement demands the use of data and rates of occurrences to determine if a process is functioning efficiently and achieving the desired outcome. Two essential types of data should be acquired: rate-based and sentinel-event data.

A rate-based indicator measures, by proportion or ratio, the occurrence of a particular event as compared to the universe of events (for example, initial appointments within x time from referral versus all new patient appointments). Should a process goal or outcome not be achieved, the organization's obligation is to initiate systems changes that will eliminate the deficits causing the unacceptable rate.

A sentinel-event indicator identifies an event or phenomenon that always will generate further analysis and immediate corrective action. In general, sentinel events are viewed as immediate threats to improved quality and should elicit an intensive evaluation to prevent similar occurrences in the future (see Figure 3 ).

In establishing such process or outcomes thresholds, it should be noted that under a perfect set of conditions, the goal always should be 100-percent achievement of an outcome and 0-percent recurrence of a problem.

However, in health care, including O&P, events beyond the control of the practice frequently will prevent achievement of "zero defects." Thus, it is important to understand that setting thresholds of acceptability (rate-based or sentinel) will likely need to reflect those uncontrollable events (for example, patient noncompliance with the proper use of the orthosis or prosthesis). While 100-percent achievement may not be realistic, the spirit of continuous improvement dictates that thresholds should continue, over time, to be elevated, thus ever improving the identified process and outcome.

Performance Improvement Tools

An organization may implement a variety of administrative methods to pursue performance improvements. A consistent theme evident in successful programs is the methods must embrace logical evaluative techniques and use methodical instruments for understanding complex processes and data.

The two most frequently used approaches are illustrated in Figure 4 . FOCUS and PDCA techniques encourage a systematic strategy for carefully analyzing and continuously monitoring and improving processes of care. But most importantly, the selection of processes must be guided by the objective of improving outcomes associated with the processes. Therefore, these methodologies stress the use of evaluation tools, including the use of flowcharting (see Figure 5a ) and fishbone diagramming (see Figure 5b ). These instruments facilitate the isolation of key functional activities of a process that influences outcomes. They help direct performance improvement activities on discrete components of the process that will most directly influence quality. In other words, they facilitate the analysis of cause and effects, support the capturing of objective data, and document the process under evaluation.

Conclusion

Several important conclusions are evident in outcomes-based performance measurement mechanisms:

The evaluation of a specific process and its associated functions must produce measurable outcomes. Merely evaluating and modifying a process without the objective of linking such refinements to a measurable outcome renders the continuous improvement model ineffective.
Outcomes-based performance measurement programs require the rigorous use of statistical tools and the acquisition and use of objective data to reach conclusions. They also demand continuous monitoring to evaluate the effectiveness of changes made to the process based on documented changes in outcomes.
A continuous quality improvement strategy demands the commitment of all people associated with the organization, including its administrators and clinicians. The organization must be viewed as a cohesive system of processes, which, in interacting with one another, ultimately affect outcome.
Finally, as orthotics and prosthetics organizations begin to implement continuous improvement programs, results of these efforts must be used to objectively demonstrate improved patient-care quality. In so doing, facilities will enhance their abilities to attract patients and strengthen reimbursement relationships.

LANCE 0. HOXIE is executive director of the American Board for Certification in Orthotics, Prosthetics & Pedorthics Inc.

References:

Bradford HA. Principles of medical statistics. 9th ed. New York: Oxford University Press, 1971 :6.
Berwick DM. Continuous improvement as an ideal in health care. New Engi J of Med 1989; 320:53-6.
Deming WE. Out of the crisis. Center for Advanced Engineering Study, Massachusetts Institute of Technology, 1986.
Joint Commission on Accreditation of Healthcare Organizations, various organizational standards of accreditation.
Juran JM. Juran's Quality Control Handbook. 4th ed. New York: McGrawHill, 1988.