American Academy of Orthotists & Prosthetists

Clinical Research

Thomas R. Lunsford, MSE, CO

ABSTRACT

This article is a discussion of the basic concepts and categories of clinical research. Because the current state-of-the-art in research is the result of more than 200 years of developments, the essential elements such as interventions, control groups, randomization, blindness, placebo, the Hawthorne effect and informed consent are reviewed.

Introduction

Clinical research for the orthotist and prosthetist means a systematic process for formulating and answering questions about the uses of, basis for, and effectiveness of orthoses and prostheses. The need for research to develop O&P management as a science and an art has been documented (1-5). The American Academy of Orthotists and Prosthetists (AAOP) at both national and local levels, as well as the editorial board of JPO, recognize the need for research to advance the O&P profession and industry.

One has only to attend either of the annual meetings to verify the paucity of research. Similar results would occur from reviewing past issues of JPO (aka CPO and Orthotics and Prosthetics). The JPO editorial board has developed specific goals and an overall plan to assist practitioners in conducting research.

Reimbursement and funding agencies are beginning to demand validating research for the techniques and devices associated with O&P. As new and improved devices emerge from higher technology, clinical research must be conducted to validate their effectiveness and help justify their reimbursement.

Also, new ideas and devices should be subjected to rigorous clinical research as a prerequisite to endorsement by NCOPE, our educational credentialing organization. Moreover, curricula at our educational institutions should identify devices proven effective through clinical trials, those that show promise or are widely advertised, and those that have no impartial evidence to demonstrate their efficacy.

It is imperative that the O&P profession and industry develop and implement clinical assessments, as well as instruments, that are bias-free and capable of providing quantifiable measures of patient (consumer) benefits derived from use of the devices provided. Patient assessment and devices must be valid, reliable and have demonstrable effectiveness.

Research Skills

You probably have latent research skills if you find yourself not accepting what people say or write. Ask yourself, or others, or the literature, is this really so? This is an attitude practitioners must cultivate to perform critical assessment. A researcher must have an inquiring mind and a skeptical outlook. Enthusiasm is also a requisite, but it must be tempered by a capacity for sustained concentration and a complete lack of bias. It is not easy to be cool and objective when continually working under pressure in a busy environment. However, if the results of your observations are to have any value, they must be untainted by any bias.

Some confuse research with development or engineering. Many practitioners are crafty and like to tinker, and, in much the same manner as an engineer, design new and improved devices, components and processes. Developing a "better mouse trap" requires a keen clinical sense and a bent for creativity. Sometimes a new device is developed not necessarily to improve function, but to reduce cost or enhance durability, etc. There is a certain freedom to this process, and few rules apply.

Research, however, is more structured, more disciplined and has many rules. Research may involve evaluating or comparing a new orthosis or prosthesis to conventional devices. Moreover, the developer of a device is not considered to be an impartial judge of the performance or comparison of the new device.

Students frequently claim they are going to the library "to do research." Although a review of pertinent literature is an important phase of research, the process of reviewing literature is not, in itself, research. The word "search" is a more appropriate term to describe reviewing literature.

Clinical Investigation

Every practitioner who performs a clinical investigation, such as an analysis of patients treated with a specific device, or a trial on the treatment of a painful or unstable knee with knee orthoses, will be contributing to the sum of knowledge in our profession, which is the aim of all research. For example, to improve or validate your clinical knowledge, you might need to know why patients with a given disease or in a condition category respond well to a certain orthosis or prosthesis while others whose conditions are apparently comparable do not improve using the same device.

If O&P is to remain independent, and at the same time grow and improve, an understanding and appreciation of the role of clinical research is necessary. All practitioners should make some contribution to the scientific validation of their practice, and that means discovering and validating the concepts and principles on which O&P devices are based.

The research mentality (it is as much a state of mind as it is a process) should be a working tool for the practitioner. In the words of Pruden, "We need to strip research of its aura of glamour or mystery, to reduce it to a size that fits our laboratory workbench and to make it a realistic part of each of our daily activities (6)."

If we as orthotists and prosthetists are not motivated to do our own re- search, others will do it for us. Those others will gain knowledge, and eventually our identity as a unique profession will be lost.

Defining "Research"

If you were to examine textbooks on research, you would discover there is no single, agreed-upon definition of what is meant by the term research. Some authors define research in a restricted sense, involving scientists in a laboratory who manipulate variables to study the effect on other variables. Other authors state that practically anything done to discover knowledge qualifies as research.

While the definitions of research differ considerably, similar elements typically appear. To have a common frame of reference about the concept of research, the following definition, which summarizes and integrates these similar elements, has been proposed: Research is a way of discovering knowledge through the systematic investigation of a problem (1).

Some of the implications of this definition are (a) research is a way and not the only way to discover knowledge, (b) research is a systematic process, and (c) research is problem-oriented. Clinical research in O&P is important in validating clinical practice, and all research eventually leads to the central question: What difference does it make to the patient?

Classification

A popular classification of research that has some degree of acceptance distinguishes basic, or pure research, from applied research. Basic research tends to be abstract and general and is conducted either to generate new theories or to gain knowledge or understanding.

The results of basic research may have some practical usefulness, but the researcher does not start with a specific application in mind, e.g., examining a new molecule or compound that may eventually form a new fabrication material. Conversely, applied research is conducted so that a device or technique can be improved or to answer specific questions.

The researcher has a particular application in mind when designing the study. The O&P practitioner has, as a rich source of research topics, many questions about devices, processes and ways to help human beings live useful, productive lives. O&P practitioners tend to be interested in practical, applied matters.

Research is sometimes classified according to where it is conducted, such as clinical research or laboratory research, or by the area in which it is conducted, such as educational research or administrative research. Perhaps a more useful classification is by differences in the actual method used in the research process.

A common distinction is between experimental and descriptive research. Experimental research is characterized by the manipulation of a variable (the independent variable) to study its affect on another variable (the dependent variable). It is the approach used when the researcher wishes to examine cause-and-effect relationship and is prospective. Descriptive research is the method used when the purpose of the study is to describe a situation, a group, an individual or other phenomena. Descriptive research includes surveys, correlational studies, trend analyses, developmental studies and methodological studies and is often retrospective.

A retrospective study is a review of cases from the past (7). The study is conceived after patients have been treated. Typically, one searches the medical charts to identify a series of patients in which one is interested. Their charts are pulled, and relevant data are collected and analyzed.

A prospective study is conceived before patients are treated. The hypotheses and experimental design are determined before the onset of the study. Protocols are written and forms are used to ensure all relevant data are collected. A randomization process may be devised for enrolling patients into either the experimental or control group. The study is carried out either for a specified time or until a specified number of patients is enrolled.

In principle, a retrospective study may be just as good as a prospective one, provided record keeping has been thorough. In practice, retrospective studies are plagued by missing data and poor control. Some patients are not identified, key points are not recorded in the charts, and timing and records of follow-up visits are haphazard. The identification of a suitable control group is often impossible. There is no way to assess bias or inconsistency in data collection. All of these factors impart a shakiness to retrospective studies. Realistically, there is very little control.

Evolution of Research

The evolution of clinical research (clinical trials if it involves patients) dates back to the 18th century and includes several important concepts, e.g., intervention, control groups, randomization, blindness, placebo, Hawthorne effect and informed consent (8,9).

Intervention

Clinical research employs one or more intervention techniques. An intervention technique in O&P might be the evaluation of a new socket or ankle-foot orthosis thought to be more effective. The intervention device should be applied to subjects in a standard fashion in an effort to change a therapeutic or functional aspect (e.g., walking, strengthening, range of motion, vocational goal, etc.) of the subject.

Follow-up of subjects who have been trying a new device over a period of time may measure the natural history of a disability, but it does not constitute a clinical trial. Clinical research requires active intervention and a real time evaluation of the prospective device. Without active interventions the study is observational or retrospective because no experiment is being performed. Intervention may be thought of as a new prosthetic foot or AK socket or a new knee or cervical orthosis.

Control Groups

Clinical research must contain a control group against which the intervention group is compared. The control group must be sufficiently similar to the intervention group that outcome differences may reasonably be attributed to the intervention. Most often a device (intervention) is compared with the best current alternative. If no such standard exists, the subjects in the new- or test-device group may be compared to subjects who have no device during the test or measurement period.

Randomization

In the simplest case, randomization is a process by which each subject has the same chance of being assigned to any one of the devices being compared or to the control device. An example would be the toss of a coin, in which heads indicates the testing of the intervention group of devices, and tails indicates the control device group or no device.

Even in the more complex randomization strategies, the element of chance underlies the allocation process. Of course, neither subject nor investigator should know what the assignment will be before the subject's decision to enter the study. Otherwise, the benefits of randomization can be lost.

Blindness

Bias is one of the main concerns in clinical research. Bias has been defined as "systematic error or difference between the true value and that actually obtained due to all causes other than sampling variability"(1O). Bias can be caused by conscious factors, subconscious factors, or both. Bias can occur at a number of stages during clinical research, from the initial design through data analysis and interpretation.

The general solution to the problem of bias is to keep the subject and the investigator blinded to the identity of the device (or intervention) being evaluated. Obvious bias exists when the developer or designer of a new or innovative device is involved in its evaluation. Evaluation by a second or third party is necessary to preserve the designer' s integrity.

Placebo

A special kind of intervention (treatment or device) used in some experiments is the placebo. The subjects think they are receiving an experimental device when in reality they are not.

In a study where control subjects are identified but not aware they are being used as controls, while experimental subjects are well aware they are participating in a study, scores (outcome) for the experimental group might be higher solely as a result of the subjects' knowing they are involved with the improved design or higher technology.

Interest or motivation may alter the behavior of the experimental group. Including a group using placebo devices that is told it is using the new or high technology device will identify whether this knowledge is responsible for enhanced performance.

In traditional pharmaceutical research the placebo is an inactive substance usually shaped and colored to look like the active drug. The placebo is given to a control group to test the therapeutic effects of the active drug. The patients (subjects) never know which drug they are getting. This may be difficult in O&P since many patients are very sophisticated and will be able to identify which of the devices being evaluated is the new or intervention device. Two researchers. Benson and Epstein, have shown that the placebo alone can produce beneficial therapeutic effects (11).

Hawthorne Effect

A concept distinct from placebo is the Hawthorne effect. This phenomenon was discovered between 1927-1932 by Elton Mayo and a group from Harvard University at the Hawthorne facility of Western Electric Co. (12). These researchers originally set out to determine the effect of lighting on fatigue and efficiency in factory workers.

The researchers discovered that no matter what they did to the lighting, production improved, and the factory workers reported less fatigue. Obviously, Mayo learned very little about lighting variables and fatigue, but discovered an important phenomenon. He discovered knowing they were participating in an experiment significantly changed the workers' perceptions of their job situations and improved their productivity. This study demonstrated the beneficial effects of being made to feel important and different in an otherwise dull work setting.

Clearly, the Hawthorne effect could operate in a clinical research study. If the subjects know they are participating in a research study, they may be motivated to perform better. Legally required informed consent forms promote the Hawthorne effect. Therefore, the clinical researcher must be aware that the Hawthorne effect, like the placebo, may be the reason a new knee orthosis or prosthetic foot, for example, produces significantly improved function.

Informed Consent

To assure compliance with the principle of informed consent, most states have health and safety codes that require investigators to inform people participating in medical research of their rights by providing each subject with a written copy of the "Rights of Human Subjects in Medical Experiments" (13). Informed consent documents generally consist of two parts: 1) "Rights of Human Subjects in Medical Experiments" and 2) a written consent form, specific to the study being conducted, which describes in layman's terms each of the topics covered in the "Rights of Human Subjects in Medical Experiments."

A truly informed subject is likely to be a better complier. Therefore, for scientific as well as ethical concerns, the subject (or guardian) in any clinical research should be clearly instructed about the study and told what is expected.

Sufficient time should be spent with subjects to ensure they understand the specific objectives of the study and the impact they will have upon the results. Subjects should be encouraged to consult with their families or physicians before participating. A question-and-answer sheet concerning the study is often helpful.

Most hospitals, all federal agencies providing grant support, and any other group serving in a consulting or controlling capacity over research projects require compliance with the recommendations of the World Medical Association regarding medical research. This document, originally entitled "Declaration of Helsinki," was adopted in 1961 and later revised at the 19th World Medical Assembly in Tokyo in 1975 (13).

In January 1981, the Department of Health and Human Services and the Food and Drug Administration published in the Federal Register final regulations amending the basic policies for the protection of human subjects in research (13-15). The regulations, effective July 27, 1981, apply to research funded by those agencies.

Conclusion

Participating in clinical research will make you a better O&P practitioner. Clinical research provides you with knowledge that distinguishes you from unqualified individuals who wish to practice your craft. You may have heard that research enhances the stature of health professionals. Indeed, the most respected professions have a strong and growing research base. D

Thomas R. Lunsford, MSE, CO, is director of the orthotic department at The Institute for Rehabilitation Research in Houston. He also is president of the Academy.

References:

1. Cox RC, West WL. Fundamentals of research for health professionals. 2nd ed. Laurel, N.J.: Ransco Publishing Co., 1986:1-14.
2. Horen IR. Back to the future. O&P Almanac 1989;38:5:49.
3. Lunsford TR. Research, why? O&P Almanac 1989;38:5:53.
4. Teague CW. Promoting research for the profession. O&P Almanac 1991 ;40:9:26.
5. Retzlaff K. Leading O&P education into the 21st century. O&P Almanac 1992;41 :4:32.
6. Pruden EL. et al. Ins and outs of research: problem to publication. Am J Med Technol. 1970;36:209.
7. Madison M, Chapman MW. Organizing a scientific clinical research project. Orthopedics - Clinical Research 1989;12:10:1297.
8. Bull JP. The historical development of clinical therapeutic trials. J Chronic Dis 1959; 10:218-48.
9. Lilienfield AM. Ceteris Paribus: the evolution of the clinical trial. Bull History Medicine 1982;56:1-18.
10. Wausner JS, Bahn AK. Epidemiology: An introductory text. Philadelphia: W.B. Saunders. 1974.
11. Benson H, Epstein MD. The placebo effect: a neglected asset in the care of patients. JAMA 1975;232:12:1225.
12. Todes JL. Management and motivation: an introduction to supervision. New York: Harper and Row, 1977.
13. Guiding medical doctors in biomedical research involving human subjects. Tokyo. 29th World Medical Assembly 1975. Distributed by the Office for the Protection from Research Risk, Office of the Director. National Institutes of Health, Bethesda, Md.
14. Final regulations amending basic health and human services policy for the protection of human research subjects. Part X. Federal Register 1981 ;46: 16: 8366-92.
15. Protection of human subjects: informed consent, part IX. Federal Register 1981 ;46:17:8942-80.