Survey of NCOPE Residents

Abstract

Surveys were mailed from in the first half of 1998 to all NCOPE residents in orthotics and prosthetics who served in a residency from 1995-1998. These surveys asked the residents about the availability of residency positions and about various aspects of their residencies, including their perceptions of its strengths and weaknesses, evaluation of residents' progress, preparation for the ABC certification examinations, and other factors. A significant number of surveys were completed and returned. Residents generally responded positively about their residencies and the concept of residency, but several potential problem areas were noted. Limitations of this research study as well as recommendations for further study are also discussed.

Acknowledgements

The author wishes to thank and acknowledge the contributions of Ms. Robin Seabrook and the National Commission on Orthotic and Prosthetic 'Education, Miki Fairley and O&P Business News, residency instructor and mentor Gordon W. Bosker, CP, CPed., Alabama Prosthetics and Orthotics, Inc., and Michael Hudgins, Ph.D. Their knowledge, assistance and generosity were vital to the completion of this research study.

Introduction

In 1995, the Board of Directors of the American Board for Certification in Orthotics and Prosthetics (ABC) adopted the National Commission on Orthotic and Prosthetic Education's (NCOPE) residency program as a requirement for all new graduates prior to being eligible to sit for the ABC practitioner certification examination. This post-graduate residency education/work program was designed to bring orthotics and prosthetics graduates to a higher level of professional competency and maturity than ever before, keeping pace with the ever-increasing responsibilities of the O&P practitioner while preparing the residents to take their upcoming certification exams. The implementation of this program made the field of O&P one of the first allied health professions to mandate post-graduate education prior to certification or licensure. NCOPE Chairman Robert Lin, CPO, called the new mandate "...the most significant stride toward professionalism and appropriate training we've ever made." (4)

Despite this, the concept of a residency in O&P is not new. There were five residencies listed in the Spring 1988 issue of Orthotics and Prosthetics (1,2). The newly founded residency sites were bringing a more structured, formalized approach to post-graduate training, whereas the quality of the postgraduate experience of other students varied greatly (largely depending on the scope of practice of the graduate'§ first employer and the employer's willingness to share his or her clinical knowledge and experience). These fledgling programs were early attempts to bring O&P out of the realm of a trade or craft, performed by skilled artisans with little or no real medical or biomechanical training, to a profession, performed by certified professionals guided by principles that are grounded in not only technical expertise but a thorough knowledge of clinical skills and scientific and biomechanical concepts. This has been the goal of ABC since its debut. There remained only a small number of these residency programs until 1995, however, until NCOPE's residency program concept was fully adopted by ABC.

Presently there are just over 100 NCOPE-accredited residency sites, with the number increasing steadily (16). In this period of rapid growth and acceptance, there has been little time for NCOPE and ABC to ensure that each and every new resident will receive adequate exposure to all the concepts and patient types listed in the Essentials and Guidelines for Residency Program A in Orthotics and Prosthetics (13). According to NCOPE, "It is the responsibility of each facility to ensure its O&P residency program fully meets the Essentials as approved by NCOPE," and "Institutions, facilities, or organizations and agencies offering programs in resident orthotic and prosthetic education must assume responsibility for the educational validity of all such programs" (I 3). The guidelines for this responsibility are vague, designed to allow for a great deal of variation between residency programs. There is at present no method of "quality assurance" other than the ambiguous statements of responsibility in the Essentials. The residency program, so far, functions on an "in good faith" basis.

What makes a residency "good" for the resident? What qualities must a residency program demonstrate in order for a resident to have a satisfactory residency experience, and to what degree must these characteristics be demonstrated? Are residencies meeting the residents' expectations? Are residents meeting their program's expectations for them? How does one assure a quality experience? Students preparing to graduate must ask these questions as they try to assure themselves the best work experience and the most thorough preparation for the upcoming ABC exam.

In order to identify and examine these issues surrounding the residency system, a survey was distributed to current and past NCOPE residents, 1995-1998. This survey asked about various aspects of their particular residency and about the search for a residency position. Since there had never been such research on orthotic and prosthetics residents, the survey results represent the first "evaluation" of NCOPE residencies by residents. The comments collected from the surveys shed valuable insight as to how the residents feel about their residency positions, residency staff, and how residency will affect their future as O&P practitioners.

Research Goals and Objectives

Concerning residency research, there is no major source of background information. The NCOPE residency program is very young, and as such there are not a lot of printed materials about the residency program and its mandate by NCOPE. The many types of residency programs (private practice, hospital, university, etc.) and the different types of experiences provided by such programs necessitate the broadness of the topic and do not lend themselves to traditional experimentation and/or deductive research. It is hoped that the results of this research will lead to the formation of conclusions or hypotheses that will eventually by proven or disproven in such a manner.

The goals and objectives of the survey project were as follows:

1. To determine the qualities the residents feel the "ideal" residency should possess/ demonstrate (i.e. broad scope of practice, frequent evaluation/feedback, etc.).

2. To determine in what areas respondents feel their residencies lack the above qualities or fail to demonstrate those qualities.

3. To determine the numerical and geographical availability of NCOPE residency sites relative to the number and location of new graduates from the orthotics and prosthetics schools.

4. To determine the efficacy of current O&P education in preparing the graduates for the residency experience.

5. To determine if the salary of the residency meets the residents' needs and/or expectations.

Method

In order to reach as many residents as possible, surveys were distributed to all current NCOPE accredited residency sites. It was hoped that, even if a residency. site did not currently have a resident, the site would perhaps forward the survey to one of their past residents. Surveys were also mailed directly to every resident for whom it was possible to obtain a home or business address. This was possible only for a few groups of residents. (It is the policy of NCOPE and the O&P schools not to release names and addresses of current or past students for reasons of privacy, and therefore all requests for contact information were denied). NCOPE did, however, very graciously provide the names and addresses of all currently accredited sites, pre-printed on address labels. This greatly facilitated the distribution of surveys.

Surveys were also sent to O&P certificate programs in hopes that some students would be pursuing their second discipline (orthotics or prosthetics) and as such might have completed a residency in their first discipline. In total, the number of surveys directly mailed (with stamped, self-addressed return envelopes provided) reached 177.

Surveys were also distributed in other ways. O&P Business News printed the survey and forwarded the responses appropriately (12). Lastly, the survey was "posted" to OANDP-L, the orthotics and prosthetics Internet listserver. Subscribers to the listserver could then either reply to the survey "on-line" via e-mail, or by printing it out and either completing the survey or giving it to a resident the subscriber knew. Due to these alternate methods of survey distribution, it is unknown as to the exact number of contacts the survey could have made. It was felt that the statistical "penalty" of uncertainty in the number of potential contacts made was outweighed by the possibility of reaching more people with the survey.

Research Respondents

There were 62 actual completed responses to the survey. In addition there were 14 surveys that were returned incomplete, with notes attached asking to receive a copy of the report once it was completed. Among the 62 completed surveys were two responses taxed (from O&P Business News) and twelve "on-line" surveys. Seven of those twelve surveys had been printed out and distributed to residents-the other five responded directly via e-mail. Therefore, 48 surveys were returned directly by mail, representing a 27. 1 % response rate for the 177 mailed surveys. Calculation of the response rates for taxed and e-mailed surveys is impossible because of the unknown number of contacts made by each alternate medium.

The respondents were two-thirds (66%) mate and one-third (33%) female, with an average age of 29.5 years (males averaged 30.53 years, while female respondents averaged 27.41 years). The respondents had an average of 4.97 years of experience in the field of orthotics and prosthetics, with males averaging 5.01 years of experience and females 4.88 years. One male respondent is a transtibial amputee secondary to trauma in childhood; he consequently listed his experience as twenty-two years. This was by far the largest amount of previous experience. Without the respondent's 22 years, the male average fell to 4.37 years.

The respondents came from a variety of schools. Fifteen residents came from Northwestern; nineteen from Newington; five from University of Texas Southwestern; nine from California State at Dominguez Hills; eight from University of Washington; one from Century College; and one from Rancho Los Amigos Medical Center. Twenty-seven point four percent (27.4%) of the residents had studied both prosthetics and orthotics; this is due to the fact that bachelor's degree programs such as University of Washington and Southwestern study both prosthetics and orthotics in a two-year period, as well as some students from the certificate programs who had studied both disciplines.

Results

Number of responses: 62
Number of surveys mailed: 177
Surveys in O&P Business News: unknown
Surveys on OANDP-L: unknown
Direct mail response rate: 27.1%
(response rates for other survey formats unknown)

Research Data

Orthotics and Prosthetics Schools Attended

Statistics for question 3a:

Discussion of Results

Geographical and Numerical Availability of NCOPE Residency Sites

The first set of questions on the survey asked about their search for a residency position. Students sent an average of 18.4 resumes to accredited residency sites and other facilities (in hopes of convincing sites to become residencies and take them as a resident). The number of resumes sent by residents ranged from 0 (in the case of residents with a previous employment agreement) to I 00 in one case, with several residents sending out over 60 resumes. Eighty-three point one percent (83. 1)% of graduates were able to find a residency before graduation from their program; 16.9% were unable to do so. Two students in that 16.9% did not find residencies at all, and instead found work in other fields.

An issue currently facing graduates is the availability of NCOPE-accredited residency sites. There are, at present, IO 1 active and accredited residency sites, with new sites being added at the rate of "about one a week," according to NCOPE Executive Director Robin Seabrook. In addition, several sites have more than one residency position available, bringing the total to 82 orthotic residency positions and 81 prosthetic residency positions as of May 29, 1998 (16). Despite this rapid growth of both the number of sites and the number of positions, this number is still not very large compared to the number of students that graduate from O&P programs. In May 1997, the Newington Certificate Programs in Orthotics and Prosthetics graduated a total of 41 students (both disciplines). Assuming each of those 41 students finds a residency, this represents almost one-quarter of the available residencies. Other programs, such as California State University at Dominguez Hills and Northwestern University, graduate similar numbers of students per academic year. While most graduates are able to find a residency, it is often a long and difficult process.

The issue is further complicated when students from schools not yet under the CAAHEP accreditation umbrella apply for and obtain residencies. Century College, University of Texas Soutliwestem, Rancho Los Amigos Medical Center, and University of Washington are not yet CAAHEP-accredited as well as NCOPE-accredited, and as such their students/graduates are not required to complete residencies. Graduates of these schools, however, are seeking residencies, choosing a structured experience over the "pot luck" of the 1900-hours requirement. This was demonstrated in the graph "NCOPE Finds Graduates Seek Residency in Lieu of 1900 Hours" (9). This choice of residency over 1900 hours has resulted in waiting lists for available positions, and students of these non-CAAHEP programs compete unfairly against CAAHEP program graduates -- these students can apply to as many residencies as they wish, and, if they are not accepted by a residency, the student can just find a 1900-hours position as a board-eligible prosthetist. The CAAHEP program graduates' future depends on him or her finding a residency site to take them. This can be a nerve-wracking experience, as is evidenced by the survey responses. This advantage of the non-CAAHEP students will disappear within the coming year, as in - 1999 all current O&P school programs will fall under the CAAHEP and NCOPE accreditations, thus requiring students from all schools to complete a residency.

Geographical availability of residency sites is another issue. Fifty-eight point four percent (5 8.4%) of residents were able to find a position that was available within I 00 miles from their home, while 3 9.6% were forced to look farther than I 00 miles for their residency. Forty-five point three percent (45.3%) of residents cited restrictions to their job search (family, home ownership, spouse's employment, children, etc.) that prevented them from looking for a residency outside of the area in which they lived. Fifty-four point seven percent (54.7%) explained that they had no such restrictions and as such were able to look throughout the country. Currently, there are NCOPE residency sites in only 36 states, and as such relocation is often required of residents (this number may change by the printing date of this report).

Strengths and Weaknesses of Residency Programs

Residents were asked to list what they considered to be the strengths and weaknesses of their residencies. These responses were read individually and grouped into categories, based. on the number of comments about a particular aspect of their residency.

First, residents want a large and diverse caseload of patients with quality patient-practitioner interaction. This was evidenced by the large number of respondents listing these qualities as strong points of their particular program -- twenty-two residents listed a broad scope of practice or diverse patient caseload as one of the strengths. Twenty respondents also listed high volume of patients as a major highlight. Another fifteen respondents spoke of the high quality of their interaction with their patients. These three strengths represented three of the five most frequently mentioned residency strong points.

The importance of this aspect of the residents' experience was also stressed by the eighteen respondents who listed a lack of this diversity of patients as their residency's main weakness. orthotics residents as well as prosthetic residents listed this lack of a broad scope of practice as a shortcoming of their programs. orthotics respondents most frequently listed spinal orthotics and pediatrics as the two areas most often lacking. Prosthetics respondents almost unanimously replied that upper extremity patients and pediatric patients were not often seen by their facility (although this is not uncommon in most prosthetic facilities around the country).

Sixteen residents considered their residency director and/or staff and the experience of those practitioners and technicians a major strength. No one listed a lack of experienced staff as a weak point.

Residents also value a program that places emphasis on quality instruction and focus on the educational aspects of the residency, rather than emphasizing production. Quality of instruction and focus on the educational aspects of the residency was listed by ten residents as a strength. Lack of instruction and inability to have questions answered adequately was listed as a weakness by nine residents, and eight residents also spoke of overemphasis on production as being a major weakness. Another similar comment related to these two characteristics was a lack of patience with the resident, with five respondents citing this as a weakness of their residency. Four residents also listed a lack of respect for the residents under their program's weaknesses.

Organization and structure to the residency program is a major strength in the eyes of residents, with nine respondents citing this as a strength and ten residents citing a lack of structure or poor organization as a weakness of their particular residency. Some residents citing lack of organization as a weakness (four) also mentioned that they were the first resident the site had ever had.

In addition to the qualities listed above, residents described many other qualities as strengths and weaknesses of their residencies. These strengths (with the number of residents that listed them) included: NCOPE accreditation (four); multiple affiliations (two); hospital experience (three); caring instructors (two); close to home (three); open communication between resident and practitioners/ staff (two); freedom to try new ideas (four); teamwork (three); autonomy to make own decisions (four); cross exposure to the other discipline (six); high ethical standards and professionalism (three); reputation of facility (three); and having previously worked at the facility (two).

Other weaknesses included: paperwork (two); lack of autonomy to make decisions (two); too much fabrication (two); poor communication (four); assessing resident's needs (one); location (six); low salary (two); lack of adequate supervision (two); director unable to admit mistakes (three); facility practice too small (one); staff hesitant to give enough responsibility (one); director verbally abusive (one): lack of business exposure (two); and high workload (one). Four residents listed that their residency had, in their opinion, no weaknesses.

Scope of Practice and Variety of Clinical Experience

One of the major concerns of a resident is that he or she receive exposure to an extremely broad base of patients. NCOPE's Accreditation Manual for Sponsoring a Residency states: "Clinical experience must be of sufficient variety and volume to afford the resident adequate educational experience in orthotics and/or prosthetics in the two (2) primary areas and within the following phases of care when appropriate: Primary areas: 1. Pediatrics; 2. Adult. Phases of Care: 1. Acute Care; 2. Rehabilitation Care; 3. Chronic Care." (I 3). As the residency is the primary institution where the resident will gain the majority of his or her clinical experience, it is imperative that the resident have extensive patient contact with all of the above patient types and phases of care.

Within each of these primary areas, however, there are many subdivisions. For example, a prosthetics resident's experiences should include all levels of amputation of all etiologies (trauma, disease, congenital malformation, etc.), such as hip disarticulation, transfemoral, transhumeral, transtibial, transradial, forequarter, etc. This is often first and foremost in the resident's mind as he or she "shops" around for a residency site. With the current availability of residency sites, however, self-assurance of a broad spectrum of patients may have to take a back seat to simply finding a residency site at all (see "Geographical and Numerical Availability of NCOPE Residency Sites). Therefore, the survey asked about the scope of practice the residents are exposed to, and what, if any, efforts were undertaken to assure a well-rounded experience in the event the resident felt the residency was severely lacking in any particular type of patient.

Seventy-eight percent (78%) of respondents replied that yes, their residency's scope of practice was adequate or broad enough. When residents chose to explain further, it was to explain their concern despite being generally pleased with the variety of patients:

"I feel it was adequate. Some things were missing, such as working with adults, but the range of disorders/diseases ranged across the spectrum so that I don't feel that it was limiting at all.,,

"Very broad (except halos and hip disartics)."

"In general, the scope is broad except for congenital pediatric cases (very few of those)."

"Overall, yes. However, this setting has virtually no pediatric care and this concerns me."

"Broad enough in regards to UE and LE orthotics ... but no spinal."

"Working in a hospital has given me a lot of freedom to see a broad spectrum of patients."

"Broad experience, but lacked some categories (i.e. halos)."

Twenty-one point two percent (21.2%) rated their residency's scope of practice as "too limited." Their comments were as follows:

"No, we see no spinal and the owner feels it is not his obligation to have us gain a wide exposure ... only what walks in the door. We do not attend clinics or other facilities as it doesn't make the company money."

"We have seen no peds. No spinal, no sports injuries and no halos. On our own we have gone to a Shriners and a spinal doctor's office, and seen a few things on our vacation time."

"My residency was at a Shriners hospital, therefore my experience was limited to pediatrics and adolescents. But the way I see it, a big kid is no different from an adult, at least anatomically."

"No, it is not broad enough. We're missing a lot of spinal and acute. Pediatric isn't that strong either. No halos or scoliosis either."

"Too limited. But this is not the fault of the residency program. Numbers of patients can vary from year to year. I was glad they paid $1,000 per resident plus my salary to TRAIN ME because my school did not."

"Too limited. At the facility where I am currently doing my residency, too much geriatrics and not enough pediatrics. Also in orthotics, we don't see many upper extremity cases or adolescent scoliosis cases which are prerequisites for board exam success."

From these responses, it would appear that some facilities are unable to offer a broad enough scope of clinical experience. NCOPE understands this, and for this reason encourages residency sites to form "affiliations" with other O&P facilities so that the resident can learn at the affiliation site what is considered lacking at the residency site. This does not necessarily mean, however, that the resident is automatically going to see every type of patient imaginable. This is described in their Accreditation Manual:

"It is the responsibility of each facility to ensure its O&P residency program fully meets the essentials as approved by NCOPE.... if a facility is unable to provide the full scope of experience for the resident then NCOPE encourages affiliations with multiple sites .... The director and teaching staff define as well as identify what experiences the resident will be exposed to based on your facility practice. The objectives in part II must be met. Ultimate responsibility for the program lies with the director."(13)

This presents an unusual situation in that the resident and the director may feel entirely differently about what makes a residency "complete." Residents had varying experiences with regards to taking steps to ensure they received a wide range of patient types:

"If additional experience was needed, no steps would be taken to ensure a broad educational experience in my residency. My residency is about business, patient care, and cash flow."

"I have an opportunity to work with another O&P facility to gain upper extremity experience if I feel it is necessary."

"I will be training with a CPO from Canada on hip disarticulation patients and a competitor I did my student work for will guide me through halos."

"Whenever it was thought that I have not been properly exposed to something, they do set me up with PT or some other part of the hospital for exposure."

While these above comments showed many residencies to be extremely accommodating, this was not the case for all:

"No, I have in all honesty been very busy as a resident incorporated into the day-to-day patient load rather than an arrangement that would afford samplings of broad spectrums of pathologies in order to fulfill the NCOPE Resident checklist ... There is no doubt, however, that as my residency closes, that I remain somewhat short-changed in overall. I'm assuming this will change in time as my career moves around."

"The director didn't set us up but we took the initiative and went by ourselves ... we also had to make all the contacts by ourselves. However, we were allowed to use comp time (i.e., hours of overtime we had worked) to go during working hours."

"I was told to do it on my own time."

"No ... they haven't let us go observe anywhere else because that would not be directly profitable to the company. The director said, it is not our responsibility to show you what doesn't come in the door, as per NCOPE."

"I've had to take it on my own to set up the training in other areas which I feel are required."

"No, that would not directly benefit the company as we are going to leave in a year anyway.

"It has been promised but seems unlikely."

Some residencies apparently regard affiliations with a waste of time when the resident could be occupying himself with other areas of the job at the residency site. This is a harmful stance to take and it ignores the resident's needs in favor of what is advantageous to the facility. Not every resident complained of these types of problems, but the amount of negative replies indicates that the issue of affiliations and the broadness or limitations of the scope of clinical practice clearly needs to be further investigated in the future.

Schooling in Orthotics and Prosthetics and how it affects NCOPE Residents

Residents were asked to rate their schooling--did their O&P education adequately prepare them for the residency experience? These responses varied greatly. Of the 62 respondents to the survey, 38 residents (61.29%) answered "yes", that their O&P schooling had prepared them fully for their residency. Twenty-four residents (38.71%) responded "no", that they felt their school training was inadequate or as not having prepared them for the experience.

Schools were rated as follows: Northwestern University, II "yes", 4 "no"; Newington Certificate Programs, IO "yes", 9 "no; University of Texas Southwestern, 5 "yes", 0 "no"; Cal State Dominguez Hills, 5 "yes", 4 "no"; University of Washington, 4 "yes", 4 "no"; Century College, I "yes", 0 "no"; Rancho Los Amigos Medical Center, I "yes", 0 "no." Four respondents declined to name their school.

The majority of respondents who answered "yes" did not comment further. However, a few chose to elaborate. The residents who replied "no" to the question all went into further detail These comments and feedback are listed below.

Residents had several positive things to say about their education:

"The teachers at Northwestern were very informative and helpful. Their willingness, knowledge, and friendship/rapport was irreplaceable.''

"Yes, school set me with the basics. There is a lot more to know and learn from experience ... like one of our professors told us, 'I've taught you the ABC's of prosthetics, now you get to learn and experience D through Z."'

"Yes, school provided great 'book instruction' about biomechanics and componentry as well as plenty of patient contact hours, fittings, castings, modifications, adjustments, etc.,,

"Yes, schools supplies the 'why' of what we do. Then we enter a residency to see 'how."'

"Yes. At Rancho Los Amigos Medical Center, we (students) were working with patients from the very beginning. Everything we learned in the classroom was greatly reinforced while working with actual patients."

Other students had more negative comments:

"All our patients were professional patients who helped us with our fittings"

"Too much time spent calculating forces and torques in Clinical Biomechanies, not enough time torquing adjustment screws during dynamic alignment."

"No, only touched a bit on everything at school…however, real learning is in the residency program."

"No! School, in general, cannot prepare one for the 'real world.' It is hands-on experience in practice that allows one to become a better practitioner. Oftentimes, I felt school ill-prepared me because of some of the things they 'freak' students out about (patient management, etc.)."

"It allowed me to be trained faster than if I had not gone to school, but in practical terms, I left school knowing how to do nothing."

"There were no real patients at school. Things make better sense when you actually do them."

"No, because they did not give us enough hands-on experience to assist in the transition from school to a residency."

As we can see from the above comments, several students felt that there was not enough technical or hands-on experience in their O&P education. However, one of the goals of NCOPE was to restructure the orthotic and prosthetic education system from an overly technical emphasis to one stressing professionalism, patient contact, patient diagnosis, and problem- solving skills along with a strong foundation in biomechanics and science. This is appropriate because in 1992 the field of orthotics and prosthetics was officially recognized by the American Medical Association as an allied health profession (3 ). In a December 1992 article in O&P Almanac, then-NCOPE Chairman William Barringer, CO, said, "The technical work is still important so practitioners can supervise the technicians and understand the strengths and limitation of the materials they work with. But, the technical work belongs in a separate residency program." In this same article, NCOPE Executive Director Robin Seabrook stated, "We're simply moving the hands-on technical experience to the residency program. The students will be learning the same skills, just not as part of the [school] curriculum."

In consideration of the above, it would appear that residents are asking too much from their O&P schooling, or that residency directors are asking too much from their residents right away. The education of today's O&P students under the NCOPE essentials was vigorously defended by Robert S. Lin, CPO, in his article, "A New Generation of Graduates: What to expect from the New Education Standards" (9). Lin restates the above notions that education in orthotics and prosthetics needs to give students understanding of the theoretical and biomechanical principles behind orthotic and prosthetics devices and services, and asserts that the resident "...is not, however, an immediate substitute for a technician or a certified practitioner, and to regard the resident as such will be detrimental to the practice and to the resident's learning curve." As both the O&P education institutions and NCOPE residency sites grow accustomed to the still- relatively-new educational guidelines, it is probable that more and more students will consider their education as "good" or adequate.

Expectations of Residents and of Residency Directors:

Residents indicated that, in general, they are meeting their own expectations and those that their residency director had for them. Eighty-six point eight percent (86.8%) answered "yes" to these questions on the survey of these, only a few decided to elaborate further. Of the 13.2% that answered "no", that they felt they were not meeting their director's or their own expectations, six decided to add additional commentary. Let us examine some of these comments. Positive comments included:

"Yes. I am having successful fittings and am able to troubleshoot on my own."

"Yes, he told me clearly what my goals are and we have an open communication.,,

"Yes. I feel he must believe I'm reaching his expectations because he gave me raises at my third and sixth month as well as a Christmas bonus."

"Yes, we communicate quite well and both agree that we meet each others' expectations."

"I do believe I'm meeting my director's expectations, but I do feel that I am not meeting my own, due to some laziness on my part."

The negative comments were as follows:

"No, I feel I'm not meeting my expectations. As a small shop, mistakes are unaffordable. I'm not given the amount of molds to modify to allow me to become proficient after a year."

"No, but that was due to the lack of organization and unwillingness of the director to follow the NCOPE, guidelines."

"No, I feel trapped in a job, finding it difficult to find answers to questions and/or receive additional educational input in my prosthetic work."

"No, because of a lack of evaluations and because of a lack of structured instruction.,,

"Expectations are not clear. They need to be set out by NCOPE for the Resident on paper. They are working on one now, I think. The director feels he knows it all and nothing could possibly be wrong.,

While a large percentage of students have confidence in their own abilities and feel they are meeting their own and their directors' expectations, the above comments show that there are several residencies where this is not the case. At present, NCOPE is working on a resident handbook that will more clearly define the responsibilities of the resident.

Feedback From the Residency Director and the Resident's Feedback to the Director and Staff

One factor that greatly affects how satisfied the resident is with his or her residency experience is the feedback or commentary that the resident receives from the director. This feedback, whether it be constructive criticism, encouragement, praise, or reprimand, is important in that it gives the resident a clear indication of where he or she stands with the director and as a resident. The resident learns first-hand from the director the areas in which they need improvement, which builds confidence and assures quality learning experiences.

In addition, resident feedback to the director is important. The residency "system" is young, and as such it is often just as much of an adjustment to the directors as to the residents. Directors may have never been in a mentoring-type position before, or they may have years of experience, including classroom teaching or speaking experience. This variation in the situations of the directors makes it vital that the resident be able to offer constructive criticism or other feedback to the residency director. This assures continued improvement on both the part of the resident and the residency. It is also much more likely that the resident will receive the most well- rounded preparation when he or she is comfortable expressing to the director areas in which he or she feels need reinforcement, whether they be technical, clinical, or other types of orthotic and/or prosthetic situations.

Presently NCOPE requires quarterly evaluations, but most residents desire more frequent reviews or discussions of their work. Residents described the frequency of their "feedback sessions" and how often they were able to give feedback or input back to the director of the residency program:

"Progress is not really evaluated per se. If I need to give feedback, though, I speak directly to the director."

"We have weekly informed evaluation. We have open communication so I am able to give feedback as needed."

"Evaluated every three months…none to give feedback to mentor."

"Evaluations are officially done every three months, but I have chances to give feedback on a daily basis."

"Quarterly, but we are already three weeks late for my 2nd evaluation. Every week I tell my director what I need help with but he is on the road 3-5 days per week, so I am solo most of the time."

"Not a lot of feedback. He lets me know what I'm doing wrong if I make a mistake. If the mistake doesn't hurt/harm the patient, he lets me make it and I earn a lot from that.,,

"Quarterly. We really can't give any feedback to our director because it would be confrontational and our director has never admitted to not knowing an answer or being wrong yet. We did underline some passages in the NCOPE residency book but have not used them yet."

"Our progress is evaluated every three months, otherwise we must specifically request a meeting to discuss our progress".

"The practitioners usually provide feedback during patient interaction. So, not many days go by without me realizing my expectations."

"The director feels he knows it all and nothing could possibly be wrong."

According to the residents, there is great variation from residency to residency in the frequency and quality of evaluation. For some, the absolute minimum number of required evaluations was performed while for others daily interaction with the director was the norm. It is quite obvious from the above comments that not enough residencies are doing more than the minimum with respect to evaluation of the resident's progress and asking for feedback from the resident as to how the residency is run. The residents desire more feedback and evaluation (both formal and informal) from their directors, and they also desire an increased amount of input as to the structure of their residency by being able to give informed feedback or constructive criticism to the residency director.

Salary

Almost every resident surveyed indicated displeasure with the salary of the residency position. Sixty-eight point four percent (68.4%) stated that the salary was both unacceptable and lower than they had expected to make. Fourteen point seven percent (14.7%) stated they had expected to receive a salary that was around the figure they were finally offered, but they still felt it was too low. The 16.9% of residents who responded as being content with their salaries had all held positions in O&P prior to going to school and 84.4% of these students were returning to employers for whom they had previously worked. These residents typically reported a substantial increase in salary, along with those already certified in one discipline who were completing residencies in the other discipline. While it was not directly asked on the survey form, some residents chose to list their salaries on the survey form. These ranged from $16,500.00 per year to $50,000.00 per year, but all residents who listed their salary as over $32,500 belonged to the above group with prior experience in the field. No average of salary amounts was computed because the number of students who gave their salaries was much lower than the number of students who did not, and therefore would not be indicative of the entire resident/survey respondent sample.

These salary figures, however, differed from those found by Elaine Uellendahl, CP, in her article "Salary Survey" in the December 1997 O&P Almanac (I 8). Ms. Uellendahl found that graduates of Northwestern's Orthotics program expected salaries ranging from $22,000 per year to $52,000 per year, with an average of $31,597. Prosthetics students ranged from $23,000 to $65,000 per year with an average of $33,772. The low end of the pay scale for residents was $16,500, far below the $22,000 expected by board-eligible 1900-hours (not residency) graduates. NCOPE stated in its pamphlet, "Becoming an Accredited Residency Site" that salaries for residents range from $18,000 to $20,000 per year (14). One practitioner stated in O&P Business News' recent survey of Residency Directors, "We get a board-eligible practitioner for about 20-30% less salary" (7). O&P Almanac's June 1995 article "From Classroom to Real Life: How Residency is Changing Practitioners' Training" shows how a facility that pays $28,000 for a resident (including NCOPE fees and health benefits) develops a resident into an employee that winds up generating between $70,000 and $150,000 in billing per resident per year (4,10).

The issue of salary is further complicated when one takes into consideration the fact that many residents are forced to relocate to find a residency. Moving costs, realtor fee's, the cost and difficulty of finding lodging in another part of the country are all things that drain a resident's budget. When this is added to the expenses of moving an entire family (some residents are married, have children, etc.), and that the spouse may not be able to find work right away in the new location, $18,000 to $23,000 does not go far. If a resident has money trouble at home, the likelihood of him or her being able to fully devote themselves to their orthotic and/or prosthetic training decreases. The salary of new residents is sorrowfully disappointing when one also realizes the costs of going to school for O&P, which is often in the tens of thousands of dollars. Most graduates should be able to defer their student loans while in the residency, but the lower salary of residents when compared to board-eligible practitioners is hurting the new generations of graduates.

Previous Employment in O&P, Employment Agreements for Residency, and Continuance of Employment After Completion of Residency

Of the respondents to the survey, 22% had worked in O&P prior to going to school and had written or verbal agreements to return to that facility after graduation to complete the residency. Thirty-eight percent (38%) responded "no", that they had previous experience, but did not have any sort of prior arrangement with an O&P facility; 40% had no previous experience in the field and thus answered "not applicable."

Fifty-nine point one percent (59.1 %) of respondents stated that they would continue working at their residency facility after their residency was completed; 40.8% responded "no", they would be leaving their residency site to seek employment elsewhere. One respondent did not know and had not yet decided whether or not to accept his facility's job offer after graduation.

This high percentage of residents staying on at their current facilities could become problematic for the future generations of residents. Not all of these facilities will continue to accept a new resident for each year, and those that keep their existing residents on as future practitioners may eventually not have enough room or business to add another practitioner. This is true despite the predictions that there will not be enough practitioners to serve the public's growing demand for orthotic and prosthetic devices and services, which were published in Issues Affecting the Future Demand for Orthotists and Prosthetists, a study prepared for NCOPE in November 1996 by Caroline C. Nielsen, Ph.D. (15).

The Ultimate Test: the ABC Examination

If one were to devise an evaluation process for determining the efficacy of the NCOPE residency versus the previous 1900 hours requirement in training new graduates, the first and most logical place to start would be the end result: are NCOPE residents succeeding on their first try at the ABC certification examination than did their 1900 hours counterparts?

According to NCOPE Executive Director Robin Seabrook, that data is still years away (16). While there have been a few classes now under the new residency mandate, as of the writing of this paper less than 20 of those residents have taken the ABC exam in their disciplines. This low number of challengers for the exams cannot yield a statistically significant result when compared to the years of exam score statistics for 1900 hours exam candidates. Ms. Seabrook suggested that five years from now would perhaps be the minimum amount of time that should elapse before a significant number of NCOPE residents will have taken the ABC examinations. Then, and only then, will any evaluation of test scores between the two groups be statistically sound and give a reasonably accurate picture of the success or failure of NCOPE residents.

Data collected from the surveys confirms this assertion. Only fifteen residents had taken the ABC exam at the time they completed the survey. Of these, ten had passed the exam, one had failed the exam, and four were waiting for their results. Three respondents had completed their residencies but had not yet taken the ABC exam. One respondent stated that he left or failed to complete the residency and therefore would not take the exam. The rest of the survey respondents were currently completing their residencies and as such were ineligible to sit for the certification examinations.

Limitations of the Study and Recommendations for Future Studies

There are many aspects of this study that could serve as limiting factors. There is a lack of previous research and background literature on the subject of residency evaluation. Therefore, without adequate information, a hypothesis could not be formed as the basic "research question" and a deductive experiment could not be constructed to either prove or disprove the hypothesis. In this case, instead, the survey seeks to ask questions, examine the answers and then draw conclusions or hypotheses based on the information in the survey. This research format holds less scientific validity than does the deductive research model.

Another limitation is in the size of our respondent pool. The NCOPE residency mandate is fairly young and as such there are not many people who have completed or are currently completing residencies at the time of this survey. Only about half of the current O&P schools are under the CAAHEP accreditation as well as the NCOPE accreditation, and therefore only half the new graduates are in fact required to complete a residency (as opposed to the 1900-hour work experience requirement). This will change quickly, as by 1999 all the O&P educational institutions currently in existence will fall under the CAAHEP and NCOPE accreditation. In this case, a follow-up study could be taken in 1999 or 2000, when all new graduates would be required to complete a residency. Further research undertaken in five and/or ten years would naturally have a much larger potential experimental or research sample, due to the increased number of graduates and residents. This increase in respondent size would increase the significance of the survey findings.

Survey distribution could always be a limiting factor. Although every effort was made to make the survey available to as many potential respondents as possible, it is almost certain that some residents did not receive the survey. NCOPE and the O&P schools would not release graduate names and addresses for reasons of privacy; surveys were mailed to all accredited sites instead. It was possible, however, to obtain addresses for some residents. It is thus probable that some residents received two copies of the survey. These would lessen the number of potential responses versus the number of surveys distributed, as no student would complete the survey twice.

Limitations could also include the wording and length of the survey itself. The survey (see appendix A for the survey form) called for several "essay" or "open" answers that take more time to complete than a simple multiple choice survey form. The length of survey may have convinced some graduates that they didn't want to be bothered with the survey. The survey was written with multiple choice as well as "essay" questions so that the resident could describe his or her particular situation. This was because there had never been a study of this type conducted with O&P residents before, and the author of the survey did not want to presume to "know all the answers" by offering only specific choices for answers. Once more research is done on this subject, an easier-to-complete survey form could be used in future studies, this study having identified several areas about which to inquire.

Lastly, there is always the possibility that some residents could have feared reprisal by their residency site or director were their survey results attributed to them. The respondents were guaranteed their anonymity, but perhaps some residents felt that their comments would be too specific and could therefore be read as having come from them. This could cause the resident to discard the survey rather than- return it.

Conclusion

As we have seen, there is great variation among NCOPE-aceredited residency programs. The youth of the residency program, the rapidity with which residency has become a requirement, the variation in facilities that host or sponsor residencies (private practice, hospital, university, "Mom and Pop", etc.), and the educational training that students receive prior to graduation are factors that contribute to this variation.

The majority of NCOPE residents are pleased with their residency experience, citing variety of patient types, understanding and helpful support staff and directors, quality instruction, frequent and thorough evaluations of their progress and recommendations for further study or improvement, ability to give feedback to the residency director, ability to make and learn from mistakes, and specific preparation for the ABC exam were all cited repeatedly as strengths of the residency system. While the majority of these residents have not yet taken the ABC exam, they feel that they are being well prepared for the exam. Many students chose residency over the 1900 hours requirement, which is looked upon by graduates today as being perhaps easier, better paying, and more convenient, but offering a less structured format and uncertainty about the quality of the experience. The residency is seen by residents as being a more challenging, broad- based educational experience that will prepare the graduates more effectively for a career as a practitioner in addition to the shorter-term goal of preparing them for the ABC exam.

One area in which there was almost universal outspoken displeasure was with the salary of the residency. The residency position pays less; most graduates know this already, and are still choosing residency despite their displeasure with the salary range of the positions ... this is due to the notion that, as better practitioners, and as having completed a residency, they will be looked upon as more "complete" and thus worth more in the job market in the long run. The short-term salary issues are further aggravated by the fact that relocation is often required of residents.

The number of NCOPE-accredited residency sites is increasing, and must continue to do so at an even more rapid rate if there is to be a residency position for each new graduate when residency becomes mandatory. Residency programs, like practitioners in the field, will undoubtedly have an "attrition rate" as well as a "newly accredited" rate. Residencies in smaller companies are often "one-shot deals" where a facility will become accredited solely for one particular student with whom they had a previous employment agreement; these companies often cannot afford to take on a new resident each year and never intended to. They are, however, barraged by resumes towards the end of each semester as a new generation of graduates looks for residency positions. Many residency sites have long waiting lists. The number of residency positions also needs to increase to meet the projected increases in the number of graduates.

The survey data demonstrates that the residency is vital to the continued development of prosthetics and orthotics as allied health professions and that residents consider the further development of their skills tremendously important. Residents generally feel this way and place great faith in the institution of residency as a whole. Support for these new generations of students and graduates is essential for continued success of the field of orthotics and prosthetics ... therefore, it is imperative that the residency system continue to grow and continue to evaluate and reflect the constantly changing needs of this developing profession.

References

1. American Academy of Orthotists and Prosthetists, The, and American Orthotic and Prosthetic Association, The. "Residency Programs." JPO: Journal of prosthetics and Orthotics, vol.4, no. 5, October 1992. pages. 257-266.

2. American Orthotic and Prosthetic Association, The. "Residency Programs." Orthotics and Prosthetics: The Journal of the American Orthotic and Prosthetic Association, vol.42, no. 1, Spring 1988. pages. 66-75.

3. Conkling, Winifred. "A New Beginning: NCOPE Redesigns Accreditation Process for Primary Education. O&P A1manac, December 1992. pages.33-40.

4. Conn, Deborah. "From Classroom to Real Life: How Residency is Changing Practitioner's Training. O&P A1manac, June 1995. Pages. 40-46.

5. Fairley, Miki. "How to Become ABC Certified?" O&P Business News, April 1, 1 998. pages. 54,58.

6. Fairley, Miki. "NCOPE: Raising the Bar." O&P Business News, April 1, 1998. pages. 1, 40-43.

7. Fairley, Miki. "NCOPE Residency Programs: What Do Educators and Directors Think?" O&P Business News, May 5,1998. pages. 1,30-34.

8. Fairley, Miki. "Residency Program: A Work in Progress. O&P Business News, May 1, 1998. pages 1, 26-37.

9. Lin, Robert S., CPO. "A New Generation of Graduates: What to Expect from the New Education Standards." O&P Almanac, July 1997. pages. 27-29.

10. Lin, Robert S., CPO. "Exploding the Myths of O&P Residencies. O&P Almanac, June 1995. pages. 5,6.

11. Lifford, William A. "NCOPE Residency Survey: A Brief Discussion." O&P Business News, May 15, 1998, pages. 24-28.

12. Lifford, William A. "Residency Survey." O&P Business News, March 15, 1998. pg. 33.

13. National Commission on Orthotic and Prosthetic Education. Accreditation Manual for Sponsoring a Residency 1996/7 Edition. Alexandria, VA.

14. National Commission on Orthotic and Prosthetic Education. "Becoming an Accredited Residency Site." (pamphlet). Alexandria, VA.

15. Nielsen, Caroline C., PhD. Issues affecting the Future Demand for Orthotists and Prosthetists-A Study Prepared for the National Commission on Orthotic and Prosthetic Education. November, 1996. Wellfleet, MA.

16. Personal Telephone Conversations with Ms. Robin Seabrook, executive director of NCOPE, March and April 1998.

17. Uellendahl, Elaine N., CP. "Salary Survey." O&P Almanac, December 1997. pages. 37-38.

18. Unger, Charles K., CAE. "Wanted: Mentors." O&P Almanac, June 1995. pages.23, 30.

Appendix: Residency Survey Form

Topic 1: The number of available residency programs in relation to the number of O&P graduates

1. Were you able to find an NCOPE accredited residency before graduation from your school program?

2. Was there an available residency position within 100 miles of your home?

3. Were there restrictions (i.e. family, spouse's job, etc.) that prevented you from seeking a residency outside of the area in which you wanted to work/live?

4. How many resumes (approximately) did you send out when looking for a residency?

Topic 2: Opinions on the residency itself

1. What are the strengths of your residency program?

2. What are the weaknesses of your residency program?

3. 1. Do you feel your prosthetics/orthotics training in school adequately prepared you for the residency experience? (yes/no, explain:)

   2. Is the scope of practice in your residency program adequate to approximate the broad spectrum of patient types you may encounter in practice? Do you feel it is broad enough / too limited?

   3. If you felt the scope of your residency's practice was too small or too narrow, did your program take steps to ensure you received training in those areas elsewhere? (i.e. sending you to a hospital / university / etc. for different experiences)

4. 1. Do you feel you are meeting your expectations and your director's objectives and expectations of you during the residency? (yes/no, explain:)

   2. How often is your progress evaluated by the residency staff. What opportunities exist for you (the resident) to give feedback on the residency back to your residency director or mentor?

5. 1. Does the salary of your residency position match what you had expected to earn? Higher or lower?

   2. If you were working in O&P prior to attending school, what was your previous salary (or salary range)? What increase in salary did you / didn't you expect after completing school?

   3. If you had previous O&P experience, did you have an agreement to return (after completing school) to your previous employer to do your residency?

6. Do you expect to continue working at your residency facility after your residency has finished?

7. Do you feel your residency has adequately prepared you for the ABC prosthetics or orthotics exam? you think would better prepare you? If so, why? and if not, what do you think would better prepare you?

8. Have you taken the ABC examination? If so, did you succeed?

9. Did you complete the residency? If not, what factors affected your decision to leave?

10. (Fun Question) What is/was the topic for your residency research project?

Topic 3: About you

1. your age:
   your sex:

2. number of years you've been involved with O&P:

3. state you live in/work in:

4. Do you work in (or study) both orthotics and prosthetics?

5. Do you have computer/internet access? If so, may I contact you at your e-mail address?

6. Which O&P school did you attend?

7. Did you have the option of choosing between a formal residency and the previous 1900 work hours requirement? Why did you choose the one you did?

Thank you very much for taking the time to complete this questionnaire. Please be sure to mail this questionnaire to:

Residency Survey
98-A Taylor Road North
Montgomery, AL 36117