March 2008 • Vol. 4, No. 2
	Advancing Orthotic and Prosthetic Care Through Knowledge

American Academy of Orthotists and Prosthetists
   State-of-the-Science Evidence Reports

Background

On Saturday, March 1, 2008, as part of the American Academy of Orthotists and Prosthetists (the Academy) Annual Meeting and Scientific Symposium, Brian Hafner, PhD, and Stefania Fatone, PhD, will present "Evidence Reports: Findings from the State-of-the-Science Conference (SSC) on Biomechanics of Ambulation after Partial Foot Amputation." This session will discuss how an Evidence Report is created and how the best available evidence is collected, summarized, and interpreted by Evidence Report authors.

The purpose of this systematic review was to establish what is known about gait and prosthetic/orthotic intervention in persons with partial foot amputation (PFA) and to identify what needs to be known to optimize gait and prosthetic/orthotic intervention. A systematic search of the literature identified 437 citations, with 28 publications selected for review based on inclusion criteria. Studies analyzing the gait of PFA were generally published in the last two decades in a myriad of journals spanning many disciplines. Publications that met the inclusion criteria were predominantly classified as case-control or cross-sectional studies and were reviewed based on various aspects of the biomechanics of gait, including temporospatial, ground reaction force (GRF), center of pressure (CoP) excursion, kinematics, kinetics, plantar pressure, electromyography, and energy expenditure. Studies reporting results for each variable were described, methodological issues identified and discussed, and the results summarized.

The quality of the evidence was then rated as high, moderate, low, or insufficient for a number of outcome statements based on the various aspects of the biomechanics of gait. Overall, there was a high level of evidence that PFA generally affects temporospatial, GRF, ankle kinetics, and plantar pressures during gait, but there was less confidence in the evidence regarding more detailed statements about exactly how these aspects of gait are affected by PFA. This is reflective of the small and heterogeneous populations included in the reviewed literature. Because the studies were largely observational, there is insufficient evidence regarding the efficacy of specific prosthetic and orthotic interventions, although generally there was low to moderate evidence that prosthetic and orthotic interventions affect ankle kinematics and moments and may moderate CoP progression. There remains the need to further our understanding of the biomechanics of PFA gait and establish hypotheses regarding prosthetic and orthotic requirements for improved ambulation/function and protection of the residuum (J Prosthet Orthot. 2007; Proceedings: P2-P61).

Fatone conducted this literature review and developed the evidence report with Michael Dillon, PhD, BPO (Hons), and Margaret Hodge, BPO (Hons). The Evidence Report is a critical component for the facilitation of an Academy SSC. It not only serves to provide a comprehensive, systematic review of the scientific literature on a relevant O&P topic, but it also helps conference leaders establish the agenda, speakers, and presentations for the conference. The Evidence Report is intended to drive pertinent discussions among conference attendees and stand as a documented review of the evidence in support of clinical care, methods, and/or interventions in O&P.

In the course of conducting SSCs, there were a number of varied methodologies and styles used to create the Evidence Reports, which prompted the Academy to initiate a separate project to develop a standardized and O&P-appropriate process. Two meetings were conducted with subject-matter experts to develop and document these guidelines. Particular attention was given to the current state of literature within O&P and projections as to where O&P literature will likely be in the near future. As with many healthcare fields, O&P is facing a growing need for evidence-based practice. It is the intent of these guidelines to support this need and provide the O&P profession with a standardized process for collecting and reporting evidence on topics important to this industry.

The Academy's Evidence Report Guidelines, found at www.oandp.org/MasterAgenda, represent a thorough and academic approach to developing these reports. A summary of this process is presented here.

Evidence Report Guidelines

Each SSC is developed around an Evidence Report, which is a key element and consists of a systematic review of the literature and an evaluation of evidence on the conference topic. The development of the Evidence Report includes 12 primary steps (see figure 1).

1. Select a Topic

   An SSC topic must meet the following criteria:

   
   

   • The topic should have public health (habilitation or rehabilitation-related) importance; it should affect or broadly apply to a significant number of people who possess physical dysfunction that is amenable to prosthetic or orthotic management. The severity of the problem (morbidity and mortality) and the feasibility of O&P intervention are key considerations.

   • Controversy or unresolved issues can be clarified, or a gap between current knowledge and current practice may be narrowed.

   • The topic must have an available base of scientific information from which to answer the conference questions and/or resolve controversies. If these cannot be fully addressed with the available evidence, then targeted research priorities may be formulated.

   • The topic should be amenable to clarification based upon evidence, and the outcome should not depend primarily on the subjective judgments of conference participants.

   
   

   Additional elements desirable in selecting a conference topic are the impact on healthcare costs and a high degree of interest in the O&P community. Those using the "Online Topic Suggestion" form located on the Academy's website should also be able to explain why they feel the topic is relevant to O&P, suggest pertinent literature regarding the topic, and formulate at least three key questions regarding the topic. Final selection of a topic is made when agreement is reached by the Academy's Planning Committee and the Academy Board of Directors.

   
   

   In order to develop a topic for an SSC, applicants are encouraged to use the "PICO Method" for developing well-built clinical questions. The PICO method uses a step-by-step approach to develop a clinically relevant question based on four key criteria: (P) patient or population; (I) intervention or exposure; (C) comparison; and (O) outcome. In order to develop a question using PICO, the applicant is asked to answer the following questions with as much specificity as possible:

   
   

   • Patient or population: What are the characteristics of the patient or population in question?

   • Intervention or exposure: What is the desired treatment or evaluation?

   • Comparison: What are the alternatives to the chosen intervention or exposure?

   • Outcome: What are the potential outcomes to treatment?




2. Survey the Literature

   A preliminary search of the literature should be conducted by those individuals responsible for selecting the SSC topic to assess the relative quantity of literature available on the selected conference topic. This search will assess the magnitude of articles that are likely to be included in the Evidence Report and expand or limit the topic based upon that result. The initial survey of the literature is best conducted with the freely available PubMed (i.e., MEDLINE) database. While the PubMed database does not provide comprehensive coverage of all O&P-related activities, it does provide sufficient coverage to approximate the relative number of scientific articles that will be expected in the full search (see section 4). If a more comprehensive assessment of the O&P literature is desired, database subscription fees may be required.

   
   

   The results of the literature survey may be used to either expand or limit the conference topic and define the SSC questions and keywords (see sections 3 and 4).




Table 1: Selected O&P-related journals indexed by healthcare databases.




3. Define the Questions

   Defining the questions is a critical step of developing the Evidence Report search process and selection of search terms (see section 4). Typically, the SSC questions will be a variation on the following themes:

   
   

   1. What is the existing evidence?

   2. What are the relationships between the available evidence and current clinical practice?

   3. What are the research priorities?

   
   

   SSC participants answer the first question through presented papers and presentations. The Evidence Report addresses the second question. Answers to the third and fourth questions stem from SSC discussions. The results of those papers, presentations, and discussions may be included in the Evidence Report, or as part of the conference proceedings summary.




4. Acquire Abstracts

   A systematic literature search is conducted in this stage to locate and obtain literature relevant to the SSC topic and questions. Specific search terms are used alone or in combination and should be explicitly documented in the Evidence Report.

   
   

   It is recommended that the Evidence Report authors begin the literature search by seeking systematic review articles or meta-analyses on the topic of interest and not just the approximate quantity of available literature. These types of publications offer the Evidence Report author a source for relevant articles and/or search terms, and may provide a foundation for the Evidence Report by allowing them to update an existing review, if appropriate, rather than replicating one that has already been performed. The Cochrane Library is an example of a source for such reviews and meta-analyses as are MEDLINE, RECAL, EMBASE, CINAHL, and Web-of-Science.

   
   

   Because no single database covers the entire spectrum of published scientific evidence (see table 1), it is recommended that the Evidence Report search include a minimum of two databases. This comprehensive search strategy will help increase the likelihood that all relevant information is included in the review.

   
   

   Abstracts of articles that meet the search criteria should be obtained directly from the database search (when available). If the full author-written abstract is unavailable from a database, it is recommended to try searching via other means to acquire it. Authors are reminded to document all search terms and results for inclusion in the Evidence Report. Once all the abstracts have been collected, they will be screened using the appropriate inclusion/exclusion criteria (see section 5).




5. Screen with Inclusion/Exclusion Criteria

   Abstracts obtained from the selected databases should next be screened against inclusion and exclusion criteria. These criteria may be defined prior to the systematic search (see section 4) or may be refined during the development of the SSC Evidence Report and used for final screening upon completion of the search. Eligibility for the inclusion of abstracts should be screened by more than one reviewer, and consensus should be achieved through discussion, if necessary. Authors are encouraged to document why articles were excluded from the review and note which inclusion/exclusion criteria were not met in a figure (see section 8) or in an appendix of the Evidence Report. Examples of selection criteria are shown in table 2.




Table 2: Examples of potential inclusion/exclusion criteria.




6. Obtain the Full Papers

   Papers that meet the inclusion criteria, are in question, or are relevant for the introduction/discussion material should be acquired in either digital or hard copy. Questionable papers should be re-screened using the inclusion/exclusion criteria (see section 5) to determine candidacy for the review. The titles of references included in the reviewed articles should be briefly inspected for other relevant articles that may not have been identified through the database search. Once all the papers have been obtained, each study should then be classified by study design (see section 7).




7. Classify the Study Designs

   Evidence Report authors are expected to classify the designs of all studies included in the final review. Two or more reviewers should examine the Methods or Methodology section of all studies and identify each study type using a consistent format. Any discrepancies should be resolved by discussion among the reviewers. The classification scale shown in table 2 is provided for classifying study type in the Evidence Report.

   
   

   It is important to note that the study design classification scale, while adhering to a general hierarchy of evidence established for healthcare research, does not necessarily indicate the appropriateness or quality of the study. The classification of study design is performed in order to assess the types of studies commonly used to investigate or research the SSC topic.




Table 3: Study Design Classification Scale (adapted)




8. Identify Relevant Papers

   Once all of the full papers have been collected and classified, the Evidence Report authors should identify papers for inclusion into the report based on the established inclusion/exclusion criteria, study design classification, and/or relevance to the established topic and questions posed for the SSC. The Evidence Report authors should remember to document the search strategies used to obtain the resultant articles. It is critical that the Evidence Report describe, in detail, the search process and the resultant articles. One method for documenting this process is a search flowchart that presents readers with an overview of the search strategy used by the Evidence Report author(s). The author(s) should, at a minimum, identify all key elements of the search and selection strategy, including database names (and relevant dates), the keywords or phrases used to perform the search, the applied inclusion/exclusion criteria, and key details of the final selection process.




9. Assess the Quality of the Articles

   Research studies in those articles selected for final review should be evaluated by two or more authors for methodological quality. Quality assessment should be performed using an established, standardized format and technique to ensure consistency among reviewers and among SSC Evidence Reports. Experimental and quasi-experimental trials and observational studies should be subjected to a full methodological quality review. Structured reviews and expert opinion articles are not subject to the standardized format, but should be weighed appropriately by the Evidence Report authors.

   
   

   Once all of the studies have been evaluated by individual reviewers, a consensus of the quality criteria and the validity assessments should be determined by the lead Evidence Report author. The results may then be combined in tabular form for the Evidence Report (see section 10).

   
   

   Note: The process of quality evaluation is a subjective task and may vary based upon the experience and background of the Evidence Report authors. Authors are strongly encouraged to review articles independently and then achieve consensus where differences in assessed quality exist.

   
   

10. Construct a Body of Evidence Table

    The consensus results of the study classification and quality assessment should next be combined into a body of evidence table(s). The goal of the body of evidence table(s) is to provide readers with an overview of the quality of the collected literature (see section 9). Because of specific differences in quality criteria between experimental trials and observational studies, an Evidence Report that includes reviewed studies of both types may have more than one body of evidence table. Content and style of the tables is ultimately up to the author(s), but should include all criteria that are critical to assessing the overall quality of the body of evidence.

    
    

    Individual articles in the body of evidence table may be listed by date of publication or categorized by other means appropriate for illustrating the overall quality of the body of evidence. Authors may elect to create additional tables to assist in the synthesis of the available evidence (see section 11).




11. Synthesize the Reviewed Information

    Traditionally, synthesis of the reviewed information in a systematic literature review includes an objective evaluation through a meta-analysis, a statistical analysis of data from different studies or sources. Properly conducted meta-analyses require consistency in the acquired data, including such things as sample population, experimental methods, and outcome measures. Given the wide range of such elements in published O&P research, a meta-analysis is often unsuited for synthesis of the data in an SSC Evidence Report. Instead, authors are encouraged to combine data in a format appropriate to the reviewed literature.

    
    

    Synthesis of the literature is accomplished through the Evidence Report narrative (i.e., main body of the Evidence Report). It is the goal of the narrative to condense the collected, reviewed, and assessed literature into a focused report on the SSC topic. The narrative is often written around (or supported by) additional evidence tables or figures. These items should build upon the body of evidence table(s) and emphasize important aspects of the systematic review. Key elements of the reviewed studies, such as population demographic information, outcome measures, and/or key results of similar studies may be included in the tables.

    
    

    Similarly, tables can be used to list statements that summarize the synthesized information into specific clinical assertions based upon the available evidence. Such tables are useful for documenting the types and quantity of studies that support a particular clinical observation.




12. Summarize the Available Evidence

    To finalize the Evidence Report, author(s) should strive to summarize the overall body of knowledge examined in the literature review. Authors are encouraged to emphasize those results most applicable to the proposed SSC topic (see section 1) and the SSC questions (see section 3). The format, length, and content of this summary are left to the discretion of the authors, but should, at a minimum, address each of the scientific and clinical questions posed to the SSC panelists.

    
    

    Once a draft of the Evidence Report is complete, it will be circulated to the SSC participants in advance of the meeting and will serve to establish the conference agenda, provide background information for the invited presentations, and promote subsequent discussions. Evidence Report authors are required to attend the SSC so as to present their findings from the literature review and to address questions from the participants. Evidence Report authors are encouraged to then incorporate any relevant information discussed at the SSC into the Evidence Report prior to finalizing the document for publication. The finalized Evidence Report will be published as part of the SSC proceedings and will serve as the official literature review of the conference.

Evaluating the Literature

The morning presentation will focus on the Evidence Report developed prior to conducting SSC#8. Those O&P professionals looking to understand how these reports fit into the movement toward evidence-based practice are encouraged to attend the Saturday afternoon session at the Academy Annual Meeting. From noon until 4 p.m., Mark Geil, PhD, and Phil Stevens, CPO, MEd, will join Brian Hafner, PhD, at the Academy's newest certificate course titled, "Evidence-Based Practice: Justifying Patient Care." This course will help attendees become better consumers of O&P research by helping them to gain an understanding of how to search for appropriate articles, define quality O&P studies, and read and evaluate an article.

Website built by oandp.com