View Options - Click to expand
Print Options - Click to expand
E-Mail Options - Click to expand

RESEARCH FORUM -- Discussing the Results

Maureen J. Simmonds, MCSP, MSc, PHD

ABSTRACT

As the final section of the research paper, the discussion serves as the forum for the examination and interpretation of the results, placing the results into theoretical and practical context.

The purpose of this article is to "discuss the discussion" as it pertains to content, format and style. The discussion consists of a summary and explanation of the results, an acknowledgment of the study's limitations, and a discussion of how the results should influence clinical practice.

Discussion implies a sifting and consideration of possibilities, and the writer should take the reader through this thought process as alternative interpretations are first considered then dismissed, modified or adopted. The results from the study are compared to others in the literature, and differences and similarities in the findings are explained. Limitations of the study must be acknowledged and remaining questions identified. The discussion should finish clearly and concisely with a simple "take-home" message of what the study means.

Introduction

The discussion can be one of the most difficult parts of a paper to write. It is the final section that pulls everything together and shows what the study means. The discussion answers the questions, "So what? What do the results mean? What do they contribute to the body of knowledge? Are the findings important?" Essentially, in the discussion, results are examined, interpreted and placed into theoretical and practical context.

Answering the "so what" demands thorough and reflective contemplation. Alternative explanations of the findings must be considered and then accepted, rejected or modified. The writer must demonstrate a sound understanding of the research process, his or her own research findings, and the advantages and limitations of the methodology used. Moreover, the writer must demonstrate he or she is familiar with the relevant body of knowledge and can put the findings into this context.

The writer must present a compelling case for the importance and meaning of the research findings. The argument must be complete but concise, persuasive but balanced. It must be based on data rather than on speculation. Finally, the writer must acknowledge the limitations of the study, identify unanswered questions and suggest future research directions.

While all this thinking, arguing and writing may sound like onerous tasks, they needn't be. The process can be rewarding because it brings closure and often provides new insights into the research question. Writing is facilitated by adequate planning and preparation. This article "discusses the discussion" as it pertains to content, format and style.

The Prewriting Phase

The discussion usually consists of four sections: a brief overview or summary of the results, an explanation of the results, an acknowledgment of the study's limitations, and a brief discussion about how the results should influence clinical practice and what questions remain (1).

The first step is to decide what to write about within each of these sections. The investigator must think about the study and the result, then identify findings that should be highlighted, those that warrant an explanation or interpretation, and those that warrant little or no discussion. Was anything said, done or discovered during the data collection period that is pertinent and would facilitate interpretation of the results? The investigator must decide which main flaws or limitations of the study should be addressed. Finally, the implications of the study's results should be identified as they pertain to clinical practice or research priorities. It is useful to jot these thoughts down using key words or phrases that can be embellished during the draft-writing phase.

It is clearly not necessary to re-report all of the results in the discussion. Specific research results can be highlighted or downplayed for a variety of reasons. One reason to downplay results is when they lack statistical or clinical significance. Downplaying such insignificant results due to inadequate power or other methodological problems is appropriate since the results lack credibility anyway. Downplaying results because they are unpopular or do not support the investigator's opinion is not appropriate. Negative results should be accepted as such without undue attempts to explain them away.

In the same way that not all of the results need to be discussed, it also is not necessary to discuss in depth all of the shortcomings of a study. Research involves balancing the advantages and disadvantages of specific research designs. Generalizability may be sacrificed for control, and the availability of time and funds frequently influences what is studied and how a study is carried out.

Whether the shortcomings of a study are major or minor depends on the specific research question. Flaws in research design may be more obvious after the fact. Honesty and integrity demand flaws or problems be mentioned, but it is not necessary to dwell on all limitations. Problems can be solved when they are acknowledged; thus, the solution to the problem can be stressed and used in future studies.

The next step in the prewriting phase is to decide on the logical order of presentation of the key findings. A logical order may progress from a discussion of the general findings to that of the specific, or from the most important to the least important findings. The format also will be influenced by the actual results and the factors or main effects that appear to influence those results.

Essentially, there is no right or wrong order; rather, the discussion should read easily, and each element should be presented clearly, succinctly and completely before the next point is raised. Because the hypothesis test is the raison d'ĂȘtre of the study, its outcome is of fundamental interest. Therefore, whatever order of presentation is ultimately accepted, the result of the hypothesis test should be stated early even if it is not discussed fully at that point.

The focus or emphasis of the manuscript will be influenced by the requirements of the journal to which the paper will be submitted. Writers should review several issues of the target journal as well as its instructions to authors to learn the acceptable content, format and style of that journal. Journals with a clinical focus are likely to be read by clinicians who are most interested in the clinical implications of the study. If the results of the study have little immediate clinical relevance, then the writer may want to rethink whether the target journal is appropriate. If a study has little immediate clinical relevance, it is inappropriate for the writer to speculate what the clinical relevance would or could be when the clinical question was not tested.

Speculation should be limited in the discussion, and, when it does occur, it should be clearly identified as such. It is, however, appropriate for the researcher to pose testable clinical questions that arise from the research.

The Writing Phase: Specific Content of the Discussion

The Overview/Summary

The first sentence of the discussion should be a clear statement that captures the reader's attention. If the study used original methodology or was the first to demonstrate or test something, then the opening is a suitable place for this statement. For example, "This is the first study to show...," or "The present human study is the first to test

Often, the purpose of the study is restated when opening the discussion. This restatement reminds the reader of the original question(s) and is especially useful when the discussion follows a complex results section. Clinical research that is complex and involves different types and combinations of interventions (independent variables), and multiple outcomes measures (dependent variables) often uses sophisticated statistical techniques. It is easy for the reader to get sidetracked and even lost in the statistics and forget the purpose of the study; so the reminder is useful. Finally, the opening statement can be used to simply state whether the hypothesis was accepted and/or to present the key finding(s).

The first paragraph can be used as an overview or summary in which the research purpose and questions are restated, and the main conclusions are relayed. This overview also can reveal whether the hypotheses were accepted or rejected and can include a comment on whether the findings were expected or surprising. Finally, the overview informs the reader of the format of the discussion. This is especially useful when the findings are numerous, and the interpretation is complex.

The following is an example of the first paragraph of a discussion that contains a comprehensive overview:

"The purpose of the present study was to investigate the active cognitive ingredients of psychological treatment for long-term chronic headache complaints. The primary two questions of the investigation were the following:]) Is a cognitive self-hypnosis (CSH) treatment that explicitly attempts to change appraisal and cognitive coping processes more effective in producing these changes than a relaxation procedure? 2) Are changes in pain appraisal and cognitive coping related to changes in pain and adjustment in the short and long term? The main conclusions that can be drawn from the present study are: The patients who were treated with a CSH or a relaxation training successfully changed their use of directly targeted coping strategies. Treatment effects were related to changes in the use of coping strategies and appraisal processes only to a limited extent. The study results are discussed below in more detail" (2).

Explanation of the Results

The discussion is not a simple reiteration of the results. As noted earlier, discussion topics may be ordered by meaning (general to specific) (3), level of importance (most to least) or significance (most to least). In regard to significance, it may be useful to discuss both statistical and clinical significance. Logical and consistent organization of the material will help clarify the information and facilitate its understanding.

Rather than reiterate the results, each new sentence in the discussion should add to the interpretation and assertion of what the results actually mean (4). Essentially, the writer should present the data, then interpret it. Discussion implies a sifting and consideration of possibilities. The writer should take the reader through this thought process as alternative interpretations are first considered then dismissed, modified or adopted. These thought units must be orderly, progressing from introduction through expansion to conclusion in a logical, systematic manner. How can this be done? Essentially, the "how" will depend on the results themselves.

When the study is driven by theory and an extension of previous research, the hypothesis should be plausible and the results expected. In such a case, the discussion should be relatively straightforward and will simply involve a reiteration of the main results and an explanation of what this information adds to the theoretical body of knowledge. The body of the discussion may then focus on the clinical implications of the results.

In cases where the results were not expected, the writer must offer alternative explanations. Are the findings true findings, or was the study seriously flawed? Why were the findings unexpected? Was the theoretical basis weak or wrong? Was the methodology appropriate? Did the research protocol itself influence the findings? Sometimes anecdotal information is useful in explaining such outcomes because the subjects may have acted in an unpredictable way. Perhaps the protocol took longer than expected; perhaps blinding was lost; etc. Unexpected problems should be noted so measures can be taken to avoid their influence in future studies.

In reality, most studies, especially clinical studies, do not provide definitive and predictable answers to all the questions posed. Clinical studies use real people as research subjects. People are complex and often perceive and behave in unpredictable manners. Thus, results of studies often are a mix of positive and negative findings and may appear ambiguous or contradictory if not adequately explained. The discussion allows the investigator the opportunity to tie up such "loose ends."

The following section uses an example from a paper by Simmonds et al. (5) to show how the interpretation of "mixed results" can be handled in a discussion.

Study Example

Simmonds et al. (5) measured the effect of an analgesic intervention (transcutaneous electrical nerve stimulation [TENS]) on two different qualities (sharp and dull) of experimentally induced pain. They measured pain threshold and pain tolerance with two novel instruments, applied TENS and then remeasured threshold and tolerance. They hypothesized that 1) the instruments would evoke different qualities of pain, 2) TENS would increase pain threshold and tolerance, and 3) there would be a differential effect of TENS based on the quality of induced pain. Essentially, the results showed the quality of pain evoked with each instrument differed. TENS increased pain threshold for dull pain but not sharp pain. TENS did not increase pain tolerance for either type of pain.

Several issues needed to be addressed in the discussion. First was the fundamental issue of instrumentation. Proof that the instruments evoked different qualities of pain was crucial because the entire study was based on this premise. If the instruments did not evoke pain that was measurably different in quality, then further discussion would be moot because the study was fundamentally and irrevocably flawed. Second, it was apparent there was a differential effect of TENS based on pain quality, but this effect also was influenced by the specific pain measure (threshold versus tolerance). Third, these results had to be compared to prior TENS research, which was contradictory. Finally, the clinical implications needed to be cautious because the study used experimental rather than clinical pain. How were these issues handled?

The first issue was important but straightforward. Thus, it was addressed in the opening statement of the discussion, "The results from the McGill Pain Questionnaire provided evidence that the two chosen stressors did produce different sensory qualities of pain..." (5).

The second issue concerned the differential effect of TENS-a more complex response because both pain quality and the pain measure (threshold or tolerance) influenced the results. This issue was first addressed very simply "...quality of pain influenced the effectiveness of TENS" (5). Detailed explanations of the specific findings were then addressed in manageable chunks of information that progressed from the most to the least important.

The significant effect of TENS occurred with dull pain threshold. This was the most important finding and as such warranted a primary place in the discussion. Thus, the body of the discussion began with the sentence, "The finding that conventional TENS reduced dull pain makes intuitive sense." The statement reminded the reader of the result and suggested it was reasonable and expected.

The next two sentences used neurophysiological and behavioral principles to support the assertion: "Dull pains often are alleviated by rubbing or massaging, which most likely generates activity in the large primary afferent nerve fibers. TENS is thought to activate this same analgesic mechanism..." In this example, the point was clearly made, supported and concluded.

TENS was not effective for sharp pain threshold, and interpretation of this negative finding needed a more de tailed explanation. Discussion of this issue began with a statement that accentuated the difference between sharp and dull pain:

"On the other hand, bright, sharp, superficial pain such as [that which] occurs after a sunburn is mechanosensitive. Non-noxious stimuli such as light touch may then be perceived as painful (allodynia). If TENS stimulates large primary afferent nerves such as those that transmit light touch information, then the same mechanism that causes noxious mechanosensitivity may lead to the aggravation of sharp pain by TENS. Normally, TENS should evoke a strong but comfortable sensation, but it sometimes is perceived as uncomfortable over sharply painful areas."

Note the discussion included a reminder of the proposed physiological action of TENS and explained why this action may influence pain differently. When explaining negative results, it is important for the writer to maintain an objective, neutral tone to avoid detracting from the importance or credibility of the positive results. Polemics, triviality and weak theoretical comparisons should be avoided.

Another point to note from the example given above is the cautious language of the explanation. Specific physiological mechanisms were not tested in this experiment so explanations of effects can only be suggestions. Words such as "may," "if" and "sometimes" suggest possibilities rather than proofs.

Crucial to the content of the discussion is a comparison of the findings with those from other related studies. The results were compared with those in the literature, and an effort was made to explain the agreement between studies, or lack thereof. A distinction was noted between literature based on clinical anecdote and that based on experiment:

"This [lack of effect of TENS on sharp pain] does not agree with the clinical impressions of Mannheimer and Lampe. They suggest conventional TENS is most effective for superficial pain. Perhaps this is a factor of the intensity used."

The latter sentence serves to link the next idea-in which intensity of TENS is explored as a likely explanation of differential effectiveness-with the results of the example study and the literature: "This notion appears to be supported by Marchand et al."

Finally, the authors complete the comparison and use pain type and methodology to support their own findings: "The results of the present study help explain the lack of effect of TENS reported by other investigators. Woolf and Nathan and Rudge found no effect of TENS on sharp pain.... They also used small sample sizes, which would have decreased their chances of showing a difference. Jette and Barr et al. reported no increase in pain threshold to electrically induced pain.... This type of pain is perceived as sharp...

The negative results from the example study pertained not only to sharp pain but also to pain tolerance measures. Pain tolerance measures were explained only after the discussion of threshold measures was completed.

The explanation for the negative effect of TENS on pain tolerance centered on the ceiling effect. In studies using experimentally induced pain, preset limits of stressor intensity must be established to avoid damaging the subjects' tissue. A consequence of this preset limit is the fact that many subjects do not reach pain tolerance before the preset limit is reached. Thus, it may not be possible to measure true pre- or post-test pain tolerance. Furthermore, the preset stressor limit restricts an investigator's ability to demonstrate the effectiveness of any intervention designed to increase pain tolerance measures. If pain tolerance measures are indicated then it is necessary to resolve this problem. How did the authors handle these issues?

The authors first raised the issue of gender because it explained the problem and offered a solution, i.e., the ceiling problem predominated in males and should be less problematic if future studies were limited to the testing of females. "The gender difference in all pain measures was expected and has been experienced by others. As in the present study, the higher values for men have caused problems in the evaluation of tolerance in previous investigations." The authors then explained why pain threshold measures are more useful than tolerance measures. "The validity of the tolerance measures was definitely compromised by the ceiling effect. However, threshold measures are more reflective of physiological as opposed to psychological aspects of pain, and the primary effect of TENS is expected to be physiological. Thus, pain threshold differences are better indicators of TENS efficacy than pain tolerance changes."

Writers promote their cases when they present their arguments clearly, use a logical progression of ideas and do not overstate their case. In the previous example, the flaws were acknowledged and put into physiological and clinical perspectives.

Acknowledging the Limitations of the Study

The third section of the discussion involves a discussion of the limitations of the study. This section usually is quite brief. Essentially, it is a reminder of the specific conditions of the study-what was and was not tested. For example, if the effect of a specific spinal orthosis on back pain was tested, the results cannot be generalized to all types of spinal orthoses. Likewise, it cannot be assumed that spinal orthoses will improve a patient's function if this was not tested, even if the orthosis reduced pain. Scientific writing demands parsimony; writers shouldn't make claims for the efficacy or use of devices or treatments that weren't tested.

In the example study, the authors noted, "The conclusions from this study may be limited to conventional TENS and the particular pain stressors used." While it is useful to identify unanswered questions, writers shouldn't speculate about the results of future studies. The discussion must stick to the point and confine itself to the findings. Based on the example study, discussion on the differential effect of TENS on clinical pain would be speculative. However, it is reasonable for the author to "wonder" whether different qualities of clinical pain would respond to TENS in a similar manner and suggest this be tested.

Implications/Final Summary

The final section of the discussion concerns the implications of the results. What does this study add, and what is next (3)? Should the results from this study influence clinical practice? What do they add to the body of knowledge?

The writer should focus on what really matters and finish the discussion clearly and concisely with the key "take-home" message. Tell the reader what that message is in clear, unambiguous terms. "The point is...," "The key finding from this study is..." Catchy phrases often are memorable and may work well (1), but they must be meaningful and have substance as well as style.

A profession is defined as possessing a unique body of knowledge and assuming responsibility for adding to that knowledge. Yet many traditional methods of patient management never have been rigorously tested. Specific techniques become part of professional ritual, and untested theories are incorporated into professional dogma. Repeated often enough, such dogma becomes an unquestionable collection of "facts." Perhaps the greatest responsibility healthcare professionals have to themselves, their profession and their patients is to question and test the "facts" through research. Consider-it's not ignorance that prevents progress, it's the illusion of knowledge (Anon).


MAUREEN J. SIMMONDS, MCSP, MSc, PhD, is assistant professor at the School of Physical Therapy at Texas Woman's University, 1130 MD Anderson Blvd., Houston, TX 77030; (713) 7942070.

References:

  1. Tornquist EM. From proposal to publication. Menlo Park, Calif.: Addison-Wesley Publishing Co., 1986.
  2. Ter Kuile MM, Spinhoven P, Linssen ACG, Van Houwelingen HC. Cognitive coping and appraisal processes in the treatment of chronic headaches. Pain 1995; 64,257-64.
  3. Oyster CK, Hanten WP, Llorens LA. Introduction to research. A guide for the health science professional. Philadelphia: J.B. Lippincott Co., 1987.
  4. Publication Manual of the American Psychological Association. 3rd ed. Washington, D.C.: APA, 1988.
  5. Simmonds MJ, Wessel J, Scudds RA. The effect of pain quality on the efficacy of conventional TENS. Physiotherapy Canada 1992; 44:3: 35-40.