RESEARCH FORUM -- Discussing the Results
Maureen J. Simmonds, MCSP, MSc, PHD
ABSTRACT
As the final section of the research paper, the discussion
serves as the forum for the examination and interpretation of
the results, placing the results into theoretical and practical
context.
The purpose of this article is to "discuss the discussion" as
it pertains to content, format and style. The discussion consists of a summary and explanation of the results, an acknowledgment of the study's limitations, and a discussion of
how the results should influence clinical practice.
Discussion implies a sifting and consideration of possibilities, and the writer should take the reader through this
thought process as alternative interpretations are first considered then dismissed, modified or adopted. The results
from the study are compared to others in the literature, and
differences and similarities in the findings are explained.
Limitations of the study must be acknowledged and remaining questions identified. The discussion should finish clearly
and concisely with a simple "take-home" message of what
the study means.
Introduction
The discussion can be one of the most difficult parts of a
paper to write. It is the final section that pulls everything together and shows what the study means. The discussion answers the questions, "So what? What do the results mean?
What do they contribute to the body of knowledge? Are
the findings important?" Essentially, in the discussion, results are examined, interpreted and placed into theoretical
and practical context.
Answering the "so what" demands thorough and reflective contemplation. Alternative explanations of the findings must be considered and then accepted, rejected or
modified. The writer must demonstrate a sound understanding of the research process, his or her own research
findings, and the advantages and limitations of the
methodology used. Moreover, the writer must demonstrate he or she is familiar with the relevant body of knowledge and can put the findings into this context.
The writer must present a compelling case for the importance and meaning of the research findings. The argument must be complete but concise, persuasive but balanced. It must be based on data rather than on speculation.
Finally, the writer must acknowledge the limitations of the
study, identify unanswered questions and suggest future research directions.
While all this thinking, arguing and writing may sound
like onerous tasks, they needn't be. The process can be rewarding because it brings closure and often provides new
insights into the research question. Writing is facilitated by
adequate planning and preparation. This article "discusses
the discussion" as it pertains to content, format and style.
The Prewriting Phase
The discussion usually consists of four sections: a brief
overview or summary of the results, an explanation of the
results, an acknowledgment of the study's limitations, and a
brief discussion about how the results should influence clinical practice and what questions remain (1).
The first step is to decide what to write about within each
of these sections. The investigator must think about the
study and the result, then identify findings that should be
highlighted, those that warrant an explanation or interpretation, and those that warrant little or no discussion. Was
anything said, done or discovered during the data collection period that is pertinent and would facilitate interpretation of the results? The investigator must decide which
main flaws or limitations of the study should be addressed.
Finally, the implications of the study's results should be
identified as they pertain to clinical practice or research
priorities. It is useful to jot these thoughts down using key
words or phrases that can be embellished during the draft-writing phase.
It is clearly not necessary to re-report all of the results in
the discussion. Specific research results can be highlighted
or downplayed for a variety of reasons. One reason to
downplay results is when they lack statistical or clinical significance. Downplaying such insignificant results due to inadequate power or other methodological problems is appropriate since the results lack credibility anyway. Downplaying results because they are unpopular or do not support the investigator's opinion is not appropriate. Negative
results should be accepted as such without undue attempts
to explain them away.
In the same way that not all of the results need to be discussed, it also is not necessary to discuss in depth all of the
shortcomings of a study. Research involves balancing the
advantages and disadvantages of specific research designs.
Generalizability may be sacrificed for control, and the
availability of time and funds frequently influences what is
studied and how a study is carried out.
Whether the shortcomings of a study are major or minor
depends on the specific research question. Flaws in research design may be more obvious after the fact. Honesty
and integrity demand flaws or problems be mentioned, but
it is not necessary to dwell on all limitations. Problems can
be solved when they are acknowledged; thus, the solution
to the problem can be stressed and used in future studies.
The next step in the prewriting phase is to decide on the
logical order of presentation of the key findings. A logical
order may progress from a discussion of the general findings to that of the specific, or from the most important to
the least important findings. The format also will be influenced by the actual results and the factors or main effects
that appear to influence those results.
Essentially, there is no right or wrong order; rather, the
discussion should read easily, and each element should be
presented clearly, succinctly and completely before the next
point is raised. Because the hypothesis test is the raison
d'être of the study, its outcome is of fundamental interest.
Therefore, whatever order of presentation is ultimately accepted, the result of the hypothesis test should be stated
early even if it is not discussed fully at that point.
The focus or emphasis of the manuscript will be influenced by the requirements of the journal to which the paper
will be submitted. Writers should review several issues of
the target journal as well as its instructions to authors to
learn the acceptable content, format and style of that journal. Journals with a clinical focus are likely to be read by
clinicians who are most interested in the clinical implications of the study. If the results of the study have little immediate clinical relevance, then the writer may want to rethink whether the target journal is appropriate. If a study
has little immediate clinical relevance, it is inappropriate for
the writer to speculate what the clinical relevance would or
could be when the clinical question was not tested.
Speculation should be limited in the discussion, and,
when it does occur, it should be clearly identified as such. It
is, however, appropriate for the researcher to pose testable
clinical questions that arise from the research.
The Writing Phase: Specific Content of the DiscussionThe Overview/Summary
The first sentence of the discussion should be a clear statement that captures the reader's attention. If the study used
original methodology or was the first to demonstrate or test
something, then the opening is a suitable place for this
statement. For example, "This is the first study to show...,"
or "The present human study is the first to test
Often, the purpose of the study is restated when opening
the discussion. This restatement reminds the reader of the
original question(s) and is especially useful when the discussion follows a complex results section. Clinical research that
is complex and involves different types and combinations of
interventions (independent variables), and multiple outcomes measures (dependent variables) often uses sophisticated statistical techniques. It is easy for the reader to get
sidetracked and even lost in the statistics and forget the purpose of the study; so the reminder is useful. Finally, the opening statement can be used to simply state whether the hypothesis was accepted and/or to present the key finding(s).
The first paragraph can be used as an overview or summary in which the research purpose and questions are restated, and the main conclusions are relayed. This overview
also can reveal whether the hypotheses were accepted or
rejected and can include a comment on whether the findings were expected or surprising. Finally, the overview informs the reader of the format of the discussion. This is especially useful when the findings are numerous, and the interpretation is complex.
The following is an example of the first paragraph of a
discussion that contains a comprehensive overview:
"The purpose of the present study was to investigate the
active cognitive ingredients of psychological treatment for
long-term chronic headache complaints. The primary two
questions of the investigation were the following:]) Is a cognitive self-hypnosis (CSH) treatment that explicitly attempts
to change appraisal and cognitive coping processes more effective in producing these changes than a relaxation procedure? 2) Are changes in pain appraisal and cognitive coping
related to changes in pain and adjustment in the short and
long term? The main conclusions that can be drawn from the
present study are: The patients who were treated with a CSH
or a relaxation training successfully changed their use of directly targeted coping strategies. Treatment effects were related to changes in the use of coping strategies and appraisal
processes only to a limited extent. The study results are discussed below in more detail" (2).
Explanation of the Results
The discussion is not a simple reiteration of the results. As
noted earlier, discussion topics may be ordered by meaning
(general to specific) (3), level of importance (most to least) or
significance (most to least). In regard to significance, it may
be useful to discuss both statistical and clinical significance.
Logical and consistent organization of the material will help
clarify the information and facilitate its understanding.
Rather than reiterate the results, each new sentence in the
discussion should add to the interpretation and assertion of
what the results actually mean (4). Essentially, the writer
should present the data, then interpret it. Discussion implies
a sifting and consideration of possibilities. The writer should
take the reader through this thought process as alternative
interpretations are first considered then dismissed, modified
or adopted. These thought units must be orderly, progressing from introduction through expansion to conclusion in a
logical, systematic manner. How can this be done? Essentially, the "how" will depend on the results themselves.
When the study is driven by theory and an extension of
previous research, the hypothesis should be plausible and
the results expected. In such a case, the discussion should
be relatively straightforward and will simply involve a reiteration of the main results and an explanation of what this
information adds to the theoretical body of knowledge. The
body of the discussion may then focus on the clinical implications of the results.
In cases where the results were not expected, the writer
must offer alternative explanations. Are the findings true
findings, or was the study seriously flawed? Why were the
findings unexpected? Was the theoretical basis weak or
wrong? Was the methodology appropriate? Did the research protocol itself influence the findings? Sometimes
anecdotal information is useful in explaining such outcomes because the subjects may have acted in an unpredictable way. Perhaps the protocol took longer than expected; perhaps blinding was lost; etc. Unexpected problems should be noted so measures can be taken to avoid
their influence in future studies.
In reality, most studies, especially clinical studies, do not
provide definitive and predictable answers to all the questions posed. Clinical studies use real people as research
subjects. People are complex and often perceive and behave in unpredictable manners. Thus, results of studies often are a mix of positive and negative findings and may appear ambiguous or contradictory if not adequately explained. The discussion allows the investigator the opportunity to tie up such "loose ends."
The following section uses an example from a paper by
Simmonds et al. (5) to show how the interpretation of
"mixed results" can be handled in a discussion.
Study Example
Simmonds et al. (5) measured the effect of an analgesic intervention (transcutaneous electrical nerve stimulation
[TENS]) on two different qualities (sharp and dull) of experimentally induced pain. They measured pain threshold
and pain tolerance with two novel instruments, applied
TENS and then remeasured threshold and tolerance. They
hypothesized that 1) the instruments would evoke different
qualities of pain, 2) TENS would increase pain threshold
and tolerance, and 3) there would be a differential effect of
TENS based on the quality of induced pain. Essentially, the
results showed the quality of pain evoked with each instrument differed. TENS increased pain threshold for dull pain
but not sharp pain. TENS did not increase pain tolerance
for either type of pain.
Several issues needed to be addressed in the discussion.
First was the fundamental issue of instrumentation. Proof
that the instruments evoked different qualities of pain was
crucial because the entire study was based on this premise.
If the instruments did not evoke pain that was measurably
different in quality, then further discussion would be moot
because the study was fundamentally and irrevocably
flawed. Second, it was apparent there was a differential effect of TENS based on pain quality, but this effect also was
influenced by the specific pain measure (threshold versus
tolerance). Third, these results had to be compared to prior
TENS research, which was contradictory. Finally, the clinical implications needed to be cautious because the study
used experimental rather than clinical pain. How were
these issues handled?
The first issue was important but straightforward. Thus, it
was addressed in the opening statement of the discussion,
"The results from the McGill Pain Questionnaire provided
evidence that the two chosen stressors did produce different
sensory qualities of pain..." (5).
The second issue concerned the differential effect of
TENS-a more complex response because both pain quality and the pain measure (threshold or tolerance) influenced the results. This issue was first addressed very simply
"...quality of pain influenced the effectiveness of TENS" (5).
Detailed explanations of the specific findings were then addressed in manageable chunks of information that progressed from the most to the least important.
The significant effect of TENS occurred with dull pain
threshold. This was the most important finding and as such
warranted a primary place in the discussion. Thus, the body
of the discussion began with the sentence, "The finding that
conventional TENS reduced dull pain makes intuitive
sense." The statement reminded the reader of the result and
suggested it was reasonable and expected.
The next two sentences used neurophysiological and behavioral principles to support the assertion: "Dull pains often are alleviated by rubbing or massaging, which most likely generates activity in the large primary afferent nerve
fibers. TENS is thought to activate this same analgesic mechanism..." In this example, the point was clearly made, supported and concluded.
TENS was not effective for sharp pain threshold, and interpretation of this negative finding needed a more de
tailed explanation. Discussion of this issue began with a
statement that accentuated the difference between sharp
and dull pain:
"On the other hand, bright, sharp, superficial pain such as
[that which] occurs after a sunburn is mechanosensitive.
Non-noxious stimuli such as light touch may then be perceived as painful (allodynia). If TENS stimulates large primary afferent nerves such as those that transmit light touch
information, then the same mechanism that causes noxious
mechanosensitivity may lead to the aggravation of sharp
pain by TENS. Normally, TENS should evoke a strong but
comfortable sensation, but it sometimes is perceived as uncomfortable over sharply painful areas."
Note the discussion included a reminder of the proposed
physiological action of TENS and explained why this action
may influence pain differently. When explaining negative
results, it is important for the writer to maintain an objective, neutral tone to avoid detracting from the importance
or credibility of the positive results. Polemics, triviality and
weak theoretical comparisons should be avoided.
Another point to note from the example given above is
the cautious language of the explanation. Specific physiological mechanisms were not tested in this experiment so
explanations of effects can only be suggestions. Words such
as "may," "if" and "sometimes" suggest possibilities rather
than proofs.
Crucial to the content of the discussion is a comparison
of the findings with those from other related studies. The
results were compared with those in the literature, and an
effort was made to explain the agreement between studies, or lack thereof. A distinction was noted between literature based on clinical anecdote and that based on experiment:
"This [lack of effect of TENS on sharp pain] does not
agree with the clinical impressions of Mannheimer and
Lampe. They suggest conventional TENS is most effective for
superficial pain. Perhaps this is a factor of the intensity used."
The latter sentence serves to link the next idea-in which
intensity of TENS is explored as a likely explanation of differential effectiveness-with the results of the example
study and the literature: "This notion appears to be supported by Marchand et al."
Finally, the authors complete the comparison and use
pain type and methodology to support their own findings:
"The results of the present study help explain the lack of effect of TENS reported by other investigators. Woolf and
Nathan and Rudge found no effect of TENS on sharp pain....
They also used small sample sizes, which would have decreased their chances of showing a difference. Jette and Barr
et al. reported no increase in pain threshold to electrically induced pain.... This type of pain is perceived as sharp...
The negative results from the example study pertained
not only to sharp pain but also to pain tolerance measures.
Pain tolerance measures were explained only after the discussion of threshold measures was completed.
The explanation for the negative effect of TENS on pain
tolerance centered on the ceiling effect. In studies using experimentally induced pain, preset limits of stressor intensity must be established to avoid damaging the subjects' tissue. A consequence of this preset limit is the fact that many
subjects do not reach pain tolerance before the preset limit is reached. Thus, it may not be possible to measure true
pre- or post-test pain tolerance. Furthermore, the preset
stressor limit restricts an investigator's ability to demonstrate the effectiveness of any intervention designed to increase pain tolerance measures. If pain tolerance measures
are indicated then it is necessary to resolve this problem.
How did the authors handle these issues?
The authors first raised the issue of gender because it explained the problem and offered a solution, i.e., the ceiling
problem predominated in males and should be less problematic if future studies were limited to the testing of females. "The gender difference in all pain measures was expected and has been experienced by others. As in the present
study, the higher values for men have caused problems in the
evaluation of tolerance in previous investigations." The authors then explained why pain threshold measures are
more useful than tolerance measures. "The validity of the
tolerance measures was definitely compromised by the ceiling effect. However, threshold measures are more reflective
of physiological as opposed to psychological aspects of pain,
and the primary effect of TENS is expected to be physiological. Thus, pain threshold differences are better indicators of
TENS efficacy than pain tolerance changes."
Writers promote their cases when they present their arguments clearly, use a logical progression of ideas and do
not overstate their case. In the previous example, the flaws
were acknowledged and put into physiological and clinical
perspectives.
Acknowledging the Limitations of the Study
The third section of the discussion involves a discussion of
the limitations of the study. This section usually is quite
brief. Essentially, it is a reminder of the specific conditions
of the study-what was and was not tested. For example, if
the effect of a specific spinal orthosis on back pain was tested, the results cannot be generalized to all types of spinal
orthoses. Likewise, it cannot be assumed that spinal orthoses will improve a patient's function if this was not tested, even if the orthosis reduced pain. Scientific writing demands parsimony; writers shouldn't make claims for the efficacy or use of devices or treatments that weren't tested.
In the example study, the authors noted, "The conclusions from this study may be limited to conventional TENS
and the particular pain stressors used." While it is useful to
identify unanswered questions, writers shouldn't speculate
about the results of future studies. The discussion must stick
to the point and confine itself to the findings. Based on the
example study, discussion on the differential effect of
TENS on clinical pain would be speculative. However, it is
reasonable for the author to "wonder" whether different
qualities of clinical pain would respond to TENS in a similar manner and suggest this be tested.
Implications/Final Summary
The final section of the discussion concerns the implications of the results. What does this study add, and what is
next (3)? Should the results from this study influence clinical practice? What do they add to the body of knowledge?
The writer should focus on what really matters and finish
the discussion clearly and concisely with the key "take-home" message. Tell the reader what that message is in
clear, unambiguous terms. "The point is...," "The key finding
from this study is..." Catchy phrases often are memorable
and may work well (1), but they must be meaningful and
have substance as well as style.
A profession is defined as possessing a unique body of
knowledge and assuming responsibility for adding to that
knowledge. Yet many traditional methods of patient management never have been rigorously tested. Specific techniques become part of professional ritual, and untested theories are incorporated into professional dogma. Repeated
often enough, such dogma becomes an unquestionable collection of "facts." Perhaps the greatest responsibility healthcare professionals have to themselves, their profession and
their patients is to question and test the "facts" through research. Consider-it's not ignorance that prevents progress,
it's the illusion of knowledge (Anon).
MAUREEN J. SIMMONDS, MCSP, MSc, PhD, is assistant professor at the School of Physical Therapy at Texas Woman's University, 1130 MD Anderson Blvd., Houston, TX 77030; (713) 7942070.
References:
- Tornquist EM. From proposal to publication. Menlo Park,
Calif.: Addison-Wesley Publishing Co., 1986.
- Ter Kuile MM, Spinhoven P, Linssen ACG, Van Houwelingen HC. Cognitive coping and appraisal processes in the treatment of chronic headaches. Pain 1995; 64,257-64.
- Oyster CK, Hanten WP, Llorens LA. Introduction to research. A guide for the health science professional. Philadelphia:
J.B. Lippincott Co., 1987.
- Publication Manual of the American Psychological Association. 3rd ed. Washington, D.C.: APA, 1988.
- Simmonds MJ, Wessel J, Scudds RA. The effect of pain quality on the efficacy of conventional TENS. Physiotherapy Canada
1992; 44:3: 35-40.
|
