SCIENCE AND PSEUDOSCIENCE IN THE DEVELOPMENT OF EYE MOVEMENT DESENSITIZATION AND REPROCESSING: IMPLICATIONS FOR CLINICAL PSYCHOLOGY
James D. Herbert
MCP Hahnemann University
Scott O. Lilienfeld
Jeffrey M. Lohr
University of Arkansas
Robert W. Montgomery
William T. O’Donohue
University of Nevada-Reno
Gerald M. Rosen
David F. Tolin
MCP Hahnemann University
Correspondence should be addressed to James D. Herbert, PhD, Department of Psychology, MCP Hahnemann University, Mail Stop 988, 245 N. 15th Street, Philadelphia, PA 19102-1192.
ABSTRACT. The enormous popularity recently achieved by Eye Movement Desensitization and Reprocessing (EMDR) as a treatment for anxiety disorders appears to have greatly outstripped the evidence for its efficacy from controlled research studies. The disparity raises disturbing questions concerning EMDR’s aggressive commercial promotion and its rapid acceptance among practitioners. In this article, we: (1) summarize the evidence concerning EMDR’s efficacy; (2) describe the dissemination and promotion of EMDR.; (3) delineate the features of pseudoscience and explicate their relevance to EMDR; (4) describe the pseudoscientific marketing practices used to promote EMDR.; (5) analyze factors contributing to the acceptance of EMDR by professional psychologists; and (6) discuss practical considerations for professional psychologists regarding the adoption of EMDR into professional practice. We argue that EMDR provides an excellent vehicle for illustrating the differences between scientific and pseudoscientific therapeutic techniques. Such distinctions are of critical importance for clinical psychologists who intend to base their practice on the best available research. © 2000 Elsevier Science Ltd.
KEY WORDS. EMDR, Pseudoscience, Trauma, Posttraumatic stress disorder.
THE PROFESSIONAL PROMOTION of psychotherapy has been based largely on the often cited Dodo-Bird verdict that all treatments are effective and equally so (Luborsky, Singer, & Luborsky, 1975). Based on this belief, the majority of psychological practitioners adhere to the dictum that “Everyone has won, and all must have prizes” (Luborsky et al., 1975); Rosenzweig, 1936; Wampold et al., 1997). As a consequence, eclecticism has gained a new found respectability (Lazarus, Beutler, & Norcross, 1992), and new treatments proliferate at a rapid rate (Figley, 1997).
Empirically oriented clinical psychologists, however, have often been skeptical of overarching claims for psychotherapy (Beutler, 1991), and have been at the forefront of research investigating the effects of specific treatments for specific disorders. For example, Eysenck’s (1994) reanalysis of earlier meta-analytic research demonstrated the potency of placebo and other nonspecific effects in most treatments, but also the power of behavioral techniques for a narrower range of disorders. Other observers are skeptical of overarching claims of psychotherapy for pragmatic, rather than empirical, reasons. These individuals (and corporate entities) have responsibility for, and a financial stake in, identifying cost-effective treatments for psychological conditions (Stro- sahl, 1994, 1995).
The necessity of methodological rigor in the empirical validation of intervention procedures has recently become a visible and contentious issue in professional psychology (Fox, 1996). The American Psychological Association’s Division of Clinical Psychology recently published reports of a task force suggesting basic methodological criteria for the empirical validation of psychological treatments, and specified treatments that meet these criteria (Chambless, 1995; Chambless et al., 1996).
The concern for empirical validation has helped to limit the clinical application of new techniques for which validation research has not yet been conducted. Most recently, experimental procedures (Delmolino & Romanczyck, 1995; Jacobson, Mulick, & Schwartz, 1995) have been used to demonstrate the lack of efficacy of facilitated communication, a technique purported to permit nonverbal autistic individuals to communicate with others that was widely promoted to replace more expensive, but effective, behavioral procedures. Although the scientific evaluation of psychological treatments has yielded substantial benefits, it is not without risk. The evaluation of treatments must rest upon the substantive aspects of the scientific enterprise, rather than on its superficial appearance. If the appearance is emphasized over the substance, the process of inquiry risks becoming pseudoscientific. The costs of adopting pseudoscientific treatments would be substantial. We argue that the professional evaluation and promotion of at least one recent and prominent innovation in psychosocial treatment has often been characterized by pseudoscientific practices.
PROFESSIONAL CONTEXT FOR THE DEVELOPMENT OF EMDR Clinical Manifestations of Fear and Trauma
The last 10 years have witnessed a rapid expansion in interest surrounding the nature of trauma and anxiety, the psychological repercussions of trauma, and the psychological treatment of those repercussions. Treatments have been applied not only to trauma-related distress, but also to more longstanding difficulties such as specific phobias and other anxiety disorders. Such behavioral interventions as graduated in vivo exposure, exposure and response prevention, and social skills training are treatments of choice across a wide range of anxiety disorders (Barlow, 1993; Chambless, 1995; Chambless et al., 1996). There are several novel treatments vying for the attention of clinicians treating anxiety and trauma that sometimes have been referred to as the Power Therapies. This moniker derives from the claim that such treatments work much more efficiently than extant interventions for anxiety disorders (Figley, 1997). The Power Therapies include Thought Field Therapy (TFT; Callahan, 1995; Gallo, 1995), Emotional Freedom Therapy (EFT; Craig, 1997), Traumatic Incident Reduction (TIR; Gerbode, 1985, 1995), and Visual-Kinesthetic Dissociation (VKD; Bandler & Grinder, 1979).
The most visible of these treatments, however, is EMDR. Despite being less than 10- years-old, the commercialization of EMDR has been remarkably successful. According to Shapiro (1998b), the developer of EMDR, over 25,000 mental health clinicians have been trained in this procedure. The dissemination of this technique is rivaled only by the number of conditions to which it has been applied in clinical contexts. EMDR Institute, Inc. distributes promotional literature that alleges effective application of this treatment for the distress associated with myriad conditions, including Posttraumatic Stress Disorder (PTSD), Attention-Deficit/Hyperactivity Disorder, dissociative disorders, self-esteem issues, and personality pathology (EMDR Institute, Inc., 1995, 1997; Fensterheim, 1996). In its most wide-ranging application, EMDR has been used as a means of spiritual development (Parnell, 1996).
A complete understanding of the reasons underlying EMDR’s substantial visibility would require a complex sociological analysis beyond the scope of the present article. Nevertheless, we tentatively propose two main causal factors for EMDR’s visibility that merit special attention. The first potential factor is that EMDR and cognitive-behavioral treatments share some similarities. Both are structured, prescriptive and time limited. Indeed, Foa and Meadows (1997) characterized EMDR as a cognitive-behavioral treatment in their review of psychosocial treatments for PTSD. These common features also lend themselves to empirical testing, and cognitive-behavioral treatments possess established empirical records as validated treatments for anxiety and mood disorders (Chambless, 1995; Chambless et al., 1996). Indeed, EMDR is now listed as a probably efficacious treatment for civilian PTSD (Chambless et al., 1998) by the American Psychological Association (APA) Division 12 Task Force on empirically supported treatments because two outcome studies (Rothbaum, 1997; S. A. Wilson, Becker, & Tinker, 1995) suggest that EMDR is superior to waitlist control procedures.
The decision of the APA Task Force has, however, generated considerable controversy. Some argue that this decision is justified given that the criterion of probably efficacious status requires only that a treatment be demonstrated to be more efficacious than no treatment in two studies (Chambless et al., 1998). Others, however, contend that: (a) this criterion is overly liberal because the null hypothesis—namely that a treatment is not more effective than no treatment—is almost certainly false for the vast majority of psychological treatments (Herbert, 1998); and (b) because EMDR may be a variant of standard exposure treatments (see section entitled “Reviews of the Efficacy of EMDR”), there is no compelling evidence to regard it as conceptually distinguishable from other commonly used exposure-based methods (e.g., imaginal flooding; see McGlynn & Lohr, 1998; for additional criticisms O’Donohue, 1998). The merits of these criticisms notwithstanding, it seems likely that the decision of the APA Task Force will further enhance the visibility and public credibility of EMDR.
Another possible reason for the visibility of EMDR is the burgeoning specialty of traumatology. In recent years, there has been an expansion of the signs and symptoms representing post-stress clinical conditions, so much so that the diagnostic criteria for PTSD have become more general and a new stress-related diagnosis (Acute Stress Disorder) has been added to the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994). In addition, the context for trauma effects has been expanded beyond natural and man-made disasters (e.g., combat and torture) to include negative life events occurring in the context of child rearing, family life, and marital conflict. As a result, an increasing demand has arisen for the provision of treatment, including EMDR, for individuals suffering from the adverse consequences of stressful life events.
EMDR is based on a theory that relies heavily on physiological concepts closely related to neurological processes. The natura of pathology and effective treatment is predicated on a model called Accelerated Information Processing that is hypothesized to be akin to a psychological immune system (Shapiro, 1995a). Healing is posited to occur after eye movements and other features of the clinical protocol unlock the pathological condition. In Accelerated Information Processing, “The key to psychological change is the ability to facilitate the appropriate information processing. This means making connections between healthier associations” (Shapiro, 1995a, p. 48). Accelerated Information Processing is offered as a “unifying theory that can be seen as underlying all psychological modalities,” but the model is distinct in defining pathology as “dysfunctionally stored information that can be properly assimilated through a dynamically activated processing system” (Shapiro, 1995a, p. 52). Although the model has intuitive appeal, Keane (1998) commented on its limitations:
Unlike exposure therapy which has a long tradition of ameliorating a range of anxiety meditated clinical problems and which is embedded in the rich conceptual tradition of
experimental psychology, EMDR falters seriously at the theoretical level…….. The primary
weakness of EMDR stems from a distinct lack of integration with existing models of psychopathology and psychotherapy. (p. 404)
Indeed, explanations for EMDR’s reported clinical effects have been addressed by alternative conceptual analyses based on well-established learning processes that subsume exposure and cognitive-behavioral treatments (Dyck, 1993; MacCulloch & Feldman; 1996).
From its inception, EMDR has often been characterized by extremely strong claims, including the purported rapidity, permanence, and generality of its effects (Shapiro, 1995a; Shapiro & Forrest, 1997), and the assertion that these effects are considerably greater than those of extant treatments (Fensterheim, 1996; Shapiro, 1996a, 1996b). For example, the original published account of EMDR (Shapiro, 1989), touted this intervention as a single session treatment for the distress associated with the traumatic memories in PTSD. Such claims are often made on the basis of clinician testimony (workshop training and word-of-mouth) and published case studies. Nevertheless, as the philosopher Hume (1748/1977) noted, extraordinary claims require extraordinary evidence. In the following section, we briefly review the nature of this evidence in the case of EMDR.
SCIENTIFIC AND PROFESSIONAL EVALUATION OF EMDR Reviews of the Efficacy of EMDR
A literature review of EMDR by Lohr, Kleinknecht, Tolin, and Barrett (1995) concluded that : (a) the protocol frequently reduces verbal report and independent observer ratings of distress; (b) psychophysiologic indices show little effect of treatment; (c) there is little evidence that indicates treatment influences behavioral measures; and (d) eye movements do not appear to be an essential component of the treatment. Similar conclusions have been reached independently by others (Acierno, Hersen, Van Hasselt, Tremont, & Mueser, 1994; DeBell & Jones, 1997). Since the review by Lohr et al. (1995), the empirical literature has expanded rapidly and experimental rigor has improved (e.g., Devilly & Spence, 1999; Devilly, Spence, & Rapee, 1998; Muris, Merckelbach, Holdrinet, & Sijsenaar, 1998; Pitman et al., 1996). A summary description of the most recent research on EMDR follows. More extensive analyses of treatment efficacy are found in Lohr, Tolin, and Lilienfeld (1998) and Lohr, Lilien- feld, Tolin, and Herbert (1999).
Wait-list and Attention Controls
Some studies that have compared EMDR with no treatment or with wait-list controls show greater effects of EMDR on self-report measures for specific phobia (Bates, McGlynn, Montgomery, & Mattke, 1996), PTSD (Boudewyns & Hyer, 1996; Roth- baum, 1997), traumatic memories (Shapiro, 1989; Wilson et al., 1995; S. A. Wilson, Becker, & Tinker, 1997), panic disorder (Feske & Goldstein, 1997), and public speaking anxiety (Foley & Spates, 1995). These results, however, are not convincing in that they can be attributed to any number of incidental, nonspecific effects (Lohr et al., 1998, 1999), including expectation for improvement, and therapist attention (see Mahoney, 1978).
Attention controls attempt to equate the amount and general nature of therapeutic contact across experimental conditions in an outcome experiment or in the treatment setting (Mahoney, 1978). Five studies have used procedures that approximate attentional controls in the treatment of PTSD and traumatic memories (Boudewyns,
Stwertka, Hyer, Albrecht, & Sperr, 1993; Carlson, Chemtob, Rusnak, Hedlund, & Mu- raoka, 1998; Jensen, 1994; Silver, Brooks, & Obenchain, 1995; Vaughan et al., 1994). Jensen (1994) randomly assigned participants to either customary care control or an EMDR group. The data analysis revealed no differences between the groups on the standardized measures of PTSD symptoms following treatment.
Boudewyns et al. (1993) randomly assigned participants to either EMDR, Exposure Control (EC), or mileau-only control. The EC group was procedurally similar to the EMDR group except for eye movements. Standardized measures showed no differential effects of treatment, and no form of treatment appeared to affect the psychophysiologic measures. Although therapist ratings of treatment responders versus nonresponders favored the EMDR group, assessors of treatment outcome were not blind to treatment conditions.
Silver et al. (1995) provided milieu treatment concurrently with either: (a) EMDR, (b) biofeedback, or (c) group relaxation training. A third control group received only milieu treatment. The authors reported that the subjects in the EMDR plus milieu treatment “did better than the control group across all variables and generally at statistically significant levels” (Silver et al., 1995, p. 340). They also reported that EMDR resulted in greater change than the biofeedback and relaxation groups. These conclusions, however, are not justified due to a number of methodological limitations such as nonrandom assignment and inappropriate statistical analyses (see, Lohr, Kleinknecht, et al., 1995), and it is impossible to draw any valid conclusions regarding the efficacy of EMDR per se because its application was confounded by concurrent milieu treatment.
Vaughan et al. (1994) assigned trauma victims to either: (a) a no-treatment control condition, (b) EMDR, (c) Imagery Habituation Training (IHT), or (d) Applied Muscle Relaxation Training (AMT). The results showed that all groups improved significantly compared with the wait list but there were no differences among treatment conditions. Post-hoc multiple t-test comparisons suggested that subjects in the EMDR condition experienced fewer flashbacks, nightmares, and avoidance symptoms after treatment relative to all treatment groups. Caution much be exercised in the interpretation of any genuine effect of EMDR for several reasons. First, because neither IHT nor AMT has been identified as a valid treatment for PTSD (Chambless et al., 1998; Foa & Meadows, 1997; Keane, 1998), EMDR was not compared with demonstratibly effective treatments. Second, the multiple t-test comparisons did not protect for Type 1 error. Third, statistically significant symptom improvement occurred in all treatment conditions, suggesting the operation of nonspecific effects in both EMDR and the control treatments (Lohr, Kleinknecht, et al. 1995).
Carlson et al. (1998) randomly assigned participants to either: (a) routine clinical care, (b) 12 sessions of biofeedback-assisted relaxation, or (c) 12 sessions of EMDR. The results showed that at posttreatment the EMDR showed greater effects than the two other conditions on self-report, psychometric, and standardized measures, which were maintained at 3-month follow-up. However, assessors were not blind to experimental conditions except for an interview measure at follow-up. There was no differential effect on psychophysiologic measures. As biofeedback-assisted relaxation is not a validated treatment for PTSD (Chambless et al., 1998; Foa & Meadows, 1997; Keane, 1998), these results show only that EMDR is more effective than no additional treatment and a treatment of no demonstrated efficacy for PTSD, and that this efficacy was apparent only on verbal-report measures. In summary, these five studies provide little evidence that EMDR provides benefits beyond attention control conditions.
Nonspecific Effect Controls
The nonspecific factors in an experimental treatment procedure include treatment credibility, expectation for improvement, experimental demand, therapist-experimenter enthusiasm, therapist-experimenter allegiance, effort justification (Cooper, 1980), and the incidental effects of any particular treatment (see Lohr et al., 1999). In an attempt to control for nonspecific factors, Hazlett-Stevens, Lytle, and Borkovec
(1996) randomly assigned participants with traumatic memories to one of three treatment conditions: (a) EMDR, (b) an identical procedure that employed eye fixation, or (c) nondirective counseling. The results showed that the non-directive counseling condition produced the same effects as EMDR on three out of four measures, suggesting that EMDR may be no more efficacious than nonspecific treatment.
Scheck, Schaeffer, and Gillette (1998) randomly assigned women with traumatic memories to two sessions of either EMDR or an Active Listening (AL; Gordon, 1974) control. EMDR and AL were administered by different groups of therapists. Outcome measures included standardized self-report indices of trauma, depression, and selfconcept. Data analyses revealed statistically significant improvement on all measures for both treatment conditions. At posttreatment, the EMDR group was different from the AL group on four of five outcome measures. This comparison, however, is obscured by the therapist by treatment procedure confound. Such factors as therapist allegiance, enthusiasm, or involvement could have contributed to the measured effects of EMDR (Gaffan, Tsaousis, & Kemp-Wheeler, 1995).
Marcus, Marquis, and Sakai (1997) conducted a similar study that compared EMDR with general outpatient care in a Health Maintenance Organization, and found that those receiving EMDR showed significantly greater and faster improvement on measures of PTSD, depression, and anxiety. However, Marcus et al. (1997) also committed the therapist by treatment confound and risked allegiance, enthusiasm, and involvement artifacts. It was also reported that some subjects in the EMDR condition were receiving HMO treatment (e.g., medication), thus confounding the effect of EMDR (Marcus et al., 1997). In addition, the statistical analyses on difference scores were limited to a large number of multiple t-tests that were not adjusted for Type 1 error. Finally, the limitations of the experimental design of both Marcus et al. (1997) and Scheck et al. (1998) do not exclude the possibility that any apparent change following EMDR was mediated by the imagery exposure that is an incidental characteristic of the treatment.
Effective Treatment Comparisons
Treatment efficacy can also be assessed by comparing a novel treatment for a given disorder with an empirically established treatment for that disorder. If the novel treatment demonstrates a stronger, more general and more effect, or if it is more efficient in its effect, then it can be said to have some incremental efficacy compared with other treatments (Critelli & Neuman, 1984; Lohr et al., 1999). EMDR has been compared with a validated treatment for spider phobia (Muris & Merckelbach, 1997; Muris et al., 1998; Muris, Merckelbach, van Haaften, & Mayer, 1997) involving imagi- nal and in vivo exposure. The three studies employed both cross-over and independent groups designs with children or adults. In all three studies, therapists had received formal training in EMDR, and all studies employed both verbal report and behavioral avoidance measures. The results of each of the three studies showed that both EMDR and exposure reduced verbal reports of fear, but that only exposure treatments resulted in significant reductions of behavioral avoidance. The authors concluded that EMDR confers no additional benefits over exposure treatment for spider phobia.
In discussing a companion study (Pitman, Orr, Altman, Longpre, Poire, Macklin, Michaels, & Steketee, 1996) examining the efficacy of flooding for PTSD, Pitman et al. suggested that EMDR is the preferable treatment. However, Cahill and Frueh
(1997) examined both studies and concluded that several methodological limitations (e.g., different inclusion-exclusion criteria, nonrandom assignment to experiments, treatment-medication confounds) render any conclusions regarding the relative efficacy of the two treatments premature. Indeed, Foa and Meadows (1997) and Keane
(1998) concluded that the methodological limitations of EMDR outcome studies make EMDR as yet an unvalidated treatment for PTSD, notwithstanding the conclusions of Chambless et al. (1998) and Feske (1998).
Devilly and Spence (1999) directly compared EMDR with a cognitive-behavioral treatment (Foa, 1995; Foa, Rothbaum, Riggs, & Murdock, 1991) for PTSD. Subjects diagnosed with PTSD were assessed for PTSD symptoms with self-report and clinician- administered questionnaires and then randomly assigned to either EMDR or Cognitive-Behavioral Therapy (CBT). All subjects received nine treatment sessions of either treatment, where CBT consisted of prolonged imaginal exposure, stress inoculation training, and cognitive therapy. Treatments were videotaped for treatment fidelity and subjects were assessed before and after treatment, and at 1-year follow-up. The results showed that CBT was statistically and clinically more efficacious than EMDR at both posttreatment and at follow-up. Although the two treatments were rated as equally distressing, CBT was rated as more credible and generated higher expectancies for change. Effect sizes were similar to those shown by previous research using the same measures.
The theory underlying EMDR’s efficacy is based on the importance of eye movements or some other stimulation such as finger taps (Shapiro, 1994a, 1994b, 1995a). However, research has shown that imagery without eye movements (or other external stimulation) results in reliable change on the same outcome measures (e.g., Bauman & Melnyk, 1994; Boudewyns et al., 1993; Boudewyns & Hyer, 1996; Devilly et al., 1998; Dunn, Schwartz, Hatfield, & Weigele, 1996; Feske & Goldstein, 1997; Foley & Spates, 1995; Gosselin & Matthews, 1995; Hazlett-Stevens et al., 1996; Pitman, Orr, Altman, Longpre, Poire, & Macklin, 1996; Renfrey & Spates, 1994; Sanderson & Carpenter, 1992). Only one study (D. L. Wilson, Silver, Covi, & Foster, 1996) has reported evidence supporting the necessity of eye movements, but this study is seriously flawed on methodological grounds, including assignment to treatment conditions, confounding of treatment conditions with the method of psychophysiological assessment, and inappropriate statistical analyses (see Lohr et al., 1998). Thus, any apparent change following EMDR is most likely a function of the imagery exposure that is common to both treatments (Muris & Merckelbach, 1997). The same shared process appears to be at work when comparing EMDR with in vivo exposure (Muris et al., 1997, 1998).
Moreover, the same studies comparing EMDR with a no movement control show no difference in either immediate (e.g., Boudewyns & Hyer, 1996; Devilly et al., 1998; Foley & Spates, 1995; Gosselin & Matthews, 1995; Pitman et al., 1996) or long-term (Dev- illy et al., 1998; Devilly and Spence, 1999; Feske & Goldstein, 1997) efficacy. Although rapid eye movements during sleep appear to play a role in the processing of memories (Winson, 1990), the apparent irrelevance of eye movements to the EMDR protocol calls into question the hypothesis that EMDR works by stimulating rapid eye movement sleep:
Further, the effect of eye movements in no way contradicts the potential effects of other stimuli. Clearly, even if treatment effects do prove to be linked to REM, this does not discount other possibilities in the waking state, simply because the body in sleep is incapable of manufacturing external auditory tones, lights, or hand-taps. (Shapiro, 1993, p. 420)
In summary, the most recent controlled research on EMDR suggests that the effects of EMDR are limited largely to verbal report indices, eye movements, and other lateral stimulation unnecessary for clinical improvements, and the observed effects of EMDR are consistent with nonspecific factors, such as factors common to exposure treatments. The findings and methodological limitations are summarized in tabular form in Lohr et al. (1998).
The Professional Evaluation of EMDR
It should be noted that some proponents of EMDR (Greenwald, 1997; Rogers, 1996) have argued that EMDR has been held to higher standards of validation than other treatments for the same conditions. However, Lohr et al. (1998) showed that when the same methodological standards used for extant treatments (Foa & Meadows, 1997) are applied to EMDR, there is no compelling evidence that EMDR is more effective than alternative treatments (e.g., exposure or component control procedures). Indeed, the limitations of EMDR theory and research are sufficiently apparent that EMDR is used as an object lesson in basic problems of research methods in a widely adopted introductory psychology text (Bernstein, Clarke-Stewart, Roy, & Wickens, 1997).
Recent critiques of EMDR (DeBell & Jones, 1997; Hudson, Chase, & Pope, 1998; Muris & Merckelbach, 1999) have expressed caution regarding the widespread adoption of EMDR based on the research evidence. Indeed, there is little evidence to support the extraordinary claims of the most vocal promoters of EMDR (Fensterheim, 1996; Shapiro, 1995a; Shapiro & Forrest, 1997) or the enthusiasm of some mental health practitioners who believe in its unique efficacy. Adherents of EMDR, however, have often resorted to a variety of explanations for negative findings. When early studies (e.g., Jensen, 1994; Sanderson & Carpenter, 1992) failed to support the effectiveness of EMDR (or eye movements), Shapiro (1995a) claimed that researchers had not received proper training, and that a fair test of the method had not been accomplished. When researchers received the sanctioned Level I training and conducted controlled studies yielding null results (e.g., Lohr, Tolin, & Kleinknecht, 1995; Pitman, Orr, Altman, Longpre, Poire, & Macklin, 1996), Level II training became required, and the null results were dismissed as a result of incomplete training (Shapiro, 1995b, 1996b). Thus, the issue of treatment fidelity was used to discount negative findings. It is important to note, however, that no published research demonstrates the necessity of formal EMDR training. Moreover, EMDR training has not been shown to increase adherence and competence regarding the treatment protocol, nor has it been shown to vary systematically with client outcomes. The only empirical findings regarding thee matters show that the correlation between ratings of treatment fidelity and magnitude of EMDR’s clinical effect appears to be relatively low (Lohr et al., 1998; Pitman et al., 1996; R. K. Pitman, personal communication, October 21, 1996), although this issue warrants additional research. For a more detailed analysis of treatment fidelity in EMDR research, see Rosen (1999).
The discrepancy between the meager research support and the extensive promotion of EMDR may be due in part to improper allocation of the burden of proof. Mc- Fall (1991) argued that the burden of proof of positive effects should rest on those who implement and promote novel therapies. Thus, it is reasonable to expect prom- ponents of new treatments to clearly and convincingly answer such questions as, “Does your treatment work better than no treatment?”; “Does your treatment work better than a placebo?”; “Does your treatment work better than standard treatments?”; and “Does your treatment work through the processes you claim it does?” Affirmative answers to these questions require high quality evidence, and the burden of proof ought not be placed on those who raise the questions. It is our opinion that the proponents of EMDR have not met their reasonable burden of evidence, but have often acted as if they have (Shapiro, 1995a).
DISSEMINATION OF INFORMATION REGARDING EFFICACY OF EMDR Professional Communication
Despite the unconvincing evidence for incremental efficacy of EMDR, the professional communication of the clinical effectiveness of this treatment has continued unabated. In the definitive book on EMDR, Shapiro (1995a) presented the extant research in a light favorable for commercial promotion, and a subsequent book (Shapiro & Forrest, 1997) describing EMDR as a breakthrough therapy has been marketed to the consuming public. Professional communication has been accelerated by the use of the electronic media, such as specialty networks and list-servers (Traumatic- email@example.com, EMDR@sjuvm.stjohns.edu, and the EMDR Institute, Inc. World Wide Web site). The list-servers provide a means of exchanging information about EMDR. They also provide a forum for individuals who identify themselves as EMDR trainers or facilitators and who advance strong claims regarding EMDR’s efficacy and clinical applications.
EMDR in the Media
Advocates of EMDR also have made wide use of both print and broadcast media to promote the technique directly to the public. Although presentations of clinical innovations and psychological research findings via various public media are widespread, the case of EMDR is unique in several respects. Most significantly, extremely strong claims have been routinely made about the effectiveness of EMDR for a wide range of disorders, using descriptors such as breakthrough technique (Shapiro & Forrest, 1997) representing a paradigm shift (Shapiro, 1995a) in psychology. To our knowledge, no other psychosocial treatment has generated the degree of media attention across a wide range of venues in such a short amount of time since its introduction. We recognize that the proponents of EMDR cannot be held accountable for inaccurate or exaggerated media coverage of this technique. Nevertheless, a brief examination of the media coverage of EMDR helps to provide a context for understanding the rapid rise in the popularity of this method.
Following a story that appeared on April 12, 1994 in The Washington Post, there occurred widespread national (New York Magazine, May 9, 1994; Newsweek, June 20, 1994; Philadelphia Inquirer, June 26, 1994; Elias, 1994) and international (Der Spiegle, May, 16, 1994) coverage of EMDR. In the broadcast media, segments on EMDR have aired on the ABC News magazine 20/20 (July 29, 1994), and on National Public Radio’s Morning Edition (August 15, 1994). The majority of the popular media stories present EMDR as a clinical breakthrough of impressive, perhaps even miraculous proportions. In many cases, anecdotes of persons cured of various problems are presented as compelling evidence of its effectiveness, whereas the scientific status of EMDR is either distorted or ignored. For example in 1994, the ABC News Magazine 20/20 described EMDR as an “amazing new therapy. . .that rescues people overwhelmed by traumatic memories,” and as a “miraculous new therapy. . .that works in cases where years of conventional treatment have failed.” The majority of the story focused on three clinical anecdotes of trauma victims successfully cured by EMDR, as well as an interview with Francine Shapiro discussing her discovery of the technique. The only hint of critical comment was two sentences totaling 15 seconds in an 11-minute story. The show’s host briefly noted that there were critics of the treatment who questioned its validity. This skepticism, however, was immediately dismissed by the host: “But don’t try telling this to Eric. . . [the client]” No airtime was allotted to critics of EMDR, despite the fact that a leading critic was interviewed extensively on camera, and spent several hours reviewing the scientific evidence concerning the technique with the show’s producers.
The often sensationalistic coverage of EMDR in the popular media is perhaps understandable. The major purpose of the popular media is entertainment in the service of selling goods and services (Nelkin, 1996). On occasion, however, professional journalists have presented a balanced view of EMDR. For example, the June 20, 1994 Newsweek article and a recent news story by Talan (1998) are examples of reasonably balanced presentation of the issues. In the popular media, however, journalistic objectivity is frequently displaced by an emphasis on presenting a story that will sell well to the public.
The public is understandably interested in developments in clinical psychology. Moreover, most psychologists recognize the potential for their work to be exaggerated or otherwise distorted by the popular media. Psychologists do not, of course, have direct control over the content or style of media presentations made by journalists. Nevertheless, they have a responsibility to attempt to ensure that their statements, particularly those concerning novel developments, are objective, balanced, and empirically supported. Unfortunately, it appears that some proponents of EMDR have not approached the media with this reserve. Examples include interviews with EMDR promoters (Coates, 1996) and news stories of magical cures (Oldenberg, 1995).
THE NATURE OF PSEUDOSCIENCE
Many of the proponents of EMDR have made extensive use of information processing meta-language to characterize pathology and the process of therapeutic change (e.g., Shapiro, 1995a). In this way, EMDR is made to appear to be both scientific and scientifically validated. We suggest that the promotion of EMDR provides a good illustration of pseudoscience in general and of how pseudoscience is marketed to mental health clinicians, some of whom may be relatively unfamiliar with the published research on EMDR.
Although philosophers of science have yet to reach complete consensus on the definition of pseudoscience, most definitions share a common core of features. The definition of pseudoscience is probably not a categorical one for which individually necessary and jointly sufficient features can be identified. Instead, the distinction between science and pseudoscience is best viewed as noncategorical or prototypical. The more features of pseudoscience a therapeutic enterprise exhibits, the more suspect it becomes as pseudoscience.
The traditional demarcation between science and pseudoscience hinges on the concept of falsifiability. A theory is scientific if, and only if, its proponents can specify a priori what findings would refute it (Popper, 1965). Thus, the proponents of a scientific position should reasonable and substantively be able to answer the question; “What observable results would lead you to acknowledge that your claim has been falsified?” In scientific psychotherapy, a reasonable answer would be the following: When reasonably well-designed research shows that this therapy is no more effective than no treatment, placebo, or an alternative validated treatment.
Disconfirmation is usually based on the test of predictions that derive from a theory. According to Lakatos (1970), scientific theories are characterized by two crucial components: (a) a hard core of fundamental presuppositions, and (b) a protective belt of auxiliary hypotheses required to test the theory in question. Scientific theories are almost always tested in conjunction with one or more auxiliary hypotheses, that is, hypotheses not directly relevant to, but nonetheless needed to test, the substantive theory of interest (Lakatos, 1970, 1978; Meehl, 1978, 1993). When the results of a test fail to corroborate a theory, the theory is virtually never immediately abandoned. Instead, its advocates typically perform a strategic retreat to the protective belt to modify or tinker with its embedded auxiliary hypotheses (e.g., measure of anxiety was not sufficiently sensitive or intervention was not delivered properly). Sometimes such hypotheses are legitimate alternative explanations, and some strategic retreats can be justified when they increase the theory’s content and predictive power (Meehl, 1993).
In the case of pseudoscience, however, auxiliary hypotheses are invoked simply to explain away results that would otherwise place the original hypothesis in doubt. Under these circumstances, auxiliary hypotheses provide a means by which disconfirma- tion of the experimental hypothesis can be avoided. For example, when controlled tests of EMDR showed no effects of eye movements (Bauman & Melnyk, 1994; Boudewyns et al., 1993; Boudewyns & Hyer, 1996; Foley & Spates, 1995; Gosselin & Matthews, 1995; Pitman et al., 1996; Sanderson & Carpenter, 1992), the null effects were explained away by reinterpreting the EMDR technique as a complex method with many other effective components (Fensterheim, 1996; Hyer & Brandsma, 1997; Shapiro, 1994a, 1995a).
Lakatos (1970) distinguished science from pseudoscience on the basis of progressive versus degenerating research programs. In a progressive research program, theoretical predictions successfully anticipate new data. In a degenerating research program, data tend to precede theory. Pseudoscientific research programs are those that: (a) have degenerated to the point of being incapable of producing corroborated hypotheses, but (b) are nevertheless proclaimed by their proponents as progressive. Despite such proclamations, pseudoscientific theories are much like the Red Queen in Lewis Carroll’s, Alice Through the Looking Glass (Carroll, 1872), who is always “running just to keep in the same place.” Unexpected, disconfirmatory, or both types of findings repeatedly send pseudoscientists into retreat to the protective belt to explain away the anomalies. In the case of EMDR, null results have often been interpreted as a consequence of inadequate training (Greenwald, 1994, 1996; Shapiro, 1995a, 1996a), invalid application of the protocol by researchers (Shapiro, 1995a, 1998b), or both. When comparable effects are found for control procedures intended to manipulate the effects of eye movement, EMDR’s proponents have argued that the control procedure actually is a variant of EMDR (Boudewyns & Hyer, 1996; Hyer & Brandsma, 1997; Renfrey & Spates, 1994; Shapiro, 1995a). As an example of pseudoscientific practice, Shapiro (1998a) interpreted the negative findings comparing EMDR to the component control procedure in Pitman et al. (1996) in the following way:
This [control procedure] duplicated the focused attention, rhythmical aspect, and bilateral stimulation of the guided eye movements all in one condition. It was unsurprising that there were no differences. To complicate it further, the success of focusing on a dot may not even rule out the possibility of bilateratlity, since the optic nerve is crossed to both hemispheres—and maintaining the focus demands bilateral muscle stimulation to hold the gaze.
The last sentence of this quotation illustrates the invocation of an ad hoc hypothesis that makes the theoretical rationale for lateral stimulation in EMDR difficult, if not impossible, to falsify.
Bunge (1967, 1991) described several additional key features shared by most pseudosciences. First, pseudosciences typically do not “ground (their) doctrines. . .in our scientific heritage” (Bunge, 1967, p. 36). In other words, pseudosciences tend not to draw or build on existing scientific concepts, but instead purport to create entirely novel paradigms. In the case of EMDR’s modification of anxiety, there is little discussion of the learning mechanisms typically thought to be responsible for treatment effects in the anxiety disorders, such as habituation or extinction. There is instead a considerable discussion of neuronetworks, bioelectric valences, and other pseudoneurological concepts (Shapiro, 1995a). Shapiro (1995a) also argued that EMDR is the first paradigm shift in psychology since Freud (pp. v, 12-17). Second, pseudosciences are not self-correcting: A pseudoscientific research program interprets every failure as confirmation and every criticism as an attack (Bunge, 1967). Third, independent evaluations of EMDR’s clinical effectiveness (Lohr, Tolin, et al., 1995) or of the methodological rigor of EMDR research (Acierno et al., 1994; Lohr, Kleinknecht, et al., 1995) have often been criticized as erroneous or incompetent (Shapiro, 1996b). Fourth, the primary goal of pseudoscience is persuasion and promotion, rather than truth seeking through the corrective skepticism of the scientific enterprise. In the case of EMDR, the treatment is aggressively marketed to mental health professionals and the general public (EMDR Institute, Inc., 1995, 1996; Shapiro & Forrest, 1997) without reference to evaluation by independent scholars (Acierno et al., 1994; DeBell & Jones, 1997; Foa & Meadows, 1997; Herbert & Mueser, 1992; Hudson et al., 1998; Keane, 1998; Lohr, Kleinknecht, et al., 1995; Tolin, Montgomery, Kleinknecht, & Lohr, 1995).
THE MARKETING OF EMDR PSEUDOSCIENCE
Pratkanis and his colleagues described the commercialization of persuasion (Pratka- nis & Aronson, 1991) and the selling of pseudoscience (Pratkanis, 1995) as related, social influence processes. The promotion of pseudoscience involves a number of social psychological principles that have been successfully used by those who sell commodities (materials, treatments, and ideas) to the public. We contend that many mental health providers, consumers, and health care agents have overestimated the efficacy of EMDR as a result of age-old fallacies ofjudgment.
Pratkanis (1995) described the way in which pseudoscience is marketed through specific promotional tactics: the creation and use of phantom goals, the construction of vivid appeals, the use of pre-persuasion, the use of the rationalization trap, and the establishment of a professional granfalloon. The initial sales tactic is the creation of phantom goals, and the development of alternative means to attain them. Phantom alternatives, according to Pratkanis and Farquhar (1992), are desirable goals that appear credible but are currently unavailable. In the case of PTSD, the phantom is the cure (Shapiro, 1989) for a relatively refractory condition (Solomon et al., 1992) using a breakthrough (Shapiro & Forrest, 1997) alternative. Shapiro’s (1989) original account of treatment that claimed a 100% success rate for traumatic memories in a single treatment session is an example of this tactic.
A related process is the construction of vivid appeals to persuade potential consumers. Vividly presented case studies can be far more convincing than scientific data. As a result, isolated hits typically receive greater weight than the more informative negative results in the laboratory. Authors of uncontrolled case reports have sometimes made extraordinary claims regarding the speed, magnitude, and generality of EMDR’s effects. For example, McCann (1992) reported that he not only cured a case of refractory PTSD caused by a catastrophic fire, but that the patient made dramatic life changes after brief EMDR treatment. Marquis (1991) used case studies to argue that EMDR was effective not only for PTSD but for depression, eating disorders, and learning disabilities. Other case reports claim that EMDR is an effective treatment for the distress associated with sexual dysfunction (Wernik, 1993), alcoholism (Shapiro, Vogelmann-Sine, & Sine, 1994), and dissociative disorders (Lazrove, 1994). Such case studies can serve to undermine the persuasive power of adequately controlled experiments that yield unconvincing results. The vivid individual case report can be more compelling to consumers than are substantive, but dry, randomized clinical trials. Some individuals may indeed respond positively after applying EMDR, but it is not difficult to find individuals who respond positively after experiencing most any form of intervention.
The use of prepersuasion is a third means of promulgating pseudoscience. Prepersuasion consists of defining the situation or setting the stage in one’s favor. One way in which this is accomplished is by interpreting disconfirmatory results in support of prior expectations. Pratkanis (1995) called this the “illusory placebo effect” (p. 23). In the case of EMDR, ambiguous or negative findings are interpreted in favor of EMDR. The illusory placebo effect capitalizes on auxiliary hypotheses (Meehl, 1978, 1990; Popper, 1965, 1983) in explaining away contrary or undesirable results. For example, Renfrey and Spates (1994) found no differences in outcome between EMDR and a control condition in which subjects tapped their fingers rather than moving their eyes. The appropriate interpretation of this null result is that EMDR is not better than a placebo treatment. Nevertheless, Renfrey and Spates (1994) interpreted these results as evidence that both EMDR and finger tapping are viable treatments. In their review of Renfrey and Spates’s study, the EMDR Institute, Inc. (1996) described the control condition as “EMDR using fixed visual attention” (p. 3); that is, as eye movement treatment without eye movement. Thus, even null results are interpreted as supporting EMDR’s efficacy. We agree with Pitman, Orr, Altman, Longpre, Poire, and Mack- lin (1996) that when experimental control reveals the null effect of eye movements, the neurological theory of eye movements must be rejected. In addition, we must seriously question the unique nature of EMDR relative to other imagery exposure treatments.
A fourth process involves the use of a rationalization trap. The rationalization trap is based on the principle that quick commitment on the part of the consumer changes the consumer’s perspective. Psychologist consumers who might initially have been skeptical are compelled to rationalize the commitment they have made and will alter their beliefs accordingly. During the past few years, the rationalization trap was achieved by requiring all EMDR trainees to make an initially small, but psychologically important, commitment. Prior to the publication of Shapiro (1995a), trainees were required to sign a consent form stating because EMDR is a powerful procedure that could be dangerous in the wrong hands, the trainee must agree not to teach others how to perform the technique. Another consent form states that because of its potential power, EMDR could be dangerous to trainees suffering from certain psychological disorders and that trainees must assume responsibility for any negative effects they might experience during the practice sessions. Shapiro (1995c) justified these forms on the basis of client protection and assurance of treatment fidelity. Although these claims may have some merit, they have the added psychological effect of persuading the trainees (even before the training has taken place) that: (a) EMDR is a powerful, quasi-mystical procedure, and (b) training by official EMDR Institute, Inc. representatives is crucial. They ask the trainee to affirm the conclusion that is at issue: the question of EMDR’s efficacy. Bearing this in mind, it is perhaps understandable that Lipke (1994) reported that 77% of the most highly trained participants (Level II), surveyed after completion of training, agreed that extensive training was a vital step in using the technique.
A fifth pseudoscientific process is the establishment of what Vonnegut (1976) called a “granfalloon”: a proud and meaningless association of human beings. Granfalloons are easy to create and establish a sense of social identify among the consumers of the persuasive message. Once such a group has been established, individuals become reluctant to express beliefs that are inconsistent with those of the group. In the EMDR granfalloon, trainees have historically signed a vow not to train others (i.e., those not in the granfalloon), after which they observe the EMDR Institute-approved trainer in the company of facilitators, a specially identified group of clinicians who have particular responsibilities at training sessions and whose special status is officially recognized by the EMDR Institute, Inc. (Leeds, 1996). The initial training workshop is followed by Level II training, during which distinctive treatment protocols and special clinical applications are discussed. This process continues when the trainee is invited to become a member of the EMDR Network, an assemblage that provides special privileges such as a newsletter, research summaries, and patient referrals. There follows eligibility to participate in an EMDR electronic mail list (EMDR@sjuvm.stjohns.edu) and membership in the Eye Movement Desensitization and Reprocessing International Association (EMDRIA).
Participants in the workshops also receive certificates: one for “Attendance” after Level I training, and another for “Completion” after Level II training. The certificates are attractive, suitable for framing, and accompanied by a foil seal in the center of which is the name of the developer of EMDR. Such certificates are also known as the “title licenses” that confer no formal or professional status and have no relationship with state licensing or certification boards (Bryant, 1997). In their book Crazy Therapies, Singer and Lalich (1996) noted the use of such certificates as a frequent sales tactic among innovators of newly invented psychological cure-alls, and pointed out the persuasive value such certificates carry for the public.
The granfalloon also functions as a means of acquiring specialized information. Collective compliance with a “no training” contract likely creates a sense of group identity among trainees, and as a consequence, trainees may gain a sense of possessing special knowledge. The group identity of EMDR trainees is solidified through a number of means, such as training in specialized terminology (Shapiro, 1995a) that is unique to those who are members of the granfalloon. Certain fears are known as “process phobias”. Eye movements are performed in “saccade sets.” Anxiety-eliciting thoughts and memories are referred to as “hot spots” and persistent rumination is referred to as “looping.” The therapeutic modeling of adaptive selfstatements is called “cognition installation,” the linking of one idea to another is called the “cognitive interweave,” and the working through of a troublesome problem is called “cleaning it out.” Van Rillaer (1991) refered to this pseudoscientifc tactic as “dissimulation.” It involves the use of obscurantist language to compensate for an absence of content and to discourage would-be skeptics. Shapiro (1995a) employed this tactic most explicitly in the development of the Accelerated Information Processing model underlying EMDR:
[The] valences of the neural receptors (synaptic potential) of the respective neuro networks, which separately store various information plateaus and levels of adaptive information, are represented by the letters Z through A. It is hypothesized that the high-valence target network (Z) cannot link up with the more adaptive information, which is stored in networks with a lower valence. That is, the synaptic potential is different for each level of affect held in the various neuro networks. . . .The theory is that when the processing system is catalyzed in EMDR, the valence of the receptors is shifted downward so that they are capable of linking with the receptors of the neuro networks with progressively lower valences. . . .(Shapiro, 1995a, pp. 317-318)
This explanation of EMDR mechanisms of action is a paradigmatic example of dissimulation as described by Van Rillaer (1991). It is the use of scientific-sounding terms to provide EMDR with the veneer of science, but not the substance (cf. O’Donohue & Thorp, 1996).
Finally, pseudoscience flourishes when skepticism is devalued. Pratkanis (1995) argued that skeptics are often attacked by pseudoscientists through innuendo and character assassination rather than reasoned argumentation. In this way, the debate is quickly removed from the theoretical and empirical issues at hand (e.g., does a given treatment work?) and instead moves to personal arena of ad hominem assault. Critics of EMDR (Jensen, 1994; Lilienfeld, 1996; Lohr, Kleinknecht, et al., 1995; Tolin et al., 1995) sometimes have been attacked with questions concerning their professional training (EMDR Institute, Inc., 1996; Shapiro, 1996c), ulterior motives, and competence (Shapiro, 1995b, 1995c, 1996a, 1996b, 1996c, 1996d). The questioning of professional competence has also been directed atjournal editors (Lipke, 1999) who have recommended publication of articles that have questioned the efficacy of EMDR (Jensen, 1994; Lohr et al., 1998).
Common Practices in EMDR Pseudoscience
Unlike science, which actively seeks empirical disconfirmations, pseudoscience seeks verification through uncontrolled but vivid demonstration. Popper (1965) observed that proponents of pseudoscience tend to search for and attend to confirming findings (alleged proofs), and to avoid and neglect potentially disconfirming findings (disproof). Moreover, Bunge (1967) observed that “the pseudoscientist, like the fisherman, exaggerates his catch and neglects his failures or excuses them” (p. 36).
Shapiro (1995b, 1995c, 1996b) provided such an analysis to justify the application of EMDR to a wide range of clinical problems. Published accounts that cast doubt on the effects of EMDR are ignored or discounted for a variety of reasons. For example, Shapiro (1995a) alleged that the null findings of researchers who have not had training sanctioned by EMDR, Incorporated, Inc. (Jensen, 1994; Montgomery & Ayllon, 1994a, 1994b) are due to invalid treatment application, when in fact no data exist to support this claim. Indeed, Rosen (1999) analyzed in detail the issue of treatment fidelity in EMDR research and has shown that it is has been used unevenly by proponents of EMDR (Greenwald, 1996, 1997; Shapiro, 1995a) as a means of minimizing scientific data that question the efficacy of the treatment and the theory upon which it is based. Moreover, Shapiro’s (1995a, 1996a, 1996b) evaluations of the research literature stand in stark contrast to independent reviews (Acierno et al., 1994; Foa & Meadows, 1997; Hudson et al., 1998; Lohr, Kleinknecht, et al., 1995; Lohr et al., 1998), which indicate that treatment effects are largely limited to verbal reports of distress. The independent reviews also identify inadequate controls for procedural artifacts (e.g., nonspecific effects) and inadequate comparisons with other treatments that severely limit conclusions regarding efficacy.
The most essential feature of science is the maximization of criticism (Bartley, 1984). Good scientific research is an attempt to expose cherished hypotheses to stark criticism in order to gain a better understanding of errors in one’s web of belief. Genuine science is not a craving to be correct, but rather a craving to learn where we are wrong so that our errors can be eliminated. It is through error elimination that knowledge grows. Thus, the best and most efficient way of rooting out error in our beliefs is to expose them to severe criticism and strong empirical tests (Borkovec & Bauer, 1982; Borkovec & Castonguay, 1998; Hazlett-Stevens & Borkovec, 1998; Platt, 1964). Severe criticism is manifested in the use of rigorous methodological controls so that one does not make the mistake of believing that the therapy causes improvement when it does not. For example, a design that does not include a placebo or procedural control may allow the experimenter to believe erroneously that the treatment was uniquely effective, when its effects were in fact due to nonspecific factors (Lohr et al., 1999).
The creative application of scientific skepticism, however, has not frequently characterized the publicity surrounding EMDR. Instead, the emphasis has been on personal anecdote and clinical observation, both of which serve as the mode of communication and a means to increase belief in the communicator. We are left with a process of belief promotion rather than science. It is this context that has led a number of commentators to characterize EMDR as pseudoscience in both contemporary (Lohr, 1996; O’Donohue & Thorp, 1996) and historical (McNally, 1996, 1999) perspective.
It should be noted that many of the criticisms of pseudoscience in this paper are not limited to EMDR or other Power Therapies. Indeed, colleagues and reviewers have correctly pointed out that several mainstream cognitive-behavioral interventions also suffer from a lack of empirical data regarding specific treatment effects and have been vigorously promoted in a fashion that far exceeds the available data. For example, Wessler (1996) characterized Rational Emotive Behavior Therapy (REBT; Ellis, 1993, 1994) as pseudoscientific for some of the same reasons we have characterized EMDR as pseudoscientific:
REBT maintains hypotheses for which there is no empirical support, and its hypotheses are largely untestable. Its founder worked in isolation, particularly developing the parallel process theory of emotion, which has attracted so little attention from serious researchers and theorists in the field that the necessary studies have not been done. What little research has been done does not support REBT theory but this has been ignored. REBT has become, ironically, a set of nonempirical assertions masquerading as a scientific psychotherapy. (Wessler, 1996, p. 52)
A thorough critique of all forms of cognitive-behavioral treatment is beyond the scope of this paper. Although we do not wish to take a stand on the merits of Wessler’s criticisms of REBT, it appears likely that the differences between the promotion of EMDR and some forms of cognitive-behavioral treatment are primarily of degree, rather than of kind. However, the discrepancy between the marketing and the data has been particularly wide in the case of EMDR. Although we do not advocate that EMDR be held to higher standards than other treatments, we do suggest that the claims and practices of EMDR proponents merit particular attention due to their extraordinary nature.
ACCEPTANCE OF EMDR BY PROFESSIONAL PSYCHOLOGISTS
In the sale of any commodity, a transaction takes place between two parties: the seller and the purchaser. Although we have emphasized the selling of EMDR, it is also necessary to examine aspects of its purchase. In an ideal scientific world (McFall, 1991), the decision to purchase and use clinical procedures would be determined solely by the content of academic and professional training (knowledge) of the purchaser and by the empirical validation of those procedures (commodity). The clinical armamentarium would then consist of effective and validated assessments and treatments. We do not live in an ideal world, however, and clinicians purchase procedures outside the context of formal training and research. Indeed, the split between scientist and practitioner appears to be ever widening (Fox, 1996), and it is important to identify the processes by which the incorporation of unvalidated procedures occurs. One process is that of clinical and financial expediency. Novel, unique, or intractable cases may require the application of experimental procedures, but they should be explicitly identified as such.
ALTERNATIVE WAYS OF KNOWING IN PROFESSIONAL PSYCHOLOGY
Other processes more subtle and substantive dissociation of the practitioner from the body of empirical science. The dissociation is based partly on the professionalization of psychology and the development of an alternative model of clinical knowledge (Tsoi Hoshmand & Polkinghorne, 1992). This alternative model is based on postmodern epistemology (Kvale, 1992). The increasing influence of postmodern attitudes in academic circles, as well as the reasons underlying this trend, have been documented by Gross and Levitt (1994) and Sokal and Bricmont (1998). Postmodern thinking may no longer be limited, however, to the halls of the academy. Although postmodernism is difficult to define, its central tenets include the propositions that: (a) all knowledge is contextual and therefore relative, and (b) science represents only one mode of discourse among many, and that scientific claims to knowledge are no more privileged than alternative claims (e.g., assertions based on intuition or personal experience). Most postmodernists therefore believe that the concept of truth is a dangerous and misleading illusion. Because all facts are situated in a specific cultural and historical context, such facts can never attain the status of universal knowledge claims. Postmodern modes of thinking lend themselves in many cases to a willingness to accept claims on the basis of subjective convictions. According to most postmodernists, such convictions are not inherently inferior to beliefs derived from systematic scientific research. As Englebretsen (1995) noted:
Premoderns and moderns based their willingness to accept or reject a speaker’s claim on their judgment of how well it seemed to fit the facts of the case and to what extent it was logically consistent with the speaker’s other claims or assumptions. By contrast, postmoderns ‘play the believing game,’ accepting the speaker’s claim according to the degree of sincerity the speaker exhibits. Truth and coherence are no longer allowed to bully us in our communicative efforts. (p. 52)
Some postmodern thinkers in psychology have further suggested that psychotherapeutic procedures should be based as much on validation through practice (Kvale, 1992), that is, on a tacit learning of what works by means of experience, as on research findings derived from controlled outcome studies (see also Schon, 1983). Tsoi Hosh- mand and Polkinghorne (1992) similarly argued that clinical reflection and intuition (i.e., “practicing knowledge”) should be placed on a par with scientific knowledge in the formal training of psychotherapists. They noted that “in relating theory to practice, research traditionally served as gatekeeper for entry into a discipline’s body of knowledge,” and that “In practicing knowledge, however, the test for admission is carried out through the use of reflective thought” (Tsoi Hoshmand & Polkinghorne, 1992, p. 62; see also Polkinghorne, 1992). Remarkably, such discussions contain virtually no mention of the factors (e.g., absence of immediate and consistent feedback) that often prevent psychotherapists from learning from experience, or of the social cognitive errors (e.g., selective recall, availability biases, and confirmation biases) that tend to create an illusion of such learning in its absence (Dawes, 1994; Dawes, Faust, & Meehl, 1989).
Are we stretching matters too far to suggest an analogy between postmodern thinking and the premature and uncritical acceptance of EMDR by many practitioners? Regrettably, we do not think so. The disturbingly rapid embrace of EMDR by thousands of clinicians prior to the publication of adequately controlled research suggests a willingness to place personal experience over scientific evidence, to value anecdote and clinical surmise over experimentation. Meehl (1993) warned of this ominous trend in much of modern clinical psychology. His comments serve as a needed reminder to those who might be inclined to dismiss EMDR as an isolated example of pseudoscience:
My teachers at Minnesota (including Hathaway, Paterson, Skinner, and Feigl). . . shared what Bertrand Russell called the dominant passion of the true scientist—the passion not to be fooled and not to fool anybody else. Only Feigl was a positivist, but all of them asked the two searching questions of positivism: “What do you mean? How do you know?” If we clinicians lose that passion and forget those questions, we are little more than be-doc- tored, well-paid soothsayers. I see disturbing signs that this is happening and I predict
that, if we do not clean up our clinical act and provide our students with role models of scientific thinking, outsiders will do it for us. ( Meehl, 1993, pp. 728-729)
IMPLICATIONS FOR PROFESSIONAL PSYCHOLOGY
If EMDR were the only treatment being commercially promoted, the task of empirical evaluation would be large but not insurmountable. It would take time and professional resources to rectify the commercial excesses, but the effort would be worth the outcome. For example, several years elapsed following the introduction of facilitated communication for the treatment of severe autistic and developmental disorders before its empirical debunking was convincing (Delmolino & Romanczyck, 1995; Jacobson et al., 1995). There are, however, a large number of largely or entirely unvalidated therapies being actively marketed to those providing traumatology services, including TFT (Callahan, 1995; Gallo, 1995), TIR (Gerbode, 1985, 1995), VKD (Bandler & Grinder, 1979), and Critical Incident Stress Debriefing (CISD; Mitchell, 1988). These interventions represent a cottage industry that is being actively promoted to the mental health profession via workshop training that is outside the context of substantive evaluation (Figley, 1995, 1997; Figley & Carbonell, 1996). These techniques either have not been empirically investigated using controlled studies (Gist, 1996; Gist, Lu- bin, & Redburn, 1998; Hooke, 1998) or have been found to be no more effective than control procedures (Lohr et al., 1998, 1999). Moreover, two of these procedures are now being promoted and marketed for the treatment of emergency service personnel (Solomon, 1996). The promotion involves a psychological service that combines CISD, a procedure that appears to have little or no effect on subsequent trauma symptoms (Gist et al., 1997, 1998; Harris, 1997), with EMDR, a procedure that has been found to be no more effective than control conditions with which it has been compared (Foa & Meadows, 1997; Keane, 1998; Lohr et al., 1998, 1999; Muris & Merck- elbach, 1999).
How are psychologists to understand the phenomenon of EMDR? We suggest that the field of psychotherapy has been insufficiently rigorous regarding the evidentiary credentials of psychotherapeutic procedures (Borkovec & Castonguay, 1998; Hazlett- Stevens & Borkovec, 1998). EMDR appears to possess the outward form of science but little of its substance. The appearance of science, such as case studies reported in peer reviewed journals, selective publicity of weak tests of effectiveness, scientific-sounding jargon, and seemingly cautious promotion (“only clinicians with sanctioned training should use it”) serve to obscure EMDR’s lack of scientific substance and have persuaded many of its scientific legitimacy. Although there is little evidence to support the strong claims of EMDR’s proponents, this treatment has resulted in a significant financial return. Twenty-five thousand trained mental health clinicians (EMDR Institute, Inc., 1997; Shapiro, 1998b) at a several hundred dollars per capita amounts to a significant sum.
PRACTICAL CONSIDERATIONS FOR PROFESSIONAL PSYCHOLOGISTS
When experimental research consistently demonstrates that EMDR without eye movement or lateral stimulation is as effective as the full treatment procedure, it is no longer reasonable for clinicians to learn the clinical intricacies of their hand movements (or the use of automated flashing light bars and sound generators) while misinforming their clients that they can expect accelerated information processing as a consequence. Nonetheless, workshop training in the full EMDR method continues at an extraordinary pace (Rosen, 1996), with large numbers of mental health clinicians learning hand movements and other methods of tactile stimulation. In the context of these paradoxical circumstances, we offer the following recommendations to practicing clinical and counseling psychologists, and other mental health professionals.
First, we recommend that psychologists remember the history of failed therapies (Rosen, Lohr, McNally, & Herbert, 1998). These are therapies that at first induce high levels of expectation and miraculous cures, but ultimately fail the test of time (Walsh, 1923). We recommend a rereading of Frank’s (1961) classic text, Persuasion and Healing, and other works (e.g., Walsh, 1923) on placebo effects and the history of failed cures. We recommend discussing with colleagues the decision rules by which the clinician decides when to ignore and apply new treatment procedures. Add to these rules the caveat that if a procedure is heavily promoted through extraordinary claims, those claims must be accompanied by equally extraordinary empirical evidence. The nature of the evidence should not be based on clinical testimony or on vivid case studies. Instead, the evidence should rest upon strong and sophisticated control conditions that can identify the effects of procedural artifacts and nonspecific factors (Borkovec & Castonguay, 1998; Lohr et al., 1999). In one of the earliest texts on the practice of psychotherapy, Walsh (1912) encouraged the same caution:
We have had ever so many more experiences of disappointment after the introduction of remedies which cured at the beginning of their history, than we have had of remedies that maintain themselves after prolonged experience. It is the attitude of skepticism and suspended judgment until after a remedy or method of treatment has been tried on many different kinds of cases in varying circumstances that constitutes the only sufficient safeguard against repeating the unfortunate errors of old times. . . (p. 51)
It is true that the attitude of skepticism carries the necessary risk of delaying the implementation of new and efficient treatments. However, an attitude of uncritical acceptance carries greater risks, both for the client and the profession (Jacobson et al., 1995; Valenstein, 1986). In closing, we would like to distill our analysis of EMDR by referring to the late astrophysicist Sagan’s (1996) book, The Demon Haunted World. In discussing skepticism as the central value of the scientific enterprise, Sagan wrote: “Keeping an open mind is a virtue—but, as the space engineer James Oberg once said, not so open that your brains fall out” (p. 187). We believe it is an admonition that contemporary professional psychologists should heed seriously.
A meta-analysis of the contribution of eye movements in processing emotional memories
a School of Psychology, Murdoch University, South St., Murdoch WA 6150, Australia
b Department of Clinical Psychology and EMGO Institute for Health and Care Research, VU University and VU University Medical Center Amsterdam, The Netherlands
Background and objectives: Eye Movement Desensitisation and Reprocessing (EMDR) is now considered evidence based practice in the treatment of trauma symptoms. Yet in a previous meta-analysis, no significant effect was found for the eye movement component. However methodological issues with this study may have resulted in a type II error. The aim of this meta-analysis was to examine current published studies to test whether eye movements significantly affect the processing of distressing memories. Method: A systematic review of the literature revealed two groups of studies. The first group comprised 15 clinical trials and compared the effects of EMDR therapy with eye movements to those of EMDR without the eye movements. The second group comprised 11 laboratory trials that investigated the effects of eye movements while thinking of a distressing memory versus the same procedure without the eye movements in a non-therapy context. The total number of participants was 849.
Results: The effect size for the additive effect of eye movements in EMDR treatment studies was moderate and significant (Cohen’s d = 0.41). For the second group of laboratory studies the effect size was large and significant (d = 0.74). The strongest effect size difference was for vividness measures in the non-therapy studies (d = 0.91). The data indicated that treatment fidelity acted as a moderator variable on the effect of eye movements in the therapy studies.
Conclusions: Results were discussed in terms of current theories that suggest the processes involved in EMDR are different from other exposure based therapies.
© 2012 Elsevier Ltd. All rights reserved.
A number of previous meta-analyses have found that EMDR has sustained and lasting treatment effects for Posttraumatic Stress Disorder (Bisson et al., 2007; Bradley, Greene, Russ, Dutra, & Westen, 2005; Seidler & Wagner, 2006). EMDR is now considered to meet criteria for evidence-based practice in the United Kingdom by the National Institute for Clinical Excellence (2005), in America by the American Psychiatric Association (2004), in Australia by the Australian Centre for Posttraumatic Mental Health (2007), and in the Netherlands by the Dutch National Steering Committee for Guidelines for Mental Health Care (2003).
Although the active processes in EMDR appear to be different to traditional exposure treatments (Lee, Taylor, & Drummond, 2006), the mechanism of action for the success of EMDR remains controversial (Rogers & Silver, 2002; Shapiro, 2012; Smyth & Poole, 2002). There is disagreement as to whether eye movements add anything to the effectiveness of EMDR (Davidson & Parker, 2001; MacCulloch, 2006).
The treatment studies that have attempted to isolate the eye movement component from the full treatment package have produced results ranging from a very large effect size consistent with eye movements enhancing processing (Wilson, Silver, Covi, & Foster, 1996) to findings of no differences (Renfrey & Spates, 1994). On the other hand, non-clinical laboratory studies that investigated the effects of eye movements on autobiographical memories have found decreases in vividness and/or emotionality compared to control conditions such as finger tapping (van den Hout, Muris, Salemink, & Kindt, 2001), spatial tapping (Andrade, Kavanagh, & Baddeley, 1997), and no eye movement (Barrowcliff, Gray, Freeman, & MacCulloch, 2004; Gunter & Bodner, 2008; Kavanagh, Freese, Andrade, & May, 2001). Whilst these laboratory studies show a clear processing effect for eye movements, they did not involve all the procedural elements of EMDR (Shapiro, 2001: p. 472).
In an attempt to discover any general trends in research that has examined the effects of eye movements on memory, Davidson and
Parker (2001) conducted a meta-analysis of published studies investigating effect size differences between EMDR with eye movements and EMDR without eye movements. Their conclusion when looking at pre-versus-post single session measures was that there was no significant additional effect of eye movements. Their measure of effect size was R, which ranges from plus one to minus one; R2 is the amount of variance in the dependent variable accounted for by the independent variable. However there were methodological problems in this meta-analysis. Initially R scores were converted to Z scores. The simple mean of these scores was converted back to R, and then subjected to a t-test using the number of studies to determine the degrees of freedom. The problem with this approach is that it treats all studies as if they are of equal weight. The usual practice in meta-analysis is to weight each study in relation to the number of participants and for the degrees of freedom to be calculated using the total number of participants (Rosenthal & DiMatteo, 2001). This provides a more appropriate test of significance and provides more power to investigate small magnitude effect sizes (Rosenthal, 1991).
Since 2001, there have been additional published papers investigating the effects of eye movements on various measures. Therefore, we decided to conduct a new meta-analysis, including all studies published in the past 23 years and adjusting for the sample size in each study.
1.1. Search procedure
Searches were conducted in Medline, PsycINFO, and Science Direct databases. The search was done in two parts: the first used the keywords non eye movement or no eye movement or eyes fixed or eyes stationary or without the eye movement or eye stationary paired with eye movements, or eyes moving or eye movement; the second also used a keyword search of eye movements paired with eye movement desensitization. The search was restricted to articles only involving humans and between 1989 (when EMDR was first published) and 2012. An a priori decision was made to search only published work and to control for publication bias by a posteriori analysis. Additional studies were identified by manual searches of past meta-analyses (Davidson & Parker, 2001; Rodenburg, Benjamin, de Roos, Meijer, & Stams, 2009) and recent reviews of the role of eye movements in EMDR (Gunter & Bodner, 2009; Smeets, Dijs, Pervan, Engelhard, & van den Hout, 2012).
1.2. Inclusion/exclusion criteria
We included randomized controlled trials in which a negative memory task with eye movements was compared to the same task but without the eye movements, under otherwise identical conditions. Thus if a study compared eye movement plus tapping to no eye movement plus tapping then such a study could be said to compare the presence or absence of eye movement in identical conditions. However a study that compared eye movement without tapping to no eye movement with tapping is not comparing the main variable of interest in identical conditions. Therefore we included only studies comparing eye movements versus no eye movement, studies in which eye movements were compared with an alternative stimulus were excluded.
We included two types of studies. In the first type, (laboratory studies) the participants simply were asked to think of a distressing memory and then they were randomized to a procedure with eye movements or to the same procedure but without eye movements. This was done in all these studies over a very short period of time and in one session (average total eye movement exposure 52 s).
The second group of studies (treatment studies) examined the effects of EMDR on participants with an anxiety disorder or a distressing memory, and compared EMDR with eye movements with exactly the same procedure but without the eye movements. These clinical interventions used between 5 and 8 phases of the EMDR treatment protocol (Shapiro, 2001: p. 472) and these studies had more extensive exposure to eye movements than the first group of studies. We decided to conduct an independent meta-analysis for each of these two groups of studies.
1.3. Quality assessment
We assessed the validity of the treatment and laboratory studies using four criteria of the ‘Risk of bias’ assessment tool, developed by the Cochrane Collaboration (Higgins & Green, 2008). This tool assesses possible sources of bias in randomized trials, including the adequate generation of allocation sequence; the concealment of allocation to conditions; the prevention of knowledge of the allocated intervention; and dealing with incomplete outcome data. The two other criteria of the ‘Risk of bias’ assessment tool (suggestions of selective outcome reporting; and other problems that could put it at a high risk of bias) were not used in this study, because we found no clear indication that they had influenced the validity of any of the studies reviewed.
We also rated the quality of the treatment implementation using three criteria which were based on an authoritative review of empirically supported psychotherapies (Chambless & Hollon, 1998): (1) the study referred to the use of a treatment manual (either a published manual, or a manual specifically designed for the study); (2) the therapists who conducted the therapy were trained for the specific therapy, either specifically for this study or as general training; (3) treatment integrity was checked during the study (by supervision of the therapists during treatment or by recording of treatment sessions, or by systematic screening of protocol adherence with a standardized measurement instrument). The ratings were made by two PhD students and each study was discussed until a consensus was reached.
For each study, we calculated Cohen’s d (standardized mean difference) by subtracting (at post-test) the average score of the control group (Mc) from the average score of the experimental group (Me) and dividing the result by the pooled standard deviations of the experimental and control group (SDec). Effect sizes of 0.80 and higher are regarded as large, while effect sizes of 0.50— 0.80 are moderate, and lower effect sizes are small (Cohen, 1988). Because several studies had small sample sizes we corrected the effect size for small sample bias according to the procedures suggested by Hedges and Olkin (1985). Each author separately calculated effect size data from each study and discrepancies were discussed until consensus was reached. When means and standard deviations were not available in the study, we used other statistics (t-value, p-value) to calculate the effect size using Comprehensive Meta-analysis software (version 2.2057; CMA). When a study reported only a non-significant difference between conditions at post-test without reporting more specific statistics, we conducted the authors and asked for more specific data otherwise we assumed a zero effect size. The calculated effect sizes were based on self report and observer rated symptoms only. An early attempt was made to include physiological measures. However, these varied largely between the studies in the type of physiological measures used and the way that they were reported. This prevented any meaningful analysis across the studies and so this data was excluded.
We pooled the mean effect sizes (Cohen’s d) with CMA. If there were multiple outcomes within a study we selected the CMA option to use the mean of the selected outcomes. We choose to conduct random effects meta-analysis. Therefore, each study was weighted by the inverse of its variance, in which the variance includes the within-studies variance plus the estimate of the between-studies variance, tau-square. More information about the exact methods for pooling studies in a random-effects model is detailed in Borenstein, Hedges, Higgins, and Rothstein (2009).
As a test of homogeneity of effect sizes, we calculated the I2— statistic which is an indicator of heterogeneity in percentages. A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity (Higgins, Thompson, Deeks, & Altman, 2003). We also calculated the Q-statistic, but only report whether this was significant or not.
Subgroup analyses were conducted according to the mixed effect model. In this model, studies within subgroups are pooled with the random effects model, while tests for significant differences between subgroups are conducted with the fixed effects model. For continuous variables, we used meta-regression analyses to test whether there was a significant relationship between the continuous variable and the effect size, as indicated with a Z-value and an associated p-value. Two subgroup analyses were planned to see if previous meta-analysis findings would be replicated in the treatment studies. The first was that treatment integrity had been found to moderate the effect size of symptom reduction following EMDR (Maxfield & Hyer, 2002) and the second was that EMDR effect size was moderated by the type of population treated with larger effect sizes associated with non-student populations (Davidson & Parker, 2001).
Publication bias was tested by inspecting the funnel plot on primary outcome measures, and by Duval and Tweedie’s trim and fill procedure (Duval & Tweedie, 2000), which yields an estimate of the effect size after the publication bias has been taken into account (as implemented in Comprehensive Meta-analysis, version 2.2.021).
2.1. Inclusion of studies
A flowchart describing the selection of studies is reported in Fig. 1. The three searches and four articles resulted in 891 unique studies. Of these 103 were excluded because they studied the effect of eye movement during sleep and 314 were excluded because they contained no original data and were review papers only. A further 297 were excluded because they were either a case report of EMDR treatment or a study looking at a treatment outcome study comparing EMDR to a waitlist or an alternative treatment procedure. A further group of 61 studies were excluded either because the eye movements was not compared to no eye movement under identical condition, for example (Elofsson, von Scheele, Theorell, & Sondergaard, 2008), or the comparison lacked sufficient randomisation, or the study was a prepost design that did not control for order effects for example (Montgomery & Ayllon, 1994).
Of the 116 remaining studies, 82 were excluded because they did not test for the effects of eye movement on any negative or trauma memory. Within these studies, 72 investigated the effects of eye movements compared to no eye movements on a purely perceptual task, for example (Schmidt, Richardson, Arsenault, & Galantucci, 2007), and were therefore excluded. A further 11 studies were excluded because the effect of eye movements was not tested on a negative memory. Such studies investigated diverse phenomena ranging such as investigating the effects of eye movements on performance of a memory recognition task (Parker, Relph, & Dagnall, 2008), or whether eye movements improved performance on a semantic flexibility task (Kuiken, Bears, Miall, & Smith, 2001), or whether it affected exposure to gory slides (Tallis & Smith, 1994), or whether it effected current distress associated with an anticipated aversive experience (Engelhard et al., 2011).
Of the 33 remaining studies, which all tested the effects of eye movement on a negative or trauma memory, 5 were excluded because the eye movements condition was not compared to a no eye movement control. In these studies, a control procedure involved another attention demanding task such as tapping, for example (Pitman, Orr, Altman, & Longpre, 1996), or auditory tones (Servan-Schreiber, Schooler, Dew, Carter, & Bartone, 2006). Given that such tasks have been described as alternatives to eye movements in EMDR therapy (Shapiro, 2001: p. 472) and that two possible theories to account for EMDR effectiveness suggest that alternative stimulation may be as effective as eye movements, namely the working memory paradigm (Gunter & Bodner, 2008) and the orienting response paradigm (Armstrong & Vaughan, 1996), it was decided to restrict the meta-analysis to studies comparing a procedure with eye movements with the same procedure but without eye movements. In studies that included an eye movements versus no eye movement trial and an eye movements versus an alternative stimulus trial, only the eye movements versus the no eye movement trial was included in the analysis. Finally 4 studies were excluded because the eye movement and no eye movements conditions were complicated by simultaneously assessing reaction time (Maxfield, Melnyk, & Hayman, 2008; van den Hout et al., 2011).
After the above exclusions, 24 studies remained containing 26 separate comparisons. Fourteen treatment studies (15 trials) compared EMDR treatment including eye movements with EMDR but without the eye movements. Ten laboratory studies (11 trials) compared eye movements with no eye movements while the respondents simply focused on an autobiographical memory.
2.2. EMDR treatment with eye movements versus EMDR without eye movements
2.2.1. Description of included studies
The 14 studies (15 comparisons) comparing eye movements versus no eye movement in full EMDR treatments, included a total of 452 respondents (239 in the EMDR conditions, and 213 in the no eye movement conditions). Selected characteristics are presented in Table 1. In six of the studies, all or most participants met criteria for a clinical diagnosis. In seven studies (eight trials) participants were students who reported various levels of distress. In one study students were used but screened for clinical levels of symptoms (Sanderson & Carpenter, 1992). Thirteen studies used self report of distress Subjective Units of Distress Scale (SUDS) as an outcome measure and five studies used additional measures relevant to the population group they were treating. For example the Body Sensations Questionnaire (BSQ) was used in assessing response to treatment for panic disorder (Feske & Goldstein, 1997), the Mississippi Scale for Combat-related PTSD (MSCR), the Impact of Events Scale (IES) or the Clinician-Administered PTSD Scale (CAPS) was used for people with PTSD (Boudewyns, Stwertka, Hyer, Albrecht, & Sperr, 1993) and the State Trait Anxiety Inventory (STAI) or the Beck Anxiety Inventory (BAI) was used to assess the levels of anxiety (Devilly, Spence, & Rapee, 1998; Renfrey & Spates, 1994).
2.2.2. Quality of included studies
None of the studies described an adequate sequence generation, and only one study reported adequate concealment of allocation to
Fig. 1. Consort flow diagram of meta-analysis study selection.
Study characteristics of investigations into EMDR with or without eye movements within a therapy context.
|Use of therapy manual||Outcome measures used in analysis|
|Boudewyns et al., 1993||16||PTSD||Clinical||+||—||—||CAPS, IES, SUDS, MSCRP|
|Carrigan & Levis, 1999||36||Public speaking anxiety||Students||—||—||—||SUDS|
|Devilly et al., 1998||25||PTSD||Clinical||+||+||+||SUDS, MSCRP, STAI|
|Dunn, Schwartz, Hatfield, & Wiegele, 1996||28||Distressing memory||Student||—||—||+||SUDS|
|Feske & Goldstein, 1997||36||Panic disorder||Clinical||+||+||+||ACQ, BAI, BSQ, FP, FPA, MI, PAI|
|Foley & Spates, 1995||20||Public speaking anxiety||Students||—||—||—||BASA, PRCA-24, PRPSA, SUDS, VOC|
|Gosselin & Matthews, 1995||42||Anxiety||Students||—||+||+||SUDS, VOC|
|Lee & Drummond, 2008 a||48||Distressing memory||Students||—||—||+||SUDS, vividness|
|Lytle, Hazlett-Stevens, & Borkovec, 2002||30||Distressing memory||Students||—||—||+||IES, STAI, SUDS, Vividness, VOC|
|Renfrey & Spates, 1994||15||PTSDb||Clinicalc||+||—||—||SUDS, VOC|
|Sanderson & Carpenter, 1992||62||Anxiety||Clinical||—||—||—||SUDS|
|Schubert et al., 2011||60||Distressing memory||Students||—||+||+||SUDS, VOC|
|Shapiro, 1989||22||Clinical trauma||Clinical||+||+||NA||SUDS, VOC|
|Wilson et al., 1996||12||PTSD||Clinical||+||+||+||SUDS, VOC|
Abbreviations ACQ: Agoraphobic Cognitions Questionnaire; FP: Fear of Panic; STAI: State-Trait Anxiety Inventory; FPA: Fear of Panic Attacks; MI: Mobility Inventory for Agoraphobia; PAI: Panic Appraisal Inventory; BASA: Behavioral Assessment of Speech Anxiety; PRCA-24: Personal Report of Communication Anxiety; PRPSA: Personal Report of Public Speaking Anxiety; VOC: Validity of Cognition. a Included 2 comparisons, one with reliving instructions, one distancing. b More than 90% met DSM criteria for PTSD.
c This was a student population, but they met criteria for a clinical population.
respondents. Because all outcome measures were self-report (apart from one measure in one study), blinding of assessors was not relevant. None of the studies described whether incomplete outcome data were handled adequately. In terms of treatment integrity, five of the 14 treatment studies did not use a treatment manual and only three checked the fidelity of the treatment. Therapists were untrained in one study and only fully trained in the procedure in six studies.
2.2.3. Effect sizes in the treatment studies
The results indicating the difference between eye movements and no eye movement in full EMDR treatments are presented in Table 2. The effect sizes and 95% confidence intervals of the individual studies are plotted in Fig. 2. The mean effect size indicating the difference between eye movements and no eye movement was Cohen’s d g = 0.41 (95% CI: 0.13—0.70), with moderate heterogeneity (I2 = 48.59).
Inspection of the funnel plot suggested that two studies were possible outliers, because their 95% confidence intervals fell outside the 95% confidence interval of the pooled effect size (Shapiro, 1989; Wilson et al., 1996). After removal of these two studies from the sample the mean effect size was 0.27 (95% CI: 0.07—0.47) with zero heterogeneity.
In our analyses, we included one study in which two separate psychological treatments were compared to the same control group (Lee & Drummond, 2008). This means that multiple comparisons from this study were included in the same analysis. These multiple comparisons, however, are not independent of each other, which may have resulted in an artificial reduction of heterogeneity and a distortion of the mean effect size. Therefore, we conducted another meta-analysis, in which we included only one comparison per study (Table 2). From the study with multiple comparisons we first included only the comparison with the largest effect size. We then conducted another meta-analysis in which we included only the smallest effect size from the study. As can be seen in Table 2, these analyses did not indicate that the mean effect size changed considerably, nor did we find indications that heterogeneity was affected by this study in either meta-analysis.
Neither the funnel plot nor Duval and Tweedie’s trim and fill procedure pointed at a significant publication bias. The effect size indicating the difference between the two conditions was only slightly smaller after adjustment for publication bias (0.35; 95% CI: 0.03—0.68; number of trimmed studies: 1), than the unadjusted effect size (0.41; 95% CI: 0.13—0.70).
2.2.4. Type of measures used
Given that Davidson and Parker (2001) reported SUDS values separately claiming that SUDS was a process measure and might be different to other outcome measures we looked at the effects of this variable separately. Only SUDS values reported after treatment was completed were used in the meta-analysis. The effect size indicating the difference between the two conditions using SUDS was moderate and significant (0.53; 95% CI: 0.20—0.85). As can be seen in Table 2, after removing the so-called process variables of SUDS and VOC the effect size for the difference between the two conditions was still significant (0.33; 95% CI: 0.07—0.060).
2.2.5. Subgroup analyses in the treatment studies
Meta-analyses of studies comparing the effects of eye movement with no eye movement during EMDR treatments.
|Effect sizes at post-test|
|Two possible outliers removedc||13||0.27||0.07—0.47||2.70**||0|
|One effect size per study (highest)d||14||0.43||0.13—0.73||2.83**||52.14*|
|One effect size per study (lowest)d||14||0.36||0.09—0.63||2.64**||42.47*|
|SUDS and VOC excluded||8||0.33||0.07—0.60||2.48*||0|
|DSM diagnosis||Yes||6||0.32||-0.15 to 0.79||1.34||61.12*||0.57|
|Aimed at posttraumatic stress||Yes||4||0.60||-0.05 to 1.25||1.80°||64.58*||0.58|
|No||5||-0.05||-0.43 to 0.34||-0.24||57.27|
|Fully trained EMDR therapist||Yes||6||0.70||0.28—1.11||3.25**||60.81*||0.09°|
|No||9||0.23||-0.11 to 0.58||1.33||22.01|
o: p < 0.10; *: p < 0.05; **: p < 0.01; ***: p < 0.001.
a The p-values in this column indicate whether the Q-statistic is significant (the 12 statistics does not include a test of significance). b The p-values in this column indicate whether the difference between the effect sizes in the subgroups is significant. c Wilson et al., 1996 and Shapiro et al., 1989. d In these analyses only one comparison from each study was used.
In order to examine the possible effect of moderators we conducted a series of subgroup analyses (Table 2). We found no indication for a significant difference between studies with clinical populations and those with student populations, between studies in which participants met diagnostic criteria for a mental disorder versus other studies, between studies which were aimed at posttraumatic stress and those aimed at other anxieties. For the subgroup analysis using variables associated with treatment fidelity we found a significant effect for whether or not the paper cited the used an EMDR treatment manual. The effect size for studies that used a manual was significantly greater than zero whereas the effect size was not significantly greater than zero for those that did not use a treatment manual (see Table 2). There was also a trend (p < 0.1) indicating that the effect sizes in studies in which the therapies were delivered by fully trained EMDR therapists were larger than the effect sizes found in other studies.
|Study name||Outcome||Statistics for each study|
|Lee, 2008 Distancing Combined||1.31||0.43||2.20||2.91||0.00|
|Lee, 2008 Reliving||Combined||0.47||-0.34||1.28||1.13||0.26|
|Schubert et al., 2011 Combined||0.44||-0.10||0.98||1.60||0.11|
5td diff in means and 95% Cl
Fig. 2. Eye movement versus no eye movement in full EMDR treatments: Standardized effect sizes.
2.3. Eye movements versus No eye movement in laboratory studies
Ten studies (11 comparisons) tested eye movements versus no eye movement while the person focused on an emotional autobiographical memory. A total of 397 participants participated in these studies with 200 in the eye movements and 197 in the no eye movement condition (see Table 3). These results are presented in Fig. 3 and Table 4. As can be seen, the mean effect size of all studies was 0.74 (95% Cl: 0.57—0.91) with low, and non-significant heterogeneity (I2 = 12.15). Several of the included studies used the same instruments to measure the effects of the interventions (in Subjective Units of Change). Therefore, we were able to calculate separate effect sizes for Subjective Units of Change in emotion associated with the memory and Subjective Units of Change in vividness of the memory. As can be seen in Table 4, the mean effect size of emotion was 0.66 (95% Cl: 0.46—0.85) with low heterogeneity, and for vividness it was 0.91 (95% Cl: 0.65—1.16) with moderate heterogeneity. Because the number of studies was small, we did not conduct subgroup analyses.
The present meta-analysis provided an up-to-date evaluation of the efficacy of eye movements in processing emotional memories. The 14 studies that investigated the additional value of eye movements in EMDR treatment averaged a significant medium effect size advantage for eye movements over no eye movement. Heterogeneity was found to be moderate in these analyses, and this was reduced to zero after removal of two possible outliers. In 10 laboratory studies that looked at the effects of eye movements in a non-therapy context, a significant medium to large effect size advantage was found for eye movements, with low heterogeneity.
The results of this study are at odds with a previous metaanalysis (Davidson & Parker, 2001) which found no significant advantage for eye movements. However, in Davidson and Parker’s analysis an adjustment for sample size was not made before calculating the average effect size. Furthermore, a fixed effects model was used rather than a random effects model, but given the heterogeneity of the studies a random effects model would have been more appropriate. These differences in methodology and the inclusion of 12 more recent studies appear to account for the differences in the findings of the two studies.
Similarly another earlier meta-analysis also failed to use a random effects model (Devilly, 2002). In addition a single rater selected the studies and calculated the effect sizes which increases bias, particularly given unpublished studies were included. Over the years there has been agreement that meta-analysis should involve multiple raters (Bullock & Sysvyantek, 1985; Stroup et al., 2000). The issue of possible publication bias in the current analysis was examined with funnel plot and Duval and Tweedie’s trim and fill procedure. Neither indicated a significant publication bias.
Davidson and Parker reported results separately for SUDS and VOC which they called process measures and they named the other measures ‘outcome measures’. In the studies examined in this study the largest effect was found for the VOC scale and then the SUDS measure. However even after excluding these measures,
Study characteristics of investigations into memory processing with or without eye movements not in a therapy context.
|Study||Number||Type of memory||Measures|
|Andrade et al., 1997||48||Distressing memory||Subjective distress, vividness|
|Barrowcliff et al., 2004||80||Distressing memory||Subjective distress, vividness|
|Christman, Garvey, Propper, & Phaneuf, 2003||40||Autobiographical memories||Accuracy of memory recall|
|Gunter & Bodner, 2008||36||Distressing memory||Subjective distress, vividness, completeness|
|Kavanagh et al., 2001||18||Distressing memory||Subjective distress, vividness|
|Kemp 2007||30||Distressing memory||Subjective distress, vividness|
|Kristjansdottir & Lee, 2011||36||Distressing memory||Subjective distress, vividness|
|Lilley, Andrade, Turpin, Sabin-Farrell, & Holmes, 2009||18||Distressing memory||Subjective distress, vividness|
|Smeets et al., 2012||61||Distressing memory||Subjective distress, vividness|
|van den Hout et al., 2001||30||Distressing memory||Subjective distress, vividness|
|Study name||Outcome||Statistics for each study|
|Andrade etal.. 1997||Combined||0.86||0.45||1.28||4.05||0.00|
|Barrowcliff et al., 2004||Combined||0.27||-0.17||0.71||1.21||0.23|
|Christman et at.. 2003||accuracy||0.54||-0.09||1.17||1.69||0.09|
|gunter & В exp1 2008||Combined||1.33||0.61||2.06||3.59||0.00|
|gunter & В exp2 2008||Combined||0.86||0.18||1.55||2.47||0.01|
|Kavanagh et al., 2001||Combined||1.05||0.36||1.75||2.96||0.00|
|Kemps et al.. 2007||Combined||089||0.36||1.43||3.30||0.00|
|Kristjansdottir and Lee 2011 Combined||0.97||0.56||1.38||4.66||0.00|
|Lilley et at. 2009||Combined||055||0.05||1.05||2.18||0.03|
|smeets et al2012||Combined||0.52||0.01||1.03||1.99||0.05|
|Van den Houtet al.,2001||Combined||0.69||0.17||1.21||2.59||0.01|
Std diff in means and 95% Cl
Fig. 3. Eye movement versus no eye movement in brief experiments: Standardized effect sizes.
there was still significant effect for eye movement. It can be argued that SUDS is both an outcome and process measure. In trauma focused cognitive behaviour therapy, SUDS is used during the session to assess how the habituation process is proceeding and to help ascertain ‘hot spots’ which are the subject of further attention by the therapist. However SUDS can also be an outcome measure. At the conclusion of treatment if this process is successful then there should be no hot spots. SUDS is also used in EMDR to check the current degree of distress to the memory. An important outcome of any PTSD treatment is that recovery should be evident by reduced frequency of avoidance and intrusive symptoms and that when a person is reminded of the trauma that the memory it is not accompanied by hyperarousal. SUDS recorded at the end of treatment (as used in the current analysis) can help assess this and is therefore also an outcome measure.
The finding of a significant effect for eye movements in both treatment and laboratory contexts is important in terms of understanding the underlying active processes in EMDR. One account for the effect of eye movements is provided by working memory theories of EMDR (Andrade et al., 1997; Gunter & Bodner, 2008; Maxfield et al., 2008; van den Hout et al., 2011). Researchers have noted that emotional memories tend to have an episodic form and are rich in sensory detail, and trauma recovery is likely to occur when these memories lose their sensory richness (Stickgold, 2002). Consistent with hypotheses from a working memory theory, holding an emotional memory in mind and performing another task such as eye movements disrupts the storage of this information and the episodic quality is reduced. Therefore the finding of a large effect size in the non-therapy studies for the specific measure of vividness is consistent with this working memory theory to explain treatment effects in EMDR. Another finding consistent with this model is that other complex visuospatial tasks can also produce a reduction in vividness and emotionality (Gunter & Bodner, 2008), although this is not always found (Kavanagh et al., 2001).
Meta-analyses of studies comparing the effects of eye movement versus no eye movement in brief experiments at post-test.
*: p < 0.05.
Another model to account for the possible role of eye movements that has some empirical support is that the eye movements elicit an orienting response (Barrowcliff, Gray, MacCulloch, Freeman, & MacCulloch, 2003; Sack, Lempa, Steinmetz, Lamprecht, & Hofmann, 2008; Schubert, Lee, & Drummond, 2011). According to orienting response theory the eye movements activate an «investigatory reflex” in which first, an alert response occurs, then, a reflexive pause produces dearousal in the face of no threat. This reflex results in a state of heightened alertness and permits exploratory behaviour in which cognitive processes become more flexible and efficient (Kuiken et al., 2001). Some physiological changes associated with the eye movements do fit with the orienting response hypothesis such as changes in skin conductance and heart rate (Elofsson et al., 2008; Sack et al., 2008; Schubert et al., 2011). However other changes during EMDR treatment sessions are not consistent with an orienting response such as an increase in respiration (Schubert et al., 2011).
Whilst the effect of eye movements in the non-therapy studies might be accounted for by a working memory model or by the eye movements triggering an orienting response, the key processes in the therapy studies are likely to be more complex. In the nontherapy studies the amount of exposure to eye moment was always a single session and lasted between 8 and 96 s. In contrast, in the treatment studies the eye movements or no eye movement period involved one to several sessions and most studies included many phases of the EMDR protocol. EMDR has been described as a complex procedure and that even without eye movements involves processes such as mindfulness to the trauma (Lee et al., 2006), cognitive restructuring, an increased sense of personal mastery, and other processes associated with exposure that would create a therapeutic benefit (Solomon & Shapiro, 2008). Thus when the effects of eye movements in an EMDR therapy context are assessed they have to provide additional value to these other processes. Thus in comparison to the effect size difference in the non-therapy studies it is not surprising that the effect was less pronounced (moderate) and the heterogeneity greater. In the nontherapy studies, these other elements are absent so the comparison is not measuring the additive value of eye movements to other useful processes but a more direct assessment of its value.
Some of the data indicated that the additional effects of the eye movements may depend on the quality of the treatment delivery. The effect size for studies that cited use of an EMDR treatment manual was higher than the effect size in studies that did not cite use of a treatment manual. This is consistent with a previous metaanalysis that found a significant correlation between effect size and
treatment fidelity (Maxfield & Hyer, 2002). There was also a trend indicating that the difference between EMDR with and EMDR without eye movements was larger when the treatment was delivered by a fully trained EMDR therapist. However, because this difference was not significant at p < 0.05, and because the number of studies was very small, such interpretations have to be considered with caution.
This study has several limitations. The most important one is that the quality of included studies was not optimal. This may have distorted the outcomes of the studies and our meta-analysis. Apart from ensuring adequate checks on treatment quality, there were other serious methodological problems with the studies in the therapy context. None of the studies described an adequate sequence generation, and only one study reported adequate concealment of allocation to respondents. There was an over reliance on self report measures and, in general, each study had an insufficient sample size to detect significant differences. In addition many of the laboratory studies included a within subjects design which can produce carry over effects. Furthermore, the total number of studies was small, especially the number of studies on brief experiments. This limited the possibilities to examine possible moderating variables. It also restricted some subgroup analysis. The total number of treatment studies that investigated the effect of eye movement and where participants had a DSM diagnosis was only 6. However the effect size of the difference between the conditions was moderate and significant.
Another possible limitation of this study is that we used standard methods to calculate the confidence intervals around our effect sizes. There are indications, however, that alternative methods to calculate confidence intervals are somewhat more conservative (Sanchez-Meca & Marin Martinez, 2008; Viechtbauer & Cheung, 2010). On the other hand, the effects of these alternative methods on the confidence intervals have been found to be small (Viechtbauer & Cheung, 2010), and probably would not have led to very different outcomes.
Despite these limitations, it seems safe to conclude that the eye movements do have an additional value in EMDR treatments. There remains a need for research to be conducted on clinical populations with adequate attention to treatment fidelity and the above methodological issues. However the results from the studies to date suggest that eye movements do alter the processing of emotional memories.
EMDR: Why the Controversy?
Charlotte Sikes and Victoria Sikes____________________________________________
Eye-movement Desensitization and Reprocessing (EMDR) has been widely supported in the literature for its effectiveness in treating Post-traumatic Stress Disorder (PTSD) and a variety of other diagnoses and symptoms. The variable findings regarding whether its effects are for reasons unique to this treatment, however, have become the focus of extensive discussion and debate. The following article reviews the studies targeting this question, and proceeds to consider why these studies’ findings, and other findings in the EMDR literature, vary so vastly. Implications of the EMDR controversy for the process of psychological research at large are considered.
Key Words: Eye Movement Desensitization and Reprocessing, EMDR, PTSD,
Today, just over ten years after its birth, Eye Movement Desensitization and Reprocessing (EMDR) treatment still has not shed the controversy surrounding it. The attention it has gained from the field of psychology at large comes not only from the often dramatic and immediate positive results demonstrated by this treatment in many empirical outcome studies, but from the fact that not all studies replicate these findings. In addition, it is no longer used only to treat Post-traumatic Stress Disorder (PTSD), but also Panic Disorder, Obsessive-Compulsive Disorder, Dissociative Disorders, and Dysmorphic Body Disorder, as well as depression, grief, addictions, eating disorders, body image disturbances, issues of self-esteem, morbid jealousy, chronic pain, test anxiety, personality disorders, public-speaking anxiety, somatoform disorders, substance abuse, and some aspects of Attention-Deficit/Hyperactivity Disorder (processing condition-related distressing experiences and learning self-management skills) (Greenwald, 1996; Herbert et al., 2000; MacCluskie, 1998; Parnell, 1997; Shapiro, 1999, 2002).
Studies demonstrating the outcomes of EMDR in treating PTSD and trauma or memory-related symptoms are now so abundant and positive that the general effectiveness of EMDR no longer seems the issue in debate. The current controversy about EMDR treatment seems to hover around two other questions. First, is EMDR effective for the same reasons as are other PTSD treatments (therapeutic alliance, exposure, etc.)? Second, why are there inconsistencies in research findings on this treatment? This article begins by reviewing some of the studies attempting to answer the first question, then looks at how opponents and proponents of EMDR answer the second question. Though perhaps ideally the authors of the articles that make up the EMDR literature are proponents of science alone, and they may be, it seems acceptable to refer to EMDR as having “opponents” and “proponents,” as many of the same authors repeatedly find either positive or negative results and critique those who find the opposite (Lohr,
Tolin, & Lilienfeld, 1998; Lohr, Lilienfeld, Tolin, & Herbert, 1999; Shapiro, 1999, 2002). Ultimately, the persistence of controversy over such a long time, and after methodological concerns about the research have been addressed in the more recent studies, raises questions about how research is conducted in psychology in general, and about the process by which a PTSD treatment becomes empirically supported.
Is EMDR effective for unique reasons?
It is apparent how much of the EMDR protocol draws on techniques already used by psychologists. Attending to client history, making treatment plans, building a therapeutic alliance, and conceptualizing the case based on symptoms, behaviors, and history are all part of the EMDR protocol and of many traditional techniques as well. Emotional and physiological responses to memory also are addressed in the EMDR treatment. In addition, education in the therapeutic process and a component of therapeutic expectation appear to play a role in treatment. Perhaps a more obvious aspect of EMDR that also plays a role in existing techniques is cognitive reprocessing. EMDR includes a cognitive component which helps clients identify negative cognitions associated with the upsetting event targeted, and, after processing those, helps them identify positive cognitions and pair them with the event. This process clearly draws on cognitive techniques that could arguably be effective in themselves. Perhaps most central to the current debate over whether EMDR is effective for unique reasons is its inclusion of exposure to the traumatic event. Exposure, a primary component of the EMDR desensitization process, leads many to wonder whether exposure, the effective component of exposure therapies for PTSD, is not the effective component of EMDR as well.
Lohr et al. (1999) comment: “Had EMDR been put forth simply as another variant of extant treatments, we suspect that much of the controversy over its efficacy and mechanisms of action could have been avoided” (201). Although it is readily apparent that components which are effective in some other treatments also play a role in EMDR, Shapiro assumedly includes those elements in the protocol for that exact reason. Still, Rogers and Silver (2002) claim: “the structure of EMDR is unlike that of flooding/implosion, systematic desensitization, or cognitive therapy” and describes the vast differences in process and target experience between exposure techniques and EMDR (56). One way to answer the question of whether or not EMDR is effective for unique reasons is to compare its outcomes to treatments whose techniques overlap those of EMDR. The approach more often used, however, has been to take the main component unique to EMDR, bilateral eye movements or another form of bilateral stimulation, and study treatment effects with variations of, and in absence of, this component.
Both Montgomery and Ayllon (1994) and Wilson et al. (1996) found that eye movements added significantly to positive treatment outcomes. Without saccadic eye
movements (or any other bilateral stimulation), EMDR did not lead to a decrease in the subjective distress of the clients, but it did do so with them (Wilson et al., 1996). Wilson et al. (1996) also showed that those in an EMDR treatment group had greater change in relaxation (measured by respiration, galvanic skin response, systolic blood pressure, heart rate, and finger tip temperature) than those in a time interval condition involving exposure, or than those in a tapping alternate phalanges EMDR-variant condition. In the former group 64.7% of participants wept during treatment, while 8.3% of participants in the latter two groups did. Montgomery and Ayllon (1994) found that adding eye movements increased the improvement made by clients over that of repeated exposure and cognitive restructuring without eye movements.
Renfrey and Spates (1994) found that the EMDR protocol with the standard eye movements, with eye movements engendered by a light tracking device, and with fixed visual attention without eye movements, all led to significant positive change. The three conditions were not significantly different, although the speed of improvement for those with standard eye movements was almost significantly (p<. 06) faster than the EMDR condition with fixed eye position. Sharpley, Montgomery, & Scalzo (1996) compared the effectiveness of EMDR-style eye-movements in reducing the vividness of a memory with both a relaxation procedure in which participants were instructed to roll their eyes up instead of move them from side to side, and another relaxation procedure designed to decrease the intensity of traumatic images without eye movements. The first two procedures, involving the eyes, were significantly effective in decreasing the vividness of the traumatic image, as measured by the Vividness of Image Scale (VOI), a self-report scale, while the simple relaxation method was not. The EMDR condition, however, was significantly more effective in lowering VOI than either of the other two conditions. Similarly, Van den Hout, Muris, Salemink, & Kindt (2001) asked participants to recall either a positive or negative memory and rate it as to vividness and emotionality. After eye movements parallel to those used in EMDR, vividness ratings decreased, and emotionality behind memories moved closer to neutral (both positive and negative feelings became less intense). This finding was not present in either a control or a fingertapping condition, suggesting that eye-movement is the effective component in reducing the vividness of images.
In contrast, Dunn, Schwartz, Hatfield, & Weigele (1996) found no significant difference in outcome of distress ratings, frontalis EMG, finger skin temperature, Skin Conductance Level (SCL), or heart rate (HR) between those with a traumatic memory treated with EMDR and those treated with a fixed-eye version of EMDR. Other studies also found no significant differences in outcomes between EMDR and the EMDR protocol without eye-movements (Devilly, Spence, & Rapee, 1998; Pitman, Orr, Altman, Longpre, Poire, & Macklin, 1996). In addition, Boudewyns and Hyer (1996) found that after eight sessions those with combat-related PTSD in both the EMDR and an imagery exposure control group showed significantly lower distress ratings, anxiety, and heart rate levels than those in a no-imagery control group, but that the differences between the former two groups were not significant. Isolating a different aspect of EMDR, Cusack &
Spates (1999) found that an EMDR group versus an EMDR group without the cognitive components of the treatment, both led to significant reduction in participants’ PTSD symptoms, but not differentially so, suggesting EMDR’s effectiveness is not based on its cognitive component.
Much of the research to discover whether the bilateral stimulation in EMDR is the unique ingredient compares treatment with eye movements to treatment with another form of stimulation. These other forms of stimulation are often ones now utilized by EMDR therapists as viable alternatives to the eye movements (such as bilateral tapping or audio tones). Although Shapiro originally thought that eye-movements specifically were the active component of the treatment, she has since reported that EMDR is “an integrated form of therapy incorporating aspects of many traditional psychological orientations…[while making] use of a variety of bilateral stimuli besides eye movements” (Shapiro, 1999). Studies that find no difference between the eye movement condition and another bilateral stimuli condition, then, do not refute the theory as it currently stands. However, some of the studies comparing eye movements with other alternating stimuli do find significant outcome differences, further confusing the matter (Wilson et al., 1996).
Nonetheless, newer studies, as presented, have compared EMDR treatments with and without bilateral stimulation, providing some means of evaluating whether bilateral stimulation in general is the effective part of EMDR. If, however, it is the added layer of stimulation, not even needing to be bilateral, that is the effective component, then studies finding no significant difference between bilateral stimulation and some non-bilateral form of stimulation may not show that EMDR’s stimulation condition is not its effective component. This idea is consistent with the theory that adding a distraction that changes the experience or facilitates an observer’s perspective is behind the outcomes found in EMDR treatment. Or, to confuse matters further, Pitman et al. (1996) claim that even a fixed eye condition could be bilaterally stimulating, as “the optic nerve is crossed to both hemispheres so maintaining the focus demands bilateral muscle stimulation to hold the gaze” (957).
MacCluskie (1998) puts forth an interesting proposal to account for varied findings in studies comparing different forms of stimulation: “Perhaps the most salient sensory memories of the trauma are most responsive to re-conditioning if the distracter is presented in the same sensory modality as the most intense and disturbing aspect of the memory” (127). Critics express frustration about the evolution of the EMDR theory because it makes disproving its having unique effective components difficult. They raise the issue that in the field of psychology in general there is a necessity of putting forth theories in a way in which they can be empirically refuted, possibly requiring theory to proceed method. Although this concern appears valid, the issue is complicated by the fact that it seems important both to explore a technique which seems effective, even when the theory behind it is lacking or nonexistent, as well as to create and adjust the theory as empirical data is collected.
A particularly interesting technique with which to compare EMDR is exposure therapy because critics often claim EMDR is a variant of this treatment: that its effects are based on its exposure component. A few central differences in technique between exposure therapy and EMDR are that EMDR is more associative than directive and that it focuses on one trigger to the traumatic memory network (Rogers & Silver, 2002). It also includes brief exposures (Rogers & Silver, 2002). Exposure therapy, in contrast, is highly directive and moves chronologically through an entire traumatic memory with the client, utilizing prolonged exposure (Rogers & Silver, 2002). The following groupings of studies, one on EMDR and the other on exposure therapy, use similar measures to calculate effect. Characteristic of the sometimes dramatic effects of EMDR, four of five recent controlled studies of EMDR “with civilian populations found that 77-100% of the single-trauma victims no longer met diagnostic criteria for PTSD after 3-6 hours of treatment” (Shapiro, 2002, 4). In comparison, the diagnosis of PTSD was no longer present in only 55% of clients after about 25 hours of exposure therapy (Foa, Olasov Rothbaum, Riggs, & Murdock, 1991), and in 80% of clients after 50 hours (Richards, Lovell, & Marks, 1994), or 100 hours (Marks, Lovell, Noshirvani, Livanou, & Thrasher, 1998) of exposure therapy. Van Etten & Taylor (1998) conducted a more direct comparison of the two treatments in terms of the time they took to relieve symptoms. They found that symptoms are relieved in three to six sessions with EMDR and fourteen sessions in exposure therapy.
In contrast to this large apparent advantage of EMDR over exposure therapy in treating PTSD, a recent meta-analysis by Davidson and Parker (2001), reviewing 34 published studies, did not show a significantly greater over-all effect size for EMDR compared to exposure therapies. This analysis, however, including “all the studies in the literature, with the exception of those that have fatal methodological flaws,” and six dissertation studies with unpublished results, did find EMDR significantly more effective than no treatment or a grouping of other treatments, including rapid induction and relaxation, biofeedback-assisted relaxation, active listening, “routine” individual treatment, and applied muscle relaxation.
What accounts for inconsistent research findings?
Proponents and critics of EMDR answer this question differently. Critics ridicule proponents for explaining away less-positive outcomes, while they do the same thing with dramatically positive outcomes. Proponents point to lack of treatment fidelity and short treatment length to explain studies not demonstrating positive effects, while critics point to the effective components of other treatments incorporated into EMDR and the use of self-report measures to explain the outstanding positive findings in other studies.
Taking the proponents’ side first, Shapiro claims that the full EMDR protocol, complete with all its phases, is not always employed by research conductors. One must evaluate whether a given study actually utilized the treatment of EMDR or some inadequate substitute. Although critics argue that much of the research uses therapists trained in EMDR, training does not assure execution of the full treatment protocol. A study investigating the relationship between treatment fidelity and outcome supported Shapiro’s claim (Pitman, Orr, Altman, Longpre, Poire, & Macklin, 1996). It found a significant positive correlation between the two, indicating that adherence to treatment protocols corresponded with better observed treatment outcome for EMDR. Similarly, Maxfield and Hyer (2002) found a significant correlation between effect size and treatment fidelity, and that more rigorous methodological studies according to the Gold Standard (GS) scale (Foa & Meadows, 1997, as cited by Maxfield & Hyer, 2002) yielded a significantly larger effect size. Greenwald (1996) explains that the “EMDR protocol refers both to the basic, standard procedure and to an expanded repertoire of intervention options guided by a set of decision-making principles” (69).
Shapiro (1999) analyzed the effect of treatment adherence on study outcome by examining a number of studies on the effectiveness of EMDR in treating phobias. Trained EMDR therapists, blind to the results of the studies, rated the methods sections of the studies according to their treatment fidelity in terms of the number of steps in the EMDR protocol actually utilized in the study. Studies using five or more found that subjects’ initial complaints were completely relieved. With less than five of the steps present, clients experienced partial remission. If no steps were adequately applied, the study found no effect of EMDR in treating phobias. Shapiro also identifies a number of other ways (besides not administering every step of the protocol) that treatment protocols in many studies of EMDR are violated (Shapiro, 1999).
Some studies, proponents claim, include too few sessions to demonstrate the effectiveness of EMDR. Concerning subjects, such as combat veterans, having experienced multiple traumatic events, more sessions may be needed to find adequate results than are often provided in research studies (Greenwald, 1996; Shapiro, 1999; Cahill, Carrigan, & Frueh, 1999). A study using a large number of EMDR sessions (12) with combat veterans found that 75% of the subjects no longer met the criteria for PTSD nine months after treatment (Carlson, Chemtob, Rusnak, Hedlund, & Muraoka, 1998). This finding exceeds the success shown by any other PTSD treatment for this often severely-traumatized population (Shapiro, 1999).
Still, the argument that more sessions are needed to find the above results, is contradicted by much of the initial and on-going research showing dramatic improvement even with severe PTSD clients in only a few sessions. One case study of a 46-year-old Vietnam veteran, for instance, reported that in one 60-minute session of EMDR, a “dramatic resolution of 2 traumatic memories” occurred. Nine months later the effects were maintained and the client reported no “intrusive images” or “fearfulness” (Young, 1995, 282). In addition to treatment length, it is difficult to imagine that all phases of treatment could be fully adhered to in the time of one session, though it feasibly could be within three or four sessions (the length of some other studies showing this kind of dramatic effect, such as Rothbaum (1997) and Wilson et al. (1995, 1997). It would seem that treatment protocol adherence and treatment length should be considered in pursuing an answer to the question of why EMDR research results are inconsistent. In addition to a consistent EMDR protocol, consistent administration of protocols for cognitive- behavioral interventions used in comparative studies might also decrease the variance in findings of comparative efficacy found in these studies (Shapiro, 2002).
Critics refer to EMDR’s over-lap in technique with other treatments as an explanation for its positive effects, an idea which has been both refuted and supported by the literature discussed (both dismantling studies and comparisons of EMDR with other exposure techniques). Regarding the conflicting findings previously discussed as to what make up the effective components of EMDR, Shapiro notes the importance of using participants predicted to be responsive to the treatment, such as those with a single trauma, so that which components are effective can be more consistently demonstrated (Shapiro, 2002).
Another explanation raised by critics for the extreme positive effects in many studies, and therefore for the gap between findings, is that many of the most positive outcomes demonstrated rely heavily on self-report measures (Lohr, Tolin, & Lilienfeld, 1998). Since this concern has been raised, however, many other outcome measures, such as collecting an array of physiological data and administering standardized instruments, have been used in studies reporting EMDR as comparatively more effective than other treatments (Levin, Lazrove, & Van Der Kolk, 1999; Rogers, Silver, Goss, Obenchain, Willis & Whiteney, 1999). In addition, self-report indices are also used in studies of other treatments, particularly in studies of exposure therapy. Adding a twist to the issue, MacCluskie (1998) claims that a client’s reaction to a treatment and report on its efficacy is no small matter: “perhaps clients’ self-reported [distress] reduction,” greater than that found using other treatments, “is ample justification to use the EMDR technique” ( 130). Still, both the effective components of a treatment and the kind of outcome measures used should be considered in evaluating positive research outcomes.
These conflicting perspectives raise issues about research and empirical validation, providing the opportunity to define our expectations for treatment research. Herbert et al. (2000) list some of the non treatment-specific factors that often affect outcome and make treatment research so complicated: “treatment credibility, expectation for improvement, experimental demand, therapist-experimenter enthusiasm, therapist- experimenter allegiance, [and] effort justification” ( 950). The fervor expressed in the literature from writers, both critics and proponents, seems to reflect unusual personal investment in one side of the debate or the other, possibly increasing the likelihood that unintentional bias in findings and interpretations also plays a role in varied research findings. Perhaps these extreme perspectives are partially explained by the radically different ways individuals respond to novelty, considering the high level of both positive and negative energy directed toward the question of the uniqueness of EMDR process and theory (Perkins and Rouanzoin, 2002). Foa, Davidson, and Frances (2000), for instance, point out: “History and the philosophy of science tell us that new approaches often receive widespread opposition from those wedded to previous approaches” (785).
An example of the kind of biting and unfounded criticisms that can be found in the EMDR literature is Herbert et al.’s (2000) statement about Shapiro’s names for concepts in her theory (such as “cognitive interweave” and “saccade sets”). He accuses her of using “obscurantist language to compensate for an absence of content and to discourage would-be skeptics” (960). In a less extreme example, Muris and Merckelbach (1999), in a criticism of Shapiro for discovering EMDR by chance rather than from a preconceived theory, use the term “lively” to describe Shapiro’s description of EMDR’s discovery. The scientific context which makes the term “lively” appear condescending highlights the seemingly unfounded description of EMDR by some opponents as “pseudoscience” (Herbert et al., 2000). Instances such as these make believable Greenwald’s (1996) comment that EMDR proponents “regularly” receive “overt bias and downright rudeness” in “otherwise civilized professional settings” (306).
Some critics of EMDR (Herbert et al., 2000), however, claim that it is the proponents of EMDR, who bring the “personal arena of ad hominem assault” into the “theoretical and empirical issues at hand” (960). They claim proponents have “attacked” critics, questioning “their professional training, ulterior motives, and competence” (Herbert et al., 2000). Still, it appears ironic that the bulk of that article is aimed at showing EMDR to be “pseudoscience,” which seems to do what they criticize the proponents for doing (questioning the competence of their colleagues in the field). An example of the strong language that is apparent in the proponents’ literature as well, however, is Greenwald’s (1996) criticism of Van Ommeren (1996), stating that he “apparently chose to ignore the bulk of the research, which supports EMDR’s efficacy, to focus on a single clearly flawed study and on a relatively tangential and unresolvable problem (method-neutral fidelity raters)” (306).
Because ten years of controversy around EMDR does not appear to have accounted for the gaps in findings, it seems important that we allow this debate to raise questions about how treatments are validated and to remind us of the need for caution in interpreting research findings in general. Discovering why outcomes vary in the body of EMDR research may reveal important lessons about how to increase research precision in
the future. It may still be proven, however, that the reason for the variability in findings is unique to this body of research. MacCluskie (1998) suggests: “EMDR may elicit differential treatment effects based on idiosyncratic client variables that have yet to be identified” (128). This factor, uncontrolled for and affecting studies nonrandomly, could lie behind some of the conflicting findings. If this hypothesis is correct, we learn a critical lesson that applies psychological research at large: not to underestimate the effect of individual differences in participants in research, and in people to whom treatments are administered.
As EMDR grows in popularity and in the number of treatment situations in which it is applied, the debate about its effective components may continue. Perhaps it will be soon, however, when research in this area may begin to be consistent and trusted, alleviating some of the controversy that now surrounds it: “The quality of research on EMDR has improved enormously over the years. Most of the recently-published studies, but certainly not all, have addressed many of the weaknesses identified by critics of EMDR” (Cahill, Carrigan, & Freuh, 1999, 25). To the extent that initial concerns (treatment fidelity, large random samples, comparisons with existing treatments, incorporation of non-self-report measures, attention to the number of sessions necessary, etc.) have been addressed already and that disagreement remains in the literature, researchers must develop other hypotheses to explain conflicting findings in EMDR research.
As suggested previously, perhaps there is yet some factor playing into the conflicting findings which has not been identified. The fact that our most conservative methodological considerations have not succeeded in ending the EMDR controversy and that empirical findings vary so widely, raises questions at large about whether our research methods are sophisticated enough for use with the material to be studied and how much trust can be put in findings when it comes to something as complex as psychological healing. Furthermore, researchers must attend ever more carefully to the role of individual differences in participants when studying treatments and look persistently for factors affecting person by person outcomes, rather than averaging over large numbers of participants? At present, clinicians are encouraged to continue to evaluate EMDR as a promising treatment, while also considering questions about psychological research in general brought to light by the EMDR controversy.