What's Wrong with the Rorschach? (Book Review)

Author:  Wood, James M., Nezworski, M. Tereza, Lillienfeld, Scott O., and Garb, Howard
Publisher: San Francisco: Jossey-Bass, 2003
Reviewed By: Marvin Acklin, Winter 2005, pp. 78-79

Readers of this book-length criticism of the Rorschach Test and, more pointedly, the Rorschach Comprehensive System (CS) developed by John Exner, may have been alerted by controversial articles appearing in professional journals, popular media outlets (e.g., New York Times Magazine, Scientific American) and symposia at professional meetings over the past several years. Wood, Nezworski, Lillienfeld, and Garb have developed a peculiar fixation on the Rorschach Test, adopting an unrelentingly skeptical posture that equates use of the test with fortune telling, claims of the paranormal, and junk science (what they refer to as “widespread quackery,” see also Lohr, Fowler, & Lillienfeld, 2002). The authors characterize Rorschach developers as “wizards” whose “tricks” were designed to defraud and mislead the gullible. They insinuate that Rorschach clinicians are members of a cult-like religious movement whose motivations are mercenary, or worse. A review of their output suggests that this fixation has come to characterize a significant portion of their scholarly interest. This overdetermined fixation on the Rorschach Test seems curious. One wonders why they haven’t focused on widespread unsubstantiated assessment methods and practices that have profoundly negative effects, for example, polygraph or plethysmographic testing, efficacy of anger management, so-called attachment therapies, or diagnostic mammography.

What’s Wrong with the Rorschach? is an informative critical biography and cultural history of the Rorschach Test’s development, advent in the United States, dissemination during the glory days of the ‘40’s and ‘50’s, period of apparent decline during the ‘60s, and resurgence with the advent of the Comprehensive System. In contrast to most books on the Rorschach, the authors take a less than adoring view of Rorschach “heroes.” What’s Wrong with the Rorschach? is not exactly Rorschach hagiography. In fact, the authors debunk the cult of Rorschach “great men,” and dispel many of the bombastic claims made about the test over the decades. They pick out certain of the Rorschach “great men” for particular lambasting, for example, Bruno Klopfer and John Exner. Marguerite Hertz, on the other hand, is lauded throughout, based on the rigor of her dogged insistence on grounding the test scientifically.

Much of this material is informative, some of it amusing. At many points, the author’s arguments are incisive and critical. They point out a number of factors that well-informed Rorschach clinicians have been aware of for years. Some of their criticisms deserve serious attention from the Rorschach establishment, for example, the weak scientific database derived from Exner’s “in house” research, Exner’s disinclination over the years to include or recruit the work of researchers outside of his establishment, the problem of “R” (why the Rorschach community didn’t adopt a fixed R a long time ago as did the Holzman Inkblot Test has long been a mystery to me), the mediocre performance of DEPI or some of the other CS indices, the recent “overpathologizing” debate, and the embarrassing issue of the duplication of a significant number of records in the so-called “normative data.” (I have known John Exner since the late ‘70s when I took my first Rorschach workshop. Exner resisted calling these samples of outpatient, inpatient, depressed, etc., subjects “normative.” He referred to them as “reference data”). To their credit, Wood, et al.’s criticisms of the Rorschach Test have stimulated a flood of Rorschach research in refereed journals examining both clinical and methodological issues. Overall, the author’s effort to convince the informed reader of the Rorschach Test’s fundamental bankruptcy fails given their selective review of the scientific evidence (MacCoun, 1998) and undisguised polemical intent. Despite the book’s easygoing, somewhat avuncular tone, it becomes clear quite early on that a dispassionate assessment of the Rorschach Test is not the author’s intent. What’s Wrong with the Rorschach? is popular, albeit well-written polemical screed, not a scientific treatise. The authors thoughtfully included the undersigned as one of the “supporters and followers” of their work.

They make one specific and two general errors that deserve comment. They take me (along with my former student, Greg Meyer, current editor of the Journal of Personality Assessment) to task on p. 232 and again on p. 266 for asserting that “Rorschach scoring reliability of .61 is “good” and that reliability of .74 is “excellent,” disingenuously, perhaps, failing to remind the reader that standards of what constitutes adequate reliability depend on whether one is referring to an intra-class correlation or a kappa coefficient. It is generally accepted that kappa coefficients .61 and above do indeed reflect “good” reliability (Acklin, McDowell, Verschell, & Chan, 2000). The second issue concerns the author’s failure, despite having the opportunity of an interested reader’s attention for 324 pages, to give a good accounting of what the Rorschach Test is really about. They suggest the test’s persistence and popularity has to do with wizardry, or perhaps chicanery, but realistically speaking, all of their attention in a growing list of publications does not yield a clue.

We have adopted the position over the years in both our theoretical and research endeavors that the Rorschach Test yields a sample of verbal behavior that embeds the respondent’s stylistic dispositions and repertoire of ego states. This verbal material, in the form of a written protocol, may be coded by any number of pertinent coding schemes (We have proposed that computerized content or text analytic approaches may render the whole statistical debate moot; see Gottschalk, 1979). As such the material provides a deeply personal, that is, idiographic view of the person (Stricker & Gold, 1999).

On occasion, the authors give credit where credit is due, for example, the previously mentioned lauding of Marguerite Hertz, recognition of the Joseph Zubin (a thoughtful and well-informed critic of the Rorschach Test), mention of the Thought Disorder Index by Philip Holzman, or the work of Edward Aronow, who uses the Rorschach as a clinical technique not a psychometric instrument, and the strongly empirical work of Goldfried, Stricker, & Weiner (1971). Consistent with their selective use of the literature, however, they skate over the weighty scientific contributions of R. Holt and S. Blatt. There are excellent suggestions/recommendations concerning canons of Rorschach research and necessary foundational psychometric requirements for scientific respectability (Exhibit 9.2). They rightly criticize misuse of the test for purposes that it should not be used, for example, vocational choice or confirmation of sexual abuse. They seem to identify the Rorschach Test as uniquely flawed among instruments commonly used in assessment psychology. Many of their criticism are relevant to any psychological test used inappropriately by a poorly trained clinician. They fail to place the Rorschach’s capabilities into context with other assessment instruments (e.g., the MMPI), or the general weakness of “soft psychology” in general (Acklin, McDowell, & Ornduff, 1992; Cohen, 1962, 1988). They suggest that the Society of Personality Assessment and the Journal of Personality Assessment, two respected, long-standing organizations dedicated to psychological evaluation, are corrupt and self-serving.

A third, serious weakness is most apparent in the way that the authors conclude their argument and, thus, the book. Chapter 12 (“Objection, Your Honor! Keeping the Rorschach Out of Court) and the Epilogue are notably weak. The authors fail to take advantage of a decisive opportunity to make their argument that the Rorschach Test should be inadmissible in court by avoiding a thorough analysis of the Rorschach literature in relationship to current admissibility standards (Frye Test, Federal Rules of Evidence, Daubert and Kumho). This, of course, they could not do since theirs is not a dispassionate inquiry. The Epilogue suggests their ultimate befuddlement: How could or does the Rorschach Test maintain its endurance and clinical popularity? As clinicians persist in using the test despite the depredations of managed care and the criticisms of Wood, et al. and their ilk, there must be some other reason beyond fad, mendacity, or self-serving economic motivation. Here the authors appear clueless.

What’s Wrong with the Rorschach? is not and does not purport to be a systematic analysis of the behavioral science foundations of the Rorschach test. This has yet to be written. Because of this, despite an informative and generally well-written text, the authors fail to make their case. It remains for the community of Rorschach researchers and clinicians to meaningfully address the valid critical points raised here in a way that establishes a firm empirical foundation for the Rorschach Test’s continuing use. Although the authors call for a moratorium on the test, which even they admit is unlikely to occur, they should be satisfied if Rorschach Test users adopt a more critical and informed approach to all of their assessment endeavors.

Acklin, M.W., McDowell, C.J., & Orndoff, S. (1992). Statistical power and the Rorschach: 1975-1991. Journal of Personality Assessment, 59, 3 66-3 79.
Acklin, M. W., McDowell, C. J., Verschell, M.J., and Chan, D. (2000). Interobserver agreement, intraobserver reliability, and the Rorschach Comprehensive System. Journal of Personality Assessment, 74, 15-47.
Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: LEA, Inc.
Gottschalk, L. A. (1979). Content analysis of verbal behavior: Further studies. Hillsdale, NJ: LEA, Inc.
Lohr, J.M., & Fowler, K.A., & Lillienfeld, S. O. (2002). The dissemination and promotion of pseudoscience in clinical psychology: The challenge to legitimate clinical science. The Clinical Psychologist, 55, 4-10).
McCoun, R. J. (1998). Biases in the interpretation and use of research results. Annual Review of Psychology, 1998, 49, 259-87.
Stricker, G., & Gold, J.S. (1999). The Rorschach: Toward a nomothetically based, idiographically applicable configurational model. Psychological Assessment, 11, 240-250.

Karen Zelan has written extensively on the psychology of children’s learning. She is the author of Between Their World and Ours:Breakthroughs with Autistic Children.

© APA Div. 39 (Psychoanalysis). All rights reserved. Readers therefore must apply the same principles of fair use to the works in this electronic archive that they would to a published, printed archive. These works may be read online, downloaded for personal or educational use, or the URL of a document (from this server) included in another electronic document. No other distribution or mirroring of the texts is allowed. The texts themselves may not be published commercially (in print or electronic form), edited, or otherwise altered without the permission of the Division of Psychoanalysis. All other interest and rights in the works, including but not limited to the right to grant or deny permission for further reproduction of the works, the right to use material from the works in subsequent works, and the right to redistribute the works by electronic means, are retained by the Division of Psychoanalysis. Direct inquiries to the chair of the Publications Committee.