Hermann Rorschach published Psychodiagnostik in 1921. Rorschach???s inkblots soon attracted a cult-like following and became the most widely used projective test in the world. The theory behind projective tests is that, when people are asked to describe an ambiguous stimulus, their descriptions will reveal their private thoughts and fantasies. In this article, Wray Herbert raises a number of issues that are worth additional comment.
First, the article confuses personality measurement with the assessment of psychopathology. From the beginning of personality measurement, every major measure of personality was also a measure of psychopathology; these measures included the Rorschach and the Minnesota Multiphasic Personality Inventory (MMPI). Research on performance found that???many people with problematic MMPI profiles perform well under pressure and many people with normal profiles perform poorly. Realizing that psychopathology is not necessarily related to effectiveness, Harrison Gough (author of the California Psychological Inventory in 1954) developed a measure of normal personality to predict competent and effective performance. The point is, it is possible to assess personality without assessing psychopathology; and it is necessary to do so if one wants to predict effectiveness.
Second, the article misrepresents the concept of test reliability. The reliability of a score is estimated by taking the same measure two or more times and comparing the scores. In contrast, many psychologists think that reliability should be estimated by how closely the items on a test cohere in a statistical fashion???but this has nothing to do with the reliability as defined in the physical sciences. The article defines reliability in terms of the degree to which two people who score the same responses on the same test, get the same results. This definition is mistaken because it concerns the reliability of the scoring method not the test scores.
Third, there is nothing wrong, in principle, with the Rorschach. Like any test, it is a collection of (10) test stimuli, which by themselves mean nothing. The utility of any test depends on its scoring key. More specifically, the utility of a test depends on validity???the degree to which scores on the test predict real world outcomes. It is possible to develop scoring keys for the Rorschach that predict outcomes, but first it is necessary to understand what the purpose of assessment is.
Finally, Wray Herbert understands the importance of validity. He notes that ???This dust-up over the Rorschach could be just the beginning of a major intellectual housecleaning in a field that has drifted from its scientific roots.??? As this comment indicates, validity is the scientific raison d???etre for assessment, but it is something that many test publishers ignore. This fact is a public scandal and one that will ultimately come to haunt the entire test publishing enterprise.
Analyze This
The real problem with the Rorschach test: It doesn't work.
Email To A Friend
Please fill in the following information and we'll email this link.
There is nothing ambiguous about the image in my mind. It clearly depicts two medieval wizards, with tall red hats and black cloaks. They are sitting facing one another other. They appear to be giving each other the high-five.
That's my interpretation of inkblot No. 2 of the Rorschach test, a psychological test used by clinical psychologists and other therapists to assess personality and diagnose psychopathology. I don't know if my interpretation is normal or aberrant, but I do know that most people see two human beings of some kind in inkblot No. 2. I know this because Wikipedia recently published all 10 of the inkblots that Swiss psychiatrist Hermann Rorschach first introduced in his book Psychodiagnostik back in 1921—along with the most common "answers" for each of the inkblots. Therapists use these common answers—or norms—to help them diagnose abnormal behavior and thinking.
Wikipedia's move has sparked a firestorm among psychotherapists who claim that publishing the norms could skew the test's results—or worse, allow patients to fool their therapists, to game the system. Free-speech advocates—including many other therapists—dismiss those claims as nonsense.
This shouting match escalated this week when The New York Times published a long article about the Wikipedia-Rorschach brouhaha. But this heated debate has failed to raise (or answer) the most important question of all: does the Rorschach work? The answer is no, and here is the best evidence:
The journal Psychological Science in the Public Interest published an exhaustive review of all data on the Rorschach (and other similar "projective" tests) in 2000. Such meta-analyses are major undertakings, so although this PSPI report is a few years old, it remains the most definitive word on the Rorschach. The authors—psychologists Scott Lilienfeld, James Wood and Howard Garb—find the Rorschach wanting in two crucial ways.
First, the test lacks what testing experts call "scoring reliability." Scoring reliability means than you get the same results no matter who is scoring the test. Psychotherapists look at more than 100 different variables when scoring an answer: Did the patient focus on stray splotches rather than the main blot, or the white spaces instead of the ink? Did the patient interpret the color? That kind of thing. The PSPI review found that therapists disagree on fully half of these variables, making the scores unreliable for diagnosis.
But it gets worse. The authors also looked at all the extant studies on the test's validity. This is testing jargon for: Does it measure what it claims to measure? Does it predict behavior? And again the answer is a clear no. With the exception of schizophrenia and similarly severe thought disorders, the Rorschach fails to spot any common mental illnesses accurately. The list of what it fails to diagnose includes depression, anxiety disorders, psychopathic personality, and violent and criminal tendencies. It also can't detect sexual abuse in children, even though it's used for that purpose. Finally, the test is most misleading for minorities: blacks, Native Americans and Hispanics are all likely to score abnormally on the inkblot test.
Despite this damning evidence, the most recent survey data indicates that four in 10 clinical psychologists still use the Rorschach "always or frequently" with patients. Why would that be? This isn't the first time the Rorschach has come under attack. The test was roundly criticized back in the '50s for lacking standardization and norms. Those problems were presumably corrected in the '70s, with the introduction of an elaborate system of instructions for therapists, and many newly trained therapists incorporated the revised test into their practices. Even so, it is this revised version of the Rorschach that still fails on both reliability and validity, according to the PSPI report.
The same psychological journal will in a few months be publishing another major review of clinical practice, with the goal of weeding out therapies and techniques that have no scientific evidence to back them up. This dust-up over the Rorschach could be just the beginning of a major intellectual housecleaning in a field that's drifted from its scientific roots. Does anyone else see a battlefield in that amorphous inkblot?
Herbert writes the We’re Only Human blog .
© 2009
Discuss