Reliability and Validity of the Rorschach
Richard Niolon, Ph.D.
Based on two articles by Irving Weiner
Some Observations on the Validity of the Rorschach Inkblot Method, (1996)
Reliability
1) past studies used a scorecard approach, all studies in favor, all opposed… this raises problems since there are four things to make a quality study
a) hypothesis adequately operationalized
b) confounding variables eliminated or minimized
c) samples selected adequately and representative
d) samples large enough to have enough power to detect results
Thus, scorecards studies, reviews that treat all studies the same, and analysis of other studies that ignore effect sizes are of little use
Validity
2) correlating attitudes and behaviors with test scores has always been problematic. Two tests may correlate highly with each other but not with behavior.
3) when using a multidimensional test, there is not test validity, because you must study the validity of individual variables
4) validity? for what assessing psychopathology, normal personality functioning, or observable aspects of personality functioning
Research Bias
5) critics offer uninformed opinions --
-research is that it is valid when used properly (Shontz and Green, 1992)
-many say projection is the basis of the test when this is false
-Ziskin and Faust (1988) present forensic evidence to counter Rorschach findings for lawyers… any scholarly critique can't use this without a caveat, since Ziskin and Faust themselves warn the reader that the book is only designed to refute the Rorschach
He offers
6) state and trait characteristics are supported through m and Y compared to more stable variables which show good test-retest after 3 years:
Exner notes that 93% of people with an EB difference of 2 points held it at 1 year
7) therapeutic improvement was noted by 24 variables (non-orthogonal), which indicates that the variables measure something that changes in therapy, but not necessarily what you might think. Studies with PTSD show low D and high MOR during impaired phase
8) developmental changes in children, adolescents, to adulthood are also supported by the research in egocentricity index and FC:CF ratio
Weiner-Current Status Of The Rorschach Inkblot Method 1997
Scientific Status--good inter-rater reliability, test-retest reliability, construct validity, and norms. Convergent validity with the Rorschach is as good as it is with the MMPI.
Clinical Status--lists 6 aspects of personality structure the Rorschach measures well, and adds that in cases of schizophrenia and depression it can offer a differential diagnosis as well.
Treatment Planning--Weiner offers 4 variables that could help predict good and slow progress in short and long term therapy, as well as dropout from therapy.
Some Observations on the Validity of the Rorschach Inkblot Method, (1996)
Reliability
1) past studies used a scorecard approach, all studies in favor, all opposed… this raises problems since there are four things to make a quality study
a) hypothesis adequately operationalized
b) confounding variables eliminated or minimized
c) samples selected adequately and representative
d) samples large enough to have enough power to detect results
Thus, scorecards studies, reviews that treat all studies the same, and analysis of other studies that ignore effect sizes are of little use
Validity
2) correlating attitudes and behaviors with test scores has always been problematic. Two tests may correlate highly with each other but not with behavior.
3) when using a multidimensional test, there is not test validity, because you must study the validity of individual variables
4) validity? for what assessing psychopathology, normal personality functioning, or observable aspects of personality functioning
Research Bias
5) critics offer uninformed opinions --
-research is that it is valid when used properly (Shontz and Green, 1992)
-many say projection is the basis of the test when this is false
-Ziskin and Faust (1988) present forensic evidence to counter Rorschach findings for lawyers… any scholarly critique can't use this without a caveat, since Ziskin and Faust themselves warn the reader that the book is only designed to refute the Rorschach
He offers
6) state and trait characteristics are supported through m and Y compared to more stable variables which show good test-retest after 3 years:
M | .87 | FC | .86 |
T | .87 | L | .82 |
Afr | .90 | Egocentricity | .87 |
D | .83 |
Exner notes that 93% of people with an EB difference of 2 points held it at 1 year
7) therapeutic improvement was noted by 24 variables (non-orthogonal), which indicates that the variables measure something that changes in therapy, but not necessarily what you might think. Studies with PTSD show low D and high MOR during impaired phase
8) developmental changes in children, adolescents, to adulthood are also supported by the research in egocentricity index and FC:CF ratio
Weiner-Current Status Of The Rorschach Inkblot Method 1997
Scientific Status--good inter-rater reliability, test-retest reliability, construct validity, and norms. Convergent validity with the Rorschach is as good as it is with the MMPI.
Clinical Status--lists 6 aspects of personality structure the Rorschach measures well, and adds that in cases of schizophrenia and depression it can offer a differential diagnosis as well.
Treatment Planning--Weiner offers 4 variables that could help predict good and slow progress in short and long term therapy, as well as dropout from therapy.