Coordinating Author Information: Isra Dar | University of Toledo
Session Abstract: INTRODUCTION: Dissertations are time-consuming for the student and the advisor. How often are these efforts rewarded by publication? The MMPI and Rorschach were chosen as the most popular multiscale self-report and performance personality tests in clinical psychology. Our primary research questions were (a) are the publication rates of MMPI and Rorschach dissertations comparable to those more broadly and to each other more specifically and (b) are dissertations with multiscale tests more likely to result in an increase in questionable research practices (QRPs) compared to clinical psychology-related dissertations more broadly. The QRPs were: (a) not reporting unsupportive hypothesized results in the article that were found in the dissertation, (b) selectively reporting significant findings from unplanned analyses, and (c) changing the hypotheses in the article after seeing the results in the dissertation.
METHODS: For the control sample, a search was conducted in ProQuest Dissertation & Thesis for dissertations between 2006 to 2015 with ‘clinical psychology’ documented as either the Classification, Subject, or Degree. The same search criteria were employed with the term “Rorschach” or the term “MMPI” in either the Title or Abstract. Subsequently, PsycINFO, MEDLINE, and ERIC were searched with the dissertation author’s name to determine if it had been published in an indexed journal. The first and second author blindly reviewed a subset of 100 dissertations, achieving excellent agreement on their study inclusion decisions (kappa = .93).
RESULTS: There were 106 Rorschach dissertations from 2006 to 2015, of which only 11 (10.4%) were published in indexed journals by January 2021—an essentially equivalent rate as the control sample (12.1%; 39 of 321) r (phi) = -.02, p = .623. Using the same methodology for the MMPI, only 10 (4.9%) of dissertations were published by January 2021 – a slightly lower rate than the control sample (r (phi) = -.12, p = .006) and a trend towards a lower rate than Rorschach dissertations (r (phi) = .10, p = .071).
Published Rorschach dissertations were less likely to contain QRPs than the control sample: t(45) = 2.16, d = -0.74, p = .036. Although the mean was lower (.89 v 1.19), published MMPI dissertations were not significantly less likely to contain QRPs than the control sample: t(43) = 0.87, d = -0.32, p = .391. The samples of published Rorschach and MMPI dissertations were too small to have enough power to detect a significant difference of total QRPs, although the published Rorschach dissertations had a lower total QRP mean than the published MMPI dissertations: Ms = .55 and .89.
CONCLUSIONS: Our concern that Rorschach and MMPI dissertations, due to the multiscale nature of these tests, would be more prone to selectively report significant findings in the published articles compared to the control group was unfounded. A study limitation was the unexpectedly small percentage of published dissertations, which impacted the ability to obtain statistical significance. Therefore, for the QRPs, we plan to review more dissertations prior to our 2022 SPA presentation to increase our sample size.
Authors:
Isra Dar | University of Toledo
Omar R. Assaly, BA | University of Toledo
Joni L. Mihura, PhD, ABAP | University of Toledo