Raising awareness for the replication crisis in clinical psychology by focusing on inconsistencies in psychotherapy research: how much can we rely on published findings from efficacy trials? Michael P. Hengartner. Front. Psychol. | doi: 10.3389/fpsyg.2018.00256
Summary and conclusions: As in other psychological specialties (see Bakker et al., 2012), effect sizes published in the clinical psychological literature are often heterogeneous and inflated due to various scientific biases including allegiance bias (Luborsky et al., 1999), publication bias (Driessen et al., 2015), unblinded outcome assessors (Khan et al., 2012), sponsorship bias (Cristea et al., 2017b), or small sample sizes (Cuijpers et al., 2010b). After adjustment for systematic biases, efficacy estimates for various psychotherapy modalities tend to be disappointingly small (Cristea et al., 2017a; Cuijpers et al., 2010b). Some evidence suggests that when efficacy is estimated based exclusively on unbiased high-quality trials, effects of psychotherapy could fall below the threshold for clinical relevance (Cuijpers et al., 2014a). Recently, some psychotherapy researchers hence raised the controversial point that effects of both psychotherapy and pharmacotherapy for depression may entirely reflect a placebo effect (Cuijpers & Cristea, 2015). Of further concern is the gap between treatment efficacy in controlled laboratory trials and treatment effectiveness in naturalistic real-world settings (Hallfors & Cho, 2007; Westen et al., 2004). The literature reviewed in this commentary was restricted to the efficacy of clinical psychological interventions, as that topic is highly relevant for clinical psychology. Nevertheless, conflicting and irreproducible findings have been detected and discussed in various other hot topics within clinical psychology, including the effect of menopause on the occurrence of depression (Hengartner, 2017; Rössler et al., 2016), the putative consequences of violent video games (Calvert et al., 2017; Ferguson and Kilburn, 2010), or inconsistent associations between psychopathology and stress physiology (Chida and Hamer, 2008; Rosmalen and Oldehinkel, 2011). Even though the replication crisis was mostly addressed within social psychology, I conclude that it is no less pernicious and prevalent in clinical psychology. Psychotherapy was a marvellous invention, but initial enthusiasm regarding its efficacy has now been obfuscated due to scientific biases that systematically inflate estimates. Being aware of these issues may certainly improve our scientific and clinical endeavours.
- Antrobus, J. (1983). REM and NREM sleep reports: Comparisons of word frequencies by cognitive classes, Psychophysiology, 20, 562–568.Google Scholar
- Avila-White, D., Schneider, A., Domhoff, G.W. (1999). The Most Recent Dreams of 12–13 year-old boys and girls: a methodological contribution to the study of dream content in teenagers. Dreaming, 9, 2/3, 163–171.Google Scholar
- Azzone, P., Freni, S., Maggiolini, A., Provantini, K., Viganò, D. (1998). How early adolescents describe their dreams: a quantitative analysis, Adolescence, 33, 129, 229–244.Google Scholar
- Barcaro, U., Calabrese, R., Cavallero, C., Diciotti, R., Navona, C. (2002). Significance of automatically detected word recurrences in dream associations, Dreaming, 12, 2, 93–107.Google Scholar
- Benjamin, L.S. (1996). Interpersonal diagnosis and treatment of personality disorders. New York: The Guilford Press.Google Scholar
- Bucci, W. (1997). Psychoanalysis and cognitive science. New York: The Guilford Press.Google Scholar
- DeLaney, G.M. (1997). In your dreams: falling, flying and other dreams themes. New York: Harper & Collins.Google Scholar
- Domhoff, G.W. (1999). New directions in study of dream content using the Hall and Van de Castle coding system, Dreaming, 9, 2/3, 115–137.Google Scholar
- Domhoff, W. (1996). Finding meaning in dreams: A quantitative approach. New York: Plenum.Google Scholar
- Garfield, P. (2001). The universal dream key: the 12 most common dream themes around the world. New York: Harperperennial.Google Scholar
- Grinstein, A. (1983). Freud's rules of dream interpretation. Madison: Int. Univ. Press.Google Scholar
- Hall, C.S., Van de Castle, R.L. (1966). The content analysis of dreams. New York: Appleton-Century-Crofts.Google Scholar
- Kilroe, P. A. (2000). The dream as text. The dream as narrative, Dreaming, 10, 3, 125–138.Google Scholar
- Luborsky, L., Crits-Christoph, P. (1990). Understanding transference: The CCRT method. New York: Basic Books.Google Scholar
- Mergenthaler, E. Stinson, C.H. (1992). Psychotherapy transcription standard. Psychotherapy research, 2, 58–75.Google Scholar
- Mergenthaler, E., Freni, S., Giampieri, E., Ferrari, R. (1998). Regole standardizzate di trascrizione delle sedute di psicoterapia, Ricerca in Psicoterapia, 1, 29–46.Google Scholar
- Pagel, J.F., Blagrove, M., Levin, R., States, B., Stickgold, B., White, S. (2001). Definitions of dream: a paradigm for comparing flying descriptive specific studies of dream, Dreaming, 11, 4, 194–202.Google Scholar
- Popp, C. A, Diguer, L., Luborsky, L., Faude, J. (1996). Repetitive relationship themes in waking narratives and dreams, Journal of Consulting & Clinical Psychology, 64(5), 1073–1078.Google Scholar
- Roccioletti, G., Zulli, A.M., & Bertini, M. (1983). Il sogno nell'età evolutiva. Una ricerca sul contenuto manifesto. Roma: Bulzoni.Google Scholar
- Schredl, M. (2002). Questionnaires and diaries as research instruments in dream research: methodological issues, Dreaming, 12, 1, 17–26.Google Scholar
- Stevens, A. (1995). Private myths. Dreams and dreaming. London: Hamish Hamilton.Google Scholar
- Strauch, I., Lederbogen, S. (1999). The home dreams and waking fantasies of boys and girls between ages 9 and 15: a longitudinal study, Dreaming, 9, 2/3, 153–161.Google Scholar