Highlights
• Participants overestimate the effectiveness of a completely inefficacious drug.
• Diseases that resolve spontaneously boost overestimations of effectiveness.
•Overestimations remain even when participants consider all evidence available.
Abstract
Rationale: Self-limited diseases resolve spontaneously without treatment or intervention. From the patient's viewpoint, this means experiencing an improvement of the symptoms with increasing probability over time. Previous studies suggest that the observation of this pattern could foster illusory beliefs of effectiveness, even if the treatment is completely ineffective. Therefore, self-limited diseases could provide an opportunity for pseudotherapies to appear as if they were effective.
Objective: In three computer-based experiments, we investigate how the beliefs of effectiveness of a pseudotherapy form and change when the disease disappears gradually regardless of the intervention.
Methods: Participants played the role of patients suffering from a fictitious disease, who were being treated with a fictitious medicine. The medicine was completely ineffective, because symptom occurrence was uncorrelated to medicine intake. However, in one of the groups the trials were arranged so that symptoms were less likely to appear at the end of the session, mimicking the experience of a self-limited disease. Except for this difference, both groups received similar information concerning treatment effectiveness.
Results: In Experiments 1 and 2, when the disease disappeared progressively during the session, the completely ineffective medicine was judged as more effective than when the same information was presented in a random fashion. Experiment 3 extended this finding to a new situation in which symptom improvement was also observed before the treatment started.
Conclusions: We conclude that self-limited diseases can produce strong overestimations of effectiveness for treatments that actually produce no effect. This has practical implications for preventative and primary health services. The data and materials that support these experiments are freely available at the Open Science Framework (https://bit.ly/2FMPrMi)
Keywords: Cognitive biasTreatment effectivenessPseudotherapyPatients' beliefsCausality bias
5.3. General discussion
Pseudotherapies are drugs or other types of treatment that produce no beneficial effect on the likelihood of improving from a disease, such that the contingency between using the pseudotherapy and observing an improvement is null, once potential placebo effects are discounted (Hellmuth et al., 2019). Nevertheless, certain situations can induce illusory beliefs of effectiveness, even for pseudotherapies that are completely ineffective. In particular, diseases with high likelihood of spontaneous remission, P(O), can result in strong overestimations of effectiveness (Matute et al., 2019). Yet in most experiments the P(O) is set to a fixed level during the whole session (Blanco et al., 2014; Chow et al., 2019), with only a few studies investigating the effect of changing the P(O) levels as the session proceeds. In this context, self-limited diseases are of interest because, given their natural evolution, they usually produce a pattern of steady increasing probability of remission. According to previous research, an increasing pattern of P(O) could in principle promote strong illusions of effectiveness in a null contingency setting (Ejova et al., 2013; Matute, 1995), though sometimes the opposite result (e.g., accurate estimations of non-effectiveness) has been reported (Langer and Roth, 1975). Additionally, none of these previous studies investigated the effect of increasing patterns of P(O) in a medical context. Testing the effect of increasing P(O) using a medical context is important, because this pattern, which is common in self-limited diseases, could represent one of the gaps through which pseudotherapies infiltrate our societies. Given that self-limited diseases are common, if they do favor the use of pseudotherapy by promoting the illusory belief in their effectiveness, then many people could turn to these pseudotherapies uncritically whenever they suffer a more serious health problem.
Our three behavioral experiments align in documenting evidence for the overestimation of effectiveness in self-limited diseases. In this regard, they coincide with results reported in non-medical scenarios (Ejova et al., 2013; Matute, 1995), and contradict a classic study (Langer and Roth, 1975). Nevertheless, we note that Langer and Roth's study tapped on a chance situation (coin toss) in which no improvement should be expected with time or practice, which could help to interpret the discrepancy. Additionally, in Ejova et al.'s study, the interpretation of the situation as chance-based was measured and controlled for in the analysis.
In our experiments, trial-by-trial judgments increased gradually with the P(O) in the self-limited group (see Fig. 4, Fig. 6). As a result, the participants in this group ended their training session with a strong belief of effectiveness. This contrasts with participants in the control group, who showed a significantly smaller illusion of effectiveness. A possible interpretation is that people assume that patterns of gradual improvement are the typically expected ones when a treatment is actually working. That is, people rarely attribute these gradually improving patterns to the natural course of a self-limited disease, and instead they look for a potential explanation, which in this case is the treatment.
Experiment 2 replicated the results of Experiment 1 (Fig. 4), differing only on the type of response scale used (unidirectional, instead of bidirectional). Although contingency can take on either positive or negative values, it is not obvious how to interpret a negative value in this particular context — a treatment for a disease. Consequently, most previous studies used a unidirectional scale (from 0, ineffective, to 100, perfectly effective). Here we obtain the same results irrespective of the scale. This speaks of the robustness of the general result: regardless of the type of response scale, the tendency to overestimate the effectiveness was higher in the group that mirrored the self-limited disease, compared with the control group.
In Experiment 3, we modeled a self-limited disease in a different way: in addition to the null contingency between taking the treatment and observing a remission of the symptoms, we showed the symptom-occurrence baseline before the treatment starts. This should allow participants to fairly compare the P(O) before and after the treatment, even under the assumption that trials are dependent on each other — that taking the drug on one day can help to improve health days later. Even with this information available, participants overestimated the effectiveness of the pseudomedicine when the pattern of P(O) was increasing, as compared to the control group (Fig. 6). Thus, a different design, resting on different assumptions than those in Experiments 1 and 2, still produced similar results.
Experiment 3 presents an interesting question, as it is designed to work under the assumption of non-independence between trials. Most contingency learning experiments describe trials that can be naturally interpreted as mutually independent: for example, it is common to present a sequence of trials arranged in random order, each one corresponding to a different individual patient (Blanco et al., 2014; Blanco and Matute, 2019). Thus, it is unlikely that participants assume that treating one patient could affect subsequent patients. This was probably the case in Ejova et al.'s (2013) experiments, and it certainly was in the case of Matute (1995) and Langer and Roth (1975) studies. Under the assumption of trial independence, it is easy to reproduce the overestimation of effectiveness when a disease has a high probability of spontaneous remission (Blanco et al., 2014; Blanco and Matute, 2019). However, in our current procedure, it is possible to interpret the training as a time-series with dependencies between trials: if the patient takes the medicine today, the effects can be seen some days later. As we have argued, assuming that trials can be mutually dependent in this fashion has important consequences, the main one being that contingency, computed as the difference between P(O|C) and P(O|∼C), stops being a useful index. This is a possibility that has been largely overlooked in most research in human contingency learning, with few exceptions (Bramley et al., 2015). In Experiment 3, we replicated the overestimation of effectiveness in these conditions, which extends the results of Ejova et al. (2013) to a new scenario. Future studies could deepen further in the implications of assuming dependency between observations.
Additionally, we collected information about the perceived effectiveness in two moments: on each trial (trial-by-trial judgments) and after the whole session (global judgment). It is well known that trial-by-trial judgments typically show recency effects: they are heavily affected by the information presented in the current or immediately previous trials (Matute et al., 2002). In global judgments, we asked participants to take into account all the information they had seen from the beginning of the session, a strategy that often results in the participants being able to integrate the information from the different phases (Matute et al., 2002). It would be useful to know whether people can be capable of integrating all the information and overcome their biases just by asking them to do so. Thus, we expected that global judgments in the self-limited group would not be so strongly influenced by the last part of the training session, which featured a very high P(O). This should probably reduce the illusion and blur the differences between the groups, given that they received exactly the same frequencies of trials when considering the overall session. However, global judgments in our experiments showed a familiar overestimation trend, with the self-limited group reporting stronger beliefs of effectiveness than the control group, although the effect was less pronounced than in trial-by-trial judgments (in Experiment 1, the difference between groups in the global judgments was just marginal). Therefore, this result is in line with previous studies showing that participants can better integrate all the information seen during the training at the end of the session if requested to do so, while trial-by-trial judgments are strongly affected by the most recent piece of information (Matute et al., 2002). Yet, note that the effect of increasing the P(O) during the training session survives despite the explicit instruction to try to avoid being too biased by the last part of the session. Consequently, the result highlights the view that self-limited diseases can potentially produce strong and resistant beliefs of effectiveness.
There were additional judgments in our task. First, through the Alternative Cause judgments, participants could report whether they felt that outcome occurrences (e.g., symptom remissions) could be attributed to causes other than the pseudomedicine. In Experiments 1 and 2, the results do not indicate any significant difference between groups. People do not tend to attribute health improvement instances to any unspecified alternative cause, irrespective of the group (average answers around 6 out of 10 points). In Experiment 3, in contrast, we found that participants in the self-limited group tended to judge that symptom remissions were due to the treatment more often than participants in the control group, which is in line with the predictions.
Second, our participants estimated the two conditional probabilities, P(O|C) and P(O|∼C) — probability of symptom remission when taking the medicine versus when not taking the medicine. In Experiments 1 and 2, P(O|C) was systematically more overestimated than P(O|∼C). The latter result is compatible with the presence of an illusion of causality, or overestimation of the effectiveness of the drug (Blanco and Matute, 2019), that exists in all groups, since participants seem to judge that improvements are generally more likely when one takes the medicine than when one does not. In fact, the difference between these two probability estimations (which is analogous to computing an estimated contingency) correlates significantly with effectiveness ratings in all six groups across the three experiments. In addition, in Experiment 3 not only this tendency to overestimate P(O|C) was found, but it was significantly stronger in the self-limited group, where effectiveness was judged higher, which also reinforces the idea that people in this group developed strong beliefs of effectiveness.
We can only speculate as to why the additional judgments (global judgment, Alternative Cause judgment, conditional probability judgments) seem to have captured stronger effects in Experiment 3 than in Experiments 1 and 2. One of the most evident reasons is that, by separating the outcome base-rate training in one phase, instead of presenting it in intermixed trials, Experiment 3 made it much easier to grasp the base-rate to which compare the P(O) once the treatment starts. Another, somewhat more trivial, reason is that we started and ended the training using more extreme values of P(O), namely 0 and 1, which might create the impression that the medicine works at the end of the session.
When interpreting Experiment 3, one should be aware that the design does not perfectly control all potential extraneous variables. Of special relevance is the difference in the P(O) during Phase 2 between the two groups: that is, the probability of symptom remission is overall higher in this phase for participants in the self-limited group, P(O) = 0.83, compared to participants in the control group, P(O) = 0.50. This difference could contribute to the effect, as we know from previous literature that higher P(O) levels typically produce higher judgments, such as with outcome-density bias (Chow et al., 2019; Musca et al., 2010). Future studies could try to address this problem by including additional controls. For instance, it would be possible to add a stable, high P(O) control condition, in which symptom remissions appear as often as in the last part of the self-limited group. We note, however, that such control would still not be perfect, as it is impossible to fix all relevant parameters (outcome probability, contingency, number of trials, etc.) while one of the groups remains stable and the other increases in P(O).
These experiments can also help us shed some light on the mechanisms underlying the overestimation of causality more generally. Traditionally, these illusory beliefs have been explained by invoking differential “cell-weighting” processes (Kao and Wasserman, 1993). That is, of the four cells in the contingency matrix of Fig. 1, it is possible that people pay more attention to those events that are more salient — typically, cells a, or medicine-remission events — or give them more importance when forming the belief. This proposal could explain both the present experiments as well as other instances of overestimation of effectiveness: even if the number of type a trials (medicine-remission) is the same as type c trials (no medicine-remission), people would weigh the former more heavily, thus strengthening the belief of effectiveness beyond the provided data. Concerning this possibility, we collected conditional probability judgments in addition to effectiveness judgments, and through the three experiments, these conditional probability estimations seem to show a tendency to overestimate the probability of remission conditional on the medicine, compared to when no medicine was taken, such that P(O|C) > P(O|∼C), especially in those individuals who developed stronger beliefs of effectiveness. This could be interpreted as an increased attention/importance given to medicine-present trials, which is in line with previous experiments investigating cell-weighting mechanisms (Kao and Wasserman, 1993).
5.4. Limitations
One obvious shortcoming of our procedure is the limited ecological validity, given that the situation is artificial, and participants must imagine how they would think in real life contexts. On the other hand, it seems difficult to experimentally study these processes of change in the beliefs of effectiveness with real treatments and outcomes. We believe that it would not be feasible for practical and ethical reasons, so our computer-based approach remains a valid candidate.
Another limitation of this research is that only two conditions were compared in each experiment: an increasing pattern of P(O) versus a stable pattern of P(O). Thus, we are not examining the potential role of consistency, but we are always comparing one group in which the outcome changes with another group in which it does not change. A more systematic approach would have included additional conditions: decreasing pattern of P(O), U-shaped pattern, etc. Some of these conditions are studied in previous experiments (Ejova et al., 2013; Matute, 1995), but, as they do not correspond well to any meaningful real situation in actual medicine usage, we decided to keep our experiments as simple as possible, by just comparing self-limited diseases with stable diseases. Future studies could put to test additional conditions.
In addition, this research rests on the assumption that the formation of effectiveness beliefs during the experimental session is due to basic processes (e.g., contingency learning) that are general-purpose, and work in similar ways for all people. Thus, we have not considered the potential impact of individual differences based on educational level, age, or attitudes, for example. This could be an interesting field of study for future research, as some studies have suggested that prior beliefs and attitudes can in fact modulate the formation of causal knowledge, leading to overestimations of causality similar to those reported here (Blanco et al., 2018). Moreover, it would be highly interesting to investigate whether unsupported beliefs of effectiveness actually translate into differences in treatment use, and whether this effect is further modulated by individual factors that could help us identify particularly vulnerable individuals or situations.
Posts
No comments:
Post a Comment