We Are Not Competent Combining Probability Forecasts: 60% and 60% Is 60%, but Likely and Likely Is Very Likely

Mislavsky, Robert and Gaertig, Celia, Combining Probability Forecasts: 60% and 60% Is 60%, but Likely and Likely Is Very Likely (September 16, 2019). SSRN: http://dx.doi.org/10.2139/ssrn.3454796

Abstract: How do we combine others’ probability forecasts? Prior research has shown that when advisors provide numeric probability forecasts, people typically average them (i.e., they move closer to the average advisor’s forecast). However, what if the advisors say that an event is “likely” or “probable?” In 7 studies (N = 6,732), we find that people “count” verbal probabilities (i.e., they move closer to certainty than any individual advisor’s forecast). For example, when the advisors both say an event is “likely,” participants will say that it is “very likely.” This effect occurs for both probabilities above and below 50%, for hypothetical scenarios and real events, and when presenting the others’ forecasts simultaneously or sequentially. We also show that this combination strategy carries over to subsequent consumer decisions that rely on advisors’ likelihood judgments. We find inconsistent evidence on whether people are using a counting strategy because they believe that a verbal forecast from an additional advisor provides more new information than a numerical forecast from an additional advisor. We also discuss and rule out several other candidate mechanisms for our effect.

Keywords: uncertainty, forecasting, verbal probabilities, combining judgments, combining forecasts, predictions