The (In)accuracy of Forecast Revisions in a Football Score Prediction Game: Better go with your gut instinct

Going with your Gut: The (In)accuracy of Forecast Revisions in a Football Score Prediction Game. Carl Singleton, James Reade, Alasdair Brown. Journal of Behavioral and Experimental Economics, December 16 2019, 101502. https://doi.org/10.1016/j.socec.2019.101502

Highlights
•    Judgement revisions led to worse performance in a football score prediction game
•    This is robust to the average forecasting ability of individuals playing the game
•    Revisions to the forecast number of goals scored in matches are generally excessive

Abstract: This paper studies 150 individuals who each chose to forecast the outcome of 380 fixed events, namely all football matches during the 2017/18 season of the English Premier League. The focus is on whether revisions to these forecasts before the matches began improved the likelihood of predicting correct scorelines and results. Against what theory might expect, we show how these revisions tended towards significantly worse forecasting performance, suggesting that individuals should have stuck with their initial judgements, or their ‘gut instincts’. This result is robust to both differences in the average forecasting ability of individuals and the predictability of matches. We find evidence this is because revisions to the forecast number of goals scored in football matches are generally excessive, especially when these forecasts were increased rather than decreased.



6  Summary and further discussion

In this paper, we have analysed the forecasting performance of individuals who each applied
their judgement to predict the outcomes of many fixed events. The context of this analysis was
the scoreline outcomes of professional football matches. We found that when individuals made
revisions their likelihood of predicting a correct scoreline, which they achieved around 9% of
the time when never making a revision, significantly decreased. The same applied for forecast
revisions to the result outcomes of matches. Not only were these findings robust to unobserved
individual forecasting ability and the predictability of events, but also there is evidence that
performance would have improved had initial judgements been followed.

As already mentioned, these results have some similarities with those found previously in
the behavioural forecasting literature. One explanation could be that game players anchor their
beliefs, expectations and, consequently, their forecasts on past or initial values. However, this
behaviour would not be consistent with our finding that on average forecasters made revisions
which not only improved on their goals scored forecast errors but which were also excessive.

There are several areas for further research, which could be explored with extensions of
the dataset used here. First, it appears to be a relatively open question as to how sources of
bias among sports forecasters interact with how they make revisions, such as the well-known
favourite-longshot bias. Second, players of the forecasting game studied here do reveal which EPL
team they have the greatest affinity for, though we are yet to observe this information ourselves. It
is an interesting question as to whether any wishful-thinking by the players manifests itself more
greatly before or after they revise their forecasts. Third, an aspect which could be studied from
these current data is whether players improve their forecasts over time, and if they learn how to
play more to the rules of the game itself, which should lead them to favour more conservative goals
forecasts. Fourth, these results concern a selective random sample of players who “completed”
the game. These are likely to be individuals who extract significant utility from making forecasts
of football match scorelines, who are thus more likely to return to their initial forecasts and make
revisions. It would be interesting whether more casual forecasters are better at sticking with their
gut instincts or better off from doing so. Finally, our results suggested an innovation to the game
which could improve the crowd’s forecasting accuracy and which could be easily tested: before
making forecasts, some of the game players could be informed that sticking with their initial
judgement, or gut instinct, is likely to improve their chances of picking a correct score.