We cannot be impartial most of the time... Our judgement gets clouded by our political preferences --- again

Kahan, Dan M. and Peters, Ellen, Rumors of the 'Nonreplication' of the 'Motivated Numeracy Effect' are Greatly Exaggerated (August 26, 2017). Available at SSRN: https://ssrn.com/abstract=3026941

Abstract: This paper does three things. First, it describes the design defects (principally, the lack of statistical power) that make it misleading for Ballarini & Sloman (2017) to claim that they “failed to replicate” the results of Kahan, Peters et al. (2017). Second, it presents the positive results of our own replication study. Finally, we conclude with a brief discussion of why confining assertions of non-replication to studies that satisfy emerging replication protocols—in particular the imperative of “faithful recreation of a study with high statistical power” (Brandt, Ijzerman et al 2014, p. 217)—is essential to the contribution such studies can make as building blocks of a cumulative science.

---
The authors ask a neutral problem (if a skin cream works well or not for some rash), and a politicized question (is a change in gun laws associated to more or to less crime?) in a form called covariance detection task (they present fictional studies with fictional results and ask the subjects whether the study supports one answer or the other). They do not ask subjects what they think of the real world, just if the study supports or not an answer or the other... But the subjects cannot be neutral. The authors add:

This “covariance detection” task is hard. Consistent with existing literature, ***only 30%*** of the MN sample overall supplied the correct answer in the “skin rash” group. Skin-rash group participants who scored in the 95th percentile of numeracy, in contrast, tended to get the correct answer in the “skin rash” treatment group ***around 75%*** of the time.

The high-numeracy participants in that group also tended to identify the correct solution—but only when that solution affirmed the position associated with their political identity (crime increases for “Liberal Democrat” vs. crime decreases for “Conservative Republican”). When the correct answer disconfirmed (or “threatened”) the position associated with their political identity (crime increased for Liberal Democrat and crime decreased for Conservative Republican), the high-numeracy participants performed no better than the low- and moderately-numerate participants who shared their identity (Kahan, Peters, et al., 2017).

This result is not consistent with the widespread assumption that political and cultural conflict over scientific data is rooted in the prevalence of System-1 thinking in the general population. It is more consistent with an alternative view that sees numeracy and other forms of System-2 reasoning as re-sources that can be used in the service of identity-protective cognition, a form of motivated information processing that aims to maintain correspondence between an individual’s factual beliefs and the factual beliefs known to be markers of membership in, and loyalty to, one’s cultural group (Bolsen et al. 2014; Kahan 2013, 2009; Sherman & Cohen 2006).

---
Check also: Biased Policy Professionals. Sheheryar Banuri, Stefan Dercon, and Varun Gauri. World Bank Policy Research Working Paper 8113. https://www.bipartisanalliance.com/2017/08/biased-policy-professionals-world-bank.html

And: Dispelling the Myth: Training in Education or Neuroscience Decreases but Does Not Eliminate Beliefs in Neuromyths. Kelly Macdonald et al. Frontiers in Psychology, Aug 10 2017. https://www.bipartisanalliance.com/2017/08/training-in-education-or-neuroscience.html

And: Wisdom and how to cultivate it: Review of emerging evidence for a constructivist model of wise thinking. Igor Grossmann. European Psychologist, in press. Pre-print: https://www.bipartisanalliance.com/2017/08/wisdom-and-how-to-cultivate-it-review.html