Madam Speaker: Are Female Presenters Treated Worse in Econ Seminars? The evidence seems statistically weak & conceptually inconclusive

Madam Speaker: Are Female Presenters Treated Worse in Econ Seminars? Uri Simonsohn. Data Colada, April 30, 2021. http://datacolada.org/96

A recent NBER paper titled "Gender and the Dynamics of Economics Seminars" (https://www.bipartisanalliance.com/2021/02/economics-seminars-women-presenters-are.html) reports analyses of audience questions asked during 462 economics seminars, concluding that

“women are asked more questions . . . and the questions asked of women are more likely to be patronizing or hostile . . . suggest[ing] yet another potential explanation for their under-representation at senior levels within the economics profession” (abstract)

In this post I explain why my interpretation of the data is different.

My prior, before reading this paper, was that women were probably treated worse in seminars, especially in economics. But, after reading this paper I am less inclined to believe that.

[...]

Do Female Speakers Get More Antagonistic Questions?

Another result highlighted in the abstract is that the questions female speakers get “are more likely to be patronizing or hostile”.

Unlike the optimal number of total questions, the optimal number of hostile and patronizing questions is zero. So noticeable differences in hostility are easier to interpret.

But the evidence behind those claims seems insufficiently clear, in my opinion, to be interpretable, let alone actionable. Specifically, the evidence is:

*  Statistically weak. The estimates are arguably small in magnitude (e.g, women get 0.1 extra hostile questions on average), and evidentially weak (the patronizing difference is p=.1, the hostile difference p=.02). Moreover, these two results were selected post-hoc from a larger set of measures collected, and the rest were not significant (e.g., if questions were critical, or disruptive, or fair). Statistically at least, this is not strong evidence against the null that all observed differences are caused by chance.

*  Conceptually inconclusive. Other estimates in the paper are conceptually inconsistent with the conclusion that female speakers receive worse treatment. For example, they get directionally fewer “criticism” questions than do male speakers  (see Table 6 .png). While this result is p > .1, so we do not rule not zero difference, the estimate is precise enough to rule out that women get ¼ additional  critical question per talk. Women also get an additional 1.7 clarification questions (p < .01), and half an additional suggestion (p < .1). Personally I like getting these kinds of questions, as they often signal audience engagement and can, of course, be useful.

---

More at the original link