Researchers seem surprised... Image Recognition Systems: Images of women received three times more annotations related to physical appearance; women in images are recognized at substantially lower rates in comparison with men

Schwemmer, Carsten, Carly Knight, Emily Bello-Pardo, Stan Oklobdzija, Martijn Schoonvelde, and Jeffrey W. Lockhart. 2018. “Diagnosing Gender Bias in Image Recognition Systems.” SocArXiv. September 21. doi:10.1177/2378023120967171

Abstract: Image recognition systems offer the promise to learn from images at scale without requiring expert knowledge. However, past research suggests that machine learning systems often produce biased output. In this article, we evaluate potential gender biases of commercial image recognition platforms using photographs of U.S. members of Congress and a large number of Twitter images posted by these politicians. Our crowdsourced validation shows that commercial image recognition systems can produce labels that are correct and biased at the same time as they selectively report a subset of many possible true labels. We find that images of women received three times more annotations related to physical appearance. Moreover, women in images are recognized at substantially lower rates in comparison with men. We discuss how encoded biases such as these affect the visibility of women, reinforce harmful gender stereotypes, and limit the validity of the insights that can be gathered from such data.