Selection Problems in the Presence of Implicit Bias

Authors Jon Kleinberg, Manish Raghavan

Thumbnail PDF


  • Filesize: 0.62 MB
  • 17 pages

Document Identifiers

Author Details

Jon Kleinberg
Manish Raghavan

Cite AsGet BibTex

Jon Kleinberg and Manish Raghavan. Selection Problems in the Presence of Implicit Bias. In 9th Innovations in Theoretical Computer Science Conference (ITCS 2018). Leibniz International Proceedings in Informatics (LIPIcs), Volume 94, pp. 33:1-33:17, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2018)


Over the past two decades, the notion of implicit bias has come to serve as an important com- ponent in our understanding of bias and discrimination in activities such as hiring, promotion, and school admissions. Research on implicit bias posits that when people evaluate others - for example, in a hiring context - their unconscious biases about membership in particular demo- graphic groups can have an effect on their decision-making, even when they have no deliberate intention to discriminate against members of these groups. A growing body of experimental work has demonstrated the effect that implicit bias can have in producing adverse outcomes. Here we propose a theoretical model for studying the effects of implicit bias on selection decisions, and a way of analyzing possible procedural remedies for implicit bias within this model. A canonical situation represented by our model is a hiring setting, in which recruiters are trying to evaluate the future potential of job applicants, but their estimates of potential are skewed by an unconscious bias against members of one group. In this model, we show that measures such as the Rooney Rule, a requirement that at least one member of an underrepresented group be selected, can not only improve the representation of the affected group, but also lead to higher payoffs in absolute terms for the organization performing the recruiting. However, identifying the conditions under which such measures can lead to improved payoffs involves subtle trade- offs between the extent of the bias and the underlying distribution of applicant characteristics, leading to novel theoretical questions about order statistics in the presence of probabilistic side information.
  • algorithmic fairness
  • power laws
  • order statistics
  • implicit bias
  • Rooney Rule


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Marianne Bertrand and Sendhil Mullainathan. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American Economic Review, 94(4):991-1013, 2004. Google Scholar
  2. Iris Bohnet, Alexandra van Geen, and Max Bazerman. When performance trumps gender bias: Joint vs. separate evaluation. Management Science, 62(5):1225-1234, 2016. Google Scholar
  3. Mark Braverman and Elchanan Mossel. Sorting from noisy information. arXiv preprint arXiv:0910.1191, 2009. Google Scholar
  4. Marilyn Cavicchia. How to fight implicit bias? With conscious thought, diversity expert tells NABE. American Bar Association: Bar Leader, 40(1), 2015. Google Scholar
  5. Aaron Clauset, Cosma R. Shalizi, and Mark E. J. Newman. Power-law distributions in empirical data. SIAM Review, 51(4):661-703, 2009. Google Scholar
  6. Brian W. Collins. Tackling unconscious bias in hiring practices: The plight of the Rooney Rule. NYU Law Review, 82(3), 2007. Google Scholar
  7. Covington and Burling. Recommendations to Uber, 13 June 2017. Google Scholar
  8. H. A. David and H. N. Nagaraja. Basic Distribution Theory, pages 9-32. John Wiley &Sons, Inc., 2005. Google Scholar
  9. Uriel Feige, Prabhakar Raghavan, David Peleg, and Eli Upfal. Computing with noisy information. SIAM Journal on Computing, 23(5):1001-1018, 1994. Google Scholar
  10. Qiang Fu and Jingfeng Lu. Micro foundations of multi-prize lottery contests: a perspective of noisy performance ranking. Social Choice and Welfare, 38(3):497-517, 2012. Google Scholar
  11. Alexander R. Green, Dana R. Carney, Daniel J. Pallin, Long H. Ngo, Kristal L. Raymond, Lisa I. Iezzoni, and Mahzarin R. Banaji. Implicit bias among physicians and its prediction of thrombolysis decisions for black and white patients. Journal of General Internal Medicine, 22(9):1231-1238, 2007. Google Scholar
  12. Anthony G. Greenwald and Mahzarin R. Banaji. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological Review, 102(1):4-27, 1995. Google Scholar
  13. Anthony G. Greenwald and Linda Hamilton Krieger. Implicit bias: Scientific foundations. California Law Review, 94:945-967, 2006. Google Scholar
  14. Christine Jolls and Cass R. Sunstein. The law of implicit bias. California Law Review, 94:969-996, 2006. Google Scholar
  15. Manuel Lopez and James Marengo. An upper bound for the expected difference between order statistics. Mathematics Magazine, 84(5):365-369, 2011. Google Scholar
  16. Henrick John Malik. Exact moments of order statistics from the pareto distribution. Scandinavian Actuarial Journal, 1966(3-4):144-157, 1966. Google Scholar
  17. Christina Passariello. Tech firms borrow football play to increase hiring of women. Wall Street Journal, 27 September 2016. Google Scholar
  18. Hamza Shaban. What is the "Rooney Rule" that Uber just adopted? Washington Post, 13 June 2017. Google Scholar
  19. Francesco Giacomo Tricomi and Arthur Erdélyi. The asymptotic expansion of a ratio of gamma functions. Pacific J. Math, 1(1):133-142, 1951. Google Scholar
  20. Eric Luis Uhlmann and Geoffrey L. Cohen. Constructed criteria: Redefining merit to justify discrimination. Psychological Science, 16(6):474-480, 2005. Google Scholar
  21. Linda van den Bergh, Eddie Denessen, Lisette Hornstra, Marinus Voeten, and Rob W. Holland. The implicit prejudiced attitudes of teachers: Relations to teacher expectations and the ethnic achievement gap. American Education Research Journal, 47(2):497-527, 2010. Google Scholar
  22. Christine Wenneras and Agnes Wold. Nepotism and sexism in peer-review. Nature, 387:341-343, 1997. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail