Senior Honors Thesis Oral Presentation at UW Honors Symposium

Decoding Bias in Letters of Recommendation: A Word Embeddings Approach

Tanay Nagar1, Dr. Fred Sala4, ‡, Dr. Sarah Jung2, ‡, Dr. Shamya Karumbaiah3, ‡

1University of Wisconsin–Madison·2Department of Surgery, UW SMPH·3Department of Educational Psychology, UW-Madison·4Department of Computer Sciences, UW-Madison

Thesis advisor

Honors Symposium TalkFellowship Proposal
TL;DRWe use word embeddings (word2Vec + transformer-based) on an 11-year corpus of UW Plastic Surgery residency letters of recommendation to quantify, interpret, and visualize implicit gender and ethnic biases/trends.

Abstract

The integrity of medical residency selection is paramount in cultivating competent physicians. With recent changes to the selection process, the weight of the letter of recommendation has increased, bringing potential gender and ethnic biases to the forefront. This study introduced a novel computational approach in applicant assessment research to quantify, interpret, and visualize the root causes of implicit gender and racial biases. The study used natural language processing techniques, including word embeddings through Word2Vec, other custom transformer-based models, and a suite of bias-detection algorithms.

Acknowledgements

This research was supported by the Hilldale Undergraduate/Faculty Research Fellowship and the Richard Ralph Phoenix Rising Humanitarian Scholarship.