Abstract: Clinical prediction models often encode group attributes like sex, age, and HIV status for personalization – i.e., to assign more accurate predictions to heterogeneous subpopulations. In this talk, I will describe how such practices inadvertently lead to worsenalization, by assigning unnecessarily inaccurate predictions to minority groups. I will discuss how these effects violate our basic expectations from personalization in medical applications, and describe how they arise due to standard practices in algorithm development. I will end by highlighting work on how to address these issues in practice – first, by setting "personalization budgets" to test for worsenalization; second, by developing "participatory systems" where we can consent to share personal data at prediction time.
Papers
Suriyakumar, Ghassemi, Ustun. When Personal
Paes, Long, Ustun, Calmon. On the Epistemic Limits of Personalized Prediction. NeurIPS 2022
Joren, Nagpal, Heller, Ustun. Participatory P