Abstract: Algorithmic predictions are increasingly guiding societal resource allocations by identifying individuals for intervention. Fueling their adoption is their strong performance in standard evaluation settings. Yet, these settings are narrow and rest on assumptions that may not hold in actual allocation contexts. In this talk, I will discuss two common deviations from these evaluation assumptions that challenge conventional wisdom about using predictive tools for allocation.
First, we investigate the necessity of such fine-grained, individual-level predictive systems when individuals are already grouped into larger units (e.g., schools, neighborhoods, or hospitals) that existing policies use to guide resource allocation. Through a simple mathematical model, we show that individual-level predictions outperform unit-level methods only when between-unit inequality is low and intervention budgets are high.
Second, I will examine a crucial yet overlooked aspect of timing in prediction-driven allocations: the tradeoff between acting early on noisier predictions versus waiting for more accurate predictions to make more precise allocations that prevent individuals from experiencing undesirable outcomes. We characterize the settings where, although individual prediction accuracy improves over time, the average ranking of individuals and, consequently, the planner's ability to improve social welfare can counter-intuitively decline. As in our first finding, inequality emerges as a key driver of this phenomenon.
These insights reveal that, amid contested notions of efficiency, prediction-driven allocation systems may prioritize predictive accuracy at the expense of societal welfare. By exploring the nuances in evaluating these systems, we aim to spark a deeper conversation about when algorithmic predictions are necessary to improve social outcomes.
Based on joint works with Rediet Abebe, Moritz Hardt, and Ariel Procaccia.
Bio: Ali Shirali is a Ph.D. student in Computer Science at UC Berkeley, advised by Rediet Abebe and Moritz Hardt. Ali’s research broadly contributes to the scientific foundations of machine learning and algorithmic decision-making, with a focus on societal applications. He is particularly interested in understanding the limitations of predictive tools and how they can be more effectively evaluated and applied in real-world decision-making settings. Currently visiting Harvard this semester, Ali would love to chat with you if these topics resonate!
LOCATION:SEC 1.413 STATUS:CONFIRMED DTSTART:20241025T173000Z DTEND:20241025T183000Z END:VEVENT END:VCALENDAR