Friday, 1-2pm in SEC 1.413 with Justin Payan coming to speak in-person all the way from UMass Amherst on:
Into the Unknown: Assigning Reviewers to Papers with Uncertain Affinities
Peer review cannot work unless qualified and interested reviewers are assigned to every paper. Nearly all automated reviewer assignment approaches estimate real-valued affinity scores for each paper-reviewer pair that serve as proxies for the predicted quality of a future review; conferences then assign reviewers to maximize the utilitarian welfare of the assignment. This procedure does not account for noise in affinity score computation --- reviewers can only bid on a small number of papers, and textual similarity models are inherently probabilistic estimators. In this talk, we will explore the case when paper-reviewer affinity scores are estimated using a probabilistic model. Using these probabilistic estimates, we bound the scores with high probability and maximize the worst-case utilitarian welfare for a reviewer allocation. We will discuss multiple ways to estimate probabilistic affinity scores, and how to robustly maximize welfare in these models. Our general approach can be used to integrate a large variety of probabilistic reviewer-paper affinity models into reviewer assignment, opening the door to a much more robust peer review process. LOCATION:SEC 1.413 https://goo.gl/maps/UjUiWMGZCEGh5qMr9 STATUS:CONFIRMED DTSTART:20230203T170000Z DTEND:20230203T183000Z END:VEVENT END:VCALENDAR