Eliciting Thinking Hierarchy without a Prior by Yuqing Kong

Date: 

Friday, March 25, 2022, 1:00pm

Location: 

https://harvard.zoom.us/j/97692964231?pwd=K05BMEhDNTZtbUhHYkZ5S21qZ2FKQT09#success

Yuqing Kong will be presenting at the EconCS seminar meeting this Friday, 3/25, at 1 pm ET. The seminar will be held Zoom only, accessible at the usual link here. Hope to see you there :).

Title: Eliciting Thinking Hierarchy without a Prior

Abstract: A key challenge in crowdsourcing is that majority may make systematic mistakes. Prior work focuses on eliciting the best answer without a prior even when the majority is wrong. Here without any prior, we want to elicit the full hierarchy where the higher-ranking answers, which may not be supported by the majority, are from more sophisticated people. We propose a new model, called the thinking web, that describes the hierarchy among people's thinking types through a weighted directed acyclic graph. To learn the thinking web without any prior, we propose a novel, powerful and practical elicitation paradigm, the Answer-Guess paradigm and it works as follows. First, we ask a single open response question and ask for both of each respondent's answer and guess(es) for other people's answers. Second, we construct an Answer-Guess matrix that records the number of people who report a specific Answer-Guess pair. Third, by ranking the answers to maximize the sum of the upper triangular area of the matrix, we obtain and visualize the hierarchy of the answers without any prior. We also conduct four empirical studies to demonstrate the superiority of our approach compared to the plurality vote and also validate our thinking web model: more sophisticated people can reason about less sophisticated people’s mind and the hierarchy can be approximately described by a directed acyclic graph.

 

Bio: Yuqing Kong is currently an assistant professor at The Center of Frontier Computing Science (CFCS), Peking University. She obtained her Ph.D. degree from the Computer Science and Engineering Department at University of Michigan in 2018 and her bachelor degree in mathematics from University of Science and Technology of China in 2013. Her research interests lie in the intersection of theoretical computer science and the areas of economics: information elicitation, prediction markets, mechanism design, and the future applications of these areas to crowdsourcing and machine learning. Her papers were published in several conferences include WINE, ITCS, EC, SODA, AAAI, NeurIPS, ICLR, ECCV, IJCAI, WWW.