Time & Location: March 29, Friday, 1:30pm - 2:30 pm, at SEC 1.413
1st Speaker: Shi Feng (Harvard)
Title: Reveal Planning Capability of Autoregressive Learning in Large Language Models
Abstract: Search and planning are fundamental constructs of human intelligence, involved in almost every aspect of our daily lives, from completing tasks at work to organizing trips, to seeking mathematical proofs of theorems, and more. Studying the planning capabilities of large language models (LLMs) can help us understand the differences in the decision-making processes between humans and artificial intelligence. To do this, we first abstract planning as a path-finding problem in a network, analogous to real-world scenarios such as decision-making for multi-step tasks or proof planning for mathematical reasoning. We find that the transformer-based autoregressive model generally achieves high accuracy in the path-finding task. We discover that the model generates the next node on the path by learning and applying two matrices: the adjacency matrix and the reachability matrix, within its multi-layer perceptron (MLP). Our analysis shows that applying gradient descent to minimize the cross-entropy loss on the training data indeed leads to the construction of these two matrices. These findings shed light on how the internal mechanism of autoregressive learning achieves planning in networks, which may help us understand the general planning capability in other related domains.
2nd Speaker: Daniel Halpern (Harvard)
Title: Computing Voting Rules with Elicited Incomplete Votes