Monday, May 6, 2024, 10:30am

AI models have the potential to support and complement human decision-makers and users. And yet, the deployment of human-AI teams still faces practical challenges. The goal of my thesis is to (i) better understand how existing AI support fails to account for downstream user interactions, (ii) develop informative proxies to enable faster prototyping of AI support, and (iii) use insights from human studies to improve the design of human-AI teams. 

In the first part of my proposal, I will focus on predictive decision-making, where explanations have been hypothesized to help decision-makers make sense of AI predictions. I will overview user studies where we observe that this is not the case and present follow-up work to understand the failure modes of existing explanation methods. In the second part of my proposal, we explore one path to addressing the challenges of evaluating human-AI team designs in practice. I will present a use-case-grounded approach for predictive decision-making contexts called simulated evaluations. The final part of my proposal will cover our ongoing and proposed work to improve human-AI teaming in more complex, interactive settings, focusing on code co-pilots. I will present a recent evaluation and data collection effort to understand the effect of a model’s benchmark performance on downstream user helpfulness and proposed work on how insights and interaction data can be incorporated to build better AI pair programmers.

Thesis Committee:
Ameet Talwalkar (Chair)
Hoda Heidari
David Sontag (Massachusetts Institute of Technology)
Hal Daumé III (University of Maryland / Microsoft Research)
Dan Weld (University of Washington / AI2)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Gates Hillman 8102 and Zoom
Speaker's Name: VALERIE CHEN
Speaker Website: valeriechen.github.io
Speaker's Professional Title: Ph.D. Student, Machine Learning Department, Carnegie Mellon University
Talk Title: Towards a science of human-AI teams
For More Information: stidle@andrew.cmu.edu
Affiliations: Machine Learning Department (MLD)
Organization(s): School of Computer Science