Wednesday, November 20, 2024, 3:30pm
Large text-to-image models learn from training data to synthesize “novel” images, but how the models use the training data remains a mystery. The problem of data attribution is to identify which training images are influential for generating a given output. Specifically, removing influential images and retraining the model would prevent it from reproducing that output image. Unfortunately, directly searching for these “ground truth” influential images is computationally infeasible since it would require repeatedly retraining from scratch.
My research aims to develop effective and scalable attribution methods and evaluation schemes for large text-to-image models. First, I present a computationally feasible attribution benchmark for large text-to-image models. Through “customization” methods, we define ground truth attribution labels by creating synthetic images computationally influenced by exemplar images. This scheme allows efficient evaluation by avoiding retraining repeatedly. Next, I will present a new data attribution approach for general text-to-image models. We simulate unlearning the synthesized image, find training images that are forgotten after the unlearning process, and label these as influential.
Finally, I will present ongoing work on improving the efficiency of attribution algorithms and propose a future research direction for developing interpretable attribution algorithms.
Thesis Committee
Jun-Yan Zhu (Chair)
Deva Ramanan
Ruslan Salakhutdinov
Alexei A. Efros (University of California, Berkeley)
David Bau (Northeastern University)
In Person and Zoom Participation. See announcement.
Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: SHENG-YU WANG
Speaker Website: peterwang512.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Data Attribution for Text-to-Image Models
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…