CMU School of Computer Science Events: Recent Episodes

View Details

Monday, January 6, 2025, 10am – 1:45pm

Event Type: Meeting
Room Number: In Person
Building: Rashid Auditorium, Gates Hillman 4401 (and adjacent spaces)
For More Information: tcortina@cs.cmu.edu
Affiliations: Computational Biology Department (CBD), Computer Science Department (CSD), Human-Computer Interaction Institute (HCII), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI), Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Sunday, November 17, 2024, 1 – 2pm

Join us for an illuminating session with CMU’s NSF-funded AI Institute for Societal Decision Making (AI-SDM) as we explore Artificial Intelligence (AI) and Generative AI in our surrounding communities.

Join Dr. Zachary Lipton, Chief Technology Officer and Chief Scientist of Abridge, and Raj Reddy Associate Professor of Machine Learning at Carnegie Mellon University, as we explore the impact of AI in our communities.

This event is geared towards the general public and community.
This is a chance to ask questions and engage in a thoughtful discussion about the future of technology.

REGISTER

Event Type: Talks
Room Number: In Person and Virtual - ET
Building: Carnegie Library of Pittsburgh, South Wing Reading Room
Speaker's Name: Exploring Artificial Intelligence (AI) and Generative AI in our Communities
For More Information: ngottron@andrew.cmu.edu
Affiliations: Computer Science Department (CSD), Dietrich College of Humanities and Social Sciences, Human-Computer Interaction Institute (HCII), Language Technologies Institute (LTI), Machine Learning Department (MLD), Partnerships, Robotics Institute (RI), Software and Societal Systems Department (S3D)
Organization(s): AI-SDM
Event Website Title: Event Website and Registration
Event Website URL: www.carnegielibrary.org…

View Details

Friday, November 22, 2024, 9am

In recent years, the integration of communication efficiency and differential privacy in distributed optimization has gained significant attention, motivated by large-scale applications such as Federated Learning (FL), where both data privacy and efficient communication are critical. This thesis explores the development of novel techniques to address these challenges, with a focus on distributed mean estimation, differentially private prediction, and private optimization for empirical risk minimization.

The first part of this work addresses communication-efficient distributed vector mean estimation, an essential subroutine in distributed optimization and FL. We propose the Rand-Proj-Spatial family estimator which utilizes cross-client correlation to reduce the estimation error under fixed communication cost, by projecting client vectors into a random subspace using a Subsampled Randomized Hadamard Transform (SRHT). This approach captures cross-client correlation more effectively, demonstrating substantial performance gains over conventional sparsification techniques in various distributed optimization tasks.

The second part of this work focuses on maximizing the privacy-utility trade-offs in differentially private prediction through majority ensembling. We introduce the Data-dependent Randomized Response Majority (DaRRM) framework, which generalizes all private majority ensembling algorithms through a data-dependent noise function. Based on DaRRM, we propose a computationally tractable optimization procedure for maximizing utility under a fixed privacy loss. Empirical results demonstrate DaRRM’s effectiveness in private label ensembling for image classification, showing significant utility improvements over existing baselines.

The third part of this work investigates differentially private optimization in solving empirical risk minimization using shuffled gradient methods. Unlike conventional private optimizers such as DP-SGD, which benefits from privacy amplification by subsampling, shuffled gradient methods face unique challenges in privacy and convergence. We develop a theoretical framework for analyzing Incremental Gradient (IG) methods, the most basic form of shuffled gradient methods, that enables noise injection for privacy and the use of surrogate objectives, introducing a new dissimilarity metric to measure the difference between true and surrogate objectives. Leveraging privacy amplification by iteration, we establish the first empirical excess risk bound for differentially private IG (DP-IG), and show how interleaving public data in training can further improve privacy-convergence trade-offs in DP-IG.

Finally, we introduce two proposed works along the line of differentially private optimization. First, we aim to extend our theoretical framework to analyze Shuffle Once (SO) and Random Reshuffling (RR), two practical shuffled gradient methods beyond Incremental Gradient (IG) methods. This will enable us to understand their private counterparts, DP-SO and DP-RR, where privacy analysis is more complex due to a lack of understanding on privacy amplification through shuffling. Second, we plan to extend our framework from a local to a distributed or decentralized setting to analyze convergence rates of distributed shuffled gradient methods in both private and non-private contexts, while also investigating the impact of data heterogeneity among clients on convergence in this setting.

Thesis Committee
Gauri Joshi (Chair)
Steven Wu
Zachary Manchester
Swanand Kadhe (IBM Research)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon Hall 4305
Speaker's Name: SHULI JIANG
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Communication Efficient and Differentially Private Optimization
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, November 19, 2024, 10am

Applications written in JavaScript are often vulnerable to a range of security threats. On the frontend, DOM-based Cross-Site Scripting (DOM-XSS) allows attackers to inject malicious JavaScript code into a webpage. On the backend, arbitrary code execution (ACE) and arbitrary command injection (ACI) enable attackers to execute arbitrary code or commands on the server. Exploiting such vulnerabilities can lead to severe consequences, including unauthorized access to sensitive data and even full system compromise.

Each potential flow identified by these tools traces a program path where attacker-controlled input, such as a URL, reaches a sensitive function that may lead to arbitrary code execution. DTA requires finding a concrete input that demonstrates a potential flow in the target application, but prior work fails to thoroughly explore program paths. In the backend, these tools miss ACI and ACE that require inputs with complex structure. We develop a novel type- and structure-aware fuzzing technique to explore Node.js packages, and an enumerator to synthesize syntactically valid payloads for ACE vulnerabilities. Incorporating these components on prior work NodeMedic led to finding 2257 potential flows and confirm vulnerabilities in 766 Node.js packages.

A unique challenge in exploring frontend code is that program behavior may depend on user actions on the webpage. To address this, we develop a fuzzer to interact with the target webpage and evaluated it against 43,436 popular pages. Furthermore, we found that including optional GET parameters in the target URL uncovers significantly more DOM-XSS vulnerabilities. This led us to use dynamic symbolic execution to automatically synthesize GET parameters satisfying program constraints. Compared to our replication of prior work DOMsday, the fuzzer increases potential DOM-XSS flows found by 37% and confirms 57% more vulnerabilities.

Finally, we find that non-exploitable potential flows may still hint towards real vulnerabilities that require additional steps to confirm, such as bypassing sanitization measures and extending the attacker’s capabilities by executing other program parts. Thus, we propose the design and implementation of exploration strategies that efficiently explores the program to discover an exploitable path, using information from a given potential flow that we assume to have found already.

Thesis Committee
Limin Jia (Chair)
Pedro Adão (Co-chair, Instituto Superior Técnico)
Rui Maranhão (Co-chair, Universidade do Porto)
Lujo Bauer
Ruben Martins
José Fragoso (Instituto Superior Técnico)
Cristian-Alexandru Staicu (CISPA Helmholtz Center for Information Security)

Event Type: Thesis Proposals
Room Number: In Person
Building: Gates Hillman 7101
Speaker's Name: NUNO SABINO
Speaker Website: cmuportugal.org…
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Improving Code-Injection Vulnerability Detection & Confirmation in JS Programs via Program Analysis
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Friday, November 15, 2024, 3:30 – 4:30pm

We demonstrate that fundamental aspects of astrocyte morphology and physiology naturally lead to a dynamic, high-capacity associative memory system. These neuron-astrocyte networks are closely related to two popular machine learning architectures: Transformers and modern Hopfield networks. In the context of associative memory, we show that neuron-astrocyte networks follow superior, supralinear memory scaling laws, outperforming all known biological implementations of modern Hopfield networks. In the context of Transformers, we show that neuron-astrocyte interactions provide a natural biological substrate for building self-attention, the core operation in Transformers.

—

Leo Kozachkov is the 2024-2025 Goldstein Fellow at IBM Research, where he works on theory-driven AI and bio-inspired computing. He earned his Ph.D. in Brain and Cognitive Sciences from MIT in 2022 and his B.S. in Physics from Rutgers University–New Brunswick in 2016. In Fall 2025, he will join Brown University as a tenure-track Assistant Professor of Engineering and an Assistant Professor of Brain Science at the Carney Institute for Brain Science. His research group will focus on understanding dynamics, control, and computation in natural and artificial systems.

In Person Group Viewing and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Baker Hall 340A and Zoom
Speaker's Name: LEO KOZACHKOV
Speaker Website: kozleo.github.io
Speaker's Professional Title: Goldstine Fellow, IBM T.J. Watson Research Research
Talk Title: Building Self-Attention with Tripartite Synapses
For More Information: rdkeller@cs.cmu.edu
Affiliations: Computer Science Department (CSD), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI)
Organization(s): Neuroscience Institute

View Details

Monday, November 25, 2024, 3:30 – 4:30pm

We show how to push the performance of high-resolution cloth simulation, making the simulation interactive (in milliseconds) for models with one million degrees of freedom (DOFs) while keeping every triangle untangled. The guarantee of being penetration-free is inspired by the interior-point method, which converts the inequality constraints to barrier potentials. Nevertheless, we propose a major overhaul of this modality by defining a novel and simple barrier formulation which does not depend on the distance between mesh primitives. Such a non-distance barrier model allows a new way to integrate collision detection into the simulation pipeline. Another contributor to the performance boost comes from the so-called subspace reuse strategy. This is based on the observation that low-frequency strain vibrations are near orthogonal to the deformation induced by collisions or self-collisions, often of high frequency. Subspace reuse then takes care of low-frequency residuals, while high-frequency residuals can also be effectively smoothed by GPU-based iterative solvers. We show that our method outperforms existing fast cloth simulators by nearly one order while keeping the entire simulation penetration-free and producing high-equality animations of high-resolution models.

—

Dr. Yin Yang is currently an Associate Professor with the Kahlert School of Computing at the University of Utah. Before joining the U, he was a faculty member at Clemson University and University of New Mexico. He received Ph.D. degree of Computer Science from The University of Texas, Dallas in 2013 (the awardee of David Daniel Fellowship Prize). He was a Research/Teaching Assistant at UT Dallas as well as UT Southwestern Medical Center. His research mainly focuses on real-time physics-based computer graphics, animation and simulation with a strong emphasis on interdisciplinarity. He was a Research Intern in Microsoft Research Asia in 2012. He received NSF CRII (2015) and CAREER (2019) awards.

Event Type: Seminars
Room Number: In Person
Building: Newell-Simon 3305
Speaker's Name: YIN YANG
Speaker Website: yangzzzy.github.io
Speaker's Professional Title: Associate Professor, Kahlert School of Computing, University of Utah
Talk Title: High-resolution cloth simulation in milliseconds: Efficient GPU Cloth Simulation with Non-distance Barriers and Subspace Reuse Interactions
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: cdowney@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, November 18, 2024, 3:30 – 4:30pm

Generative modelling has been extremely successful in synthesizing text, images, and videos. Can the same machinery also help us better understand how to physically interact with the multimodal 3D world? In this talk, I will introduce some of my group's work in answering this question. I will first discuss how we can enable 2D image generation models to edit images in a 3D-aware manner, and how to generate audio for muted egocentric videos. I will then zoom in specifically on hand interactions by introducing (1) FoundHand, a large-scale generative model for synthesizing realistic 2D hand images, and (2) GigaHands, a new large-scale 3D hand activities dataset designed to push the boundary of hand interaction modeling. Finally, I will conclude with an outlook of the future of generative modeling for understanding 3D human interactions.

 —

Srinath Sridhar is an Assistant Professor of Computer Science at Brown University, where he leads the Interactive 3D Vision & Learning Lab. He received his PhD at the Max Planck Institute for Informatics and was subsequently a postdoctoral researcher at Stanford. His research interests are in 3D computer vision and machine learning. Specifically, his group focuses on visual understanding of 3D human physical interactions with applications ranging from robotics to mixed reality. He is a recipient of the NSF CAREER award, a Google Research Scholar award, and his work received the Eurographics Best Paper Honorable Mention. He spends part of his time as a visiting academic at Amazon Robotics, and has previously spent time at Microsoft Research Redmond and Honda Research Institute.

Event Type: Seminars
Room Number: In Person
Building: Newell-Simon 3305
Speaker's Name: SRINATH SRIDHAR
Speaker Website: srinathsridhar.com
Speaker's Professional Title: Assistant Professor, Department of Computer Science, Brown University
Talk Title: Generative Modelling for 3D Multimodal Understanding of Human Physical Interactions
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: cdowney@andrew.cmu.edu
Affiliations: Computer Science Department (CSD), Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, November 21, 2024, 12 – 1:30pm

Targeting is a central challenge in the design of humanitarian programs and policies: given available data, how does one identify the individuals and households with the greatest need? Here we show that machine learning, applied to non-traditional data from satellites and mobile phones, can improve the targeting of humanitarian interventions. Our analysis is based on data from several field-based projects — in Togo, Afghanistan, Bangladesh, and Kenya — that illustrate the promise, as well as some of the potential challenges, of this new approach to targeting. Collectively, the results highlight the potential for machine learning and new data sources to improve humanitarian efforts, particularly in crisis settings when traditional data are missing or out of date.

—

Joshua Blumenstock is a Chancellor’s Associate Professor at the U.C. Berkeley School of Information and the Goldman School of Public Policy. He is the Co-director of the Global Opportunity Lab and the Center for Effective Global Action. Blumenstock's research combines methods from computer science and empirical economics, with a focus on how novel data and technology can better address the needs of very poor and vulnerable populations. He has a Ph.D. in Information Science and a M.A. in Economics from U.C. Berkeley, and Bachelor’s degrees in Computer Science and Physics from Wesleyan University. He is a recipient of awards including the NSF CAREER award, the Intel Faculty Early Career Honor, and the U.C. Berkeley Chancellor's Award for Public Service. His work appear in general interest journals including Science, Nature, and PNAS, as well as prominent economics journals (e.g., American Economic Review, Review of Economic Studies) and computer science conferences (e.g., ICML, KDD, AAAI, WWW, CHI).

Event Type: Talks
Room Number: 358
Building: TCS Hall
Speaker's Name: JOSHUA BLUMENSTOCK
Speaker Website: www.jblumenstock.com
Speaker's Professional Title: Chancellor's Associate Professor, School of Information and Goldman School of Public Policy, Co-director, Global Opportunity Lab and Center for Effective Global Action, University of California, Berkeley
Talk Title: Applications of Machine Learning to the Targeting of Humanitarian Aid
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: poprocky@cs.cmu.edu
Affiliations: Computer Science Department (CSD), Human-Computer Interaction Institute (HCII), Machine Learning Department (MLD), Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Tuesday, November 19, 2024, 10:15 – 11:45am

ML Engineers from the Tesla AI team will be sharing exclusive content about building physical AI

► Talk 1
World Model for End-to-End Simulation and Evaluation

• Eric Huang — Senior ML Scientist, Autopilot

Tesla Autopilot engineers will share some recent work in generative modeling for high-fidelity multi-view video generation in simulation and model evaluation. One of the end-to-end driving model's biggest challenges is evaluating the system's performance in axes of both comfort and safety, since many undesirable behaviors are only observed in closed-loop controls. To reduce the gap between open-loop metrics and close-loop performance, we introduce a controllable, close-loop simulation system based on large-scale pre-trained video generative model that enables scalable model evaluation prior to deployment. We will demonstrate the system's capabilities under various driving conditions and challenges we encountered in practice, and how your research can make an impact in advancing the frontier of end-to-end simulation.

— Zhichun (Eric) Huang earned his Master’s degree in Machine Learning from CMU in 2021, where he collaborated with Prof. Zico Kolter on applications of deep equilibrium models. He joined Tesla Autopilot in 2022, leading efforts in scaling generative models for perception, end-to-end control and simulation.

► Talk 2
Scaling Generative Models for Decision Making Dhaval Shroff — Senior Staff Autopilot Machine Learning Engineer * Rohan Dhesikan* — Senior Autopilot Software Engineer – Autopilot

Tesla’s Autopilot engineers will share some recent work ingenerative modeling for high-fidelity multi-view video generation in simulation and model evaluation. One of the end-to-end driving model's biggest challenges is evaluating the system's performance in axes of both comfort and safety, since many undesirable behaviors are only observed in closed-loop controls. To reduce the gap between open-loop metrics and close-loop performance, we introduce a controllable, close-loop simulation system based on large-scale pre-trained video generative model that enables scalable model evaluation prior to deployment. We will demonstrate the system's capabilities under various driving conditions and challenges we encountered in practice, and how your research can make an impact in advancing the frontier of end-to-end simulation.

— Dhaval Shroff earned his Master's in Robotics Systems Development from CMU, focusing on classical computer vision and multi agent path planning. He has worked at Tesla on the Autopilot/ AI team for over 10 years and currently trains large models that predict end-to-end learned policies from video.

— Rohan Dhesikan earned both his undergraduate and Master's degrees in Electrical and Computer Engineering from CMU. He conducted research at the Robotics Institute, developing planning algorithms for aerial exploration of subterranean environments. After graduating, he joined Tesla's Autopilot team, where he focuses on training, optimization, and compression of large models for end-to-end self-driving systems.

► Talk 3
From Driver-Assist to Driver-Less Theo Guerin — Staff Autopilot Machine Learning Engineer - Autopilot * Amy Lin* — Autopilot Machine Learning Engineer - Autopilot

In this talk, Autopilot engineers will share the evolution of autonomy at Tesla, with an emphasis on high-precision products such as AutoPark and Actually Smart Summon. We will discuss the motivations behind and the challenges of training deep neural networks for real-time 3D perception and vehicle control.

— Amy Lin earned her undergraduate degree in Computer Science at CMU, where she conducted research in using deep learning for 3D reconstruction and pose estimation. She joined Autopilot's Fleet Learning team shortly after graduating, where she works on the high fidelity occupancy networks.

— Theo Guerin received his Bachelor of Science from Brown University in 2021, where he specialized in robotics. After two summer internships in the self-driving space and upon graduation, Theo joined the Motion Planning and Controls team in Autopilot. His work has spanned the planning and controls stack and most recently has been focused on the training and application of end-to-end models to driverless features like Actually Smart Summon.

Event Type: Career Presentation
Room Number: In Person
Building: ASA Conference Room, Gates Hillman 6115
Speaker's Name: ENGINEERING BREAKOUT SESSIONS - Deep Dive Technical Talks
Event Poster Title: Posters - All Tesla Presentations
Event Poster URL: www.cs.cmu.edu…
Event Website Title: SCS Careers
Event Website URL: www.cs.cmu.edu…

View Details

Tuesday, November 19, 2024, 1 – 3pm

The Autopilot Software and Autopilot Hardware teams from Tesla will be visiting Carnegie Mellon University for a one-of-a-kind recruiting event.  Meet with  the Tesla Autopilot team to experience an autonomous ride with our latest build. Explore firsthand the advanced AI piloting our autonomous vehicles.

SCS students with backgrounds in AI and/or Robotics (major profiles in ECE/CS/MechE) and an interest in revolutionizing the world’s future with the Autopilot team at Tesla are welcomed.

REGISTRATION steps

• Complete this Interest Form
• Submit your Resume Here
• Poster with Registration Links

View Details

Thursday, November 14, 2024, 12 – 1pm

Agenda

• 12:00 - 12:05 pm → Welcome and Introduction to ICF and NSF programs
• 12:05 - 12:40 pm → Panel Discusision
• 12:40 -   1:00 pm  →Q&A

View Details

Friday, November 22, 2024, 12:30 – 1:50pm

The almost unlimited multimedia content available on video-sharing websites has opened new challenges and opportunities for building robust multimodal solutions. This seminar will describe our novel multimodal architectures that (1) are robust to missing modalities, (2) can identify noisy or less discriminative features, and (3) can leverage unlabeled data. First, we present a strategy that effectively combines transformer architecture, share representations, reconstruction-based auxiliary tasks, and an optimized training mechanism for handling missing features. This problem is relevant since it is expected that the multimodal system will face cases with missing features due to noise or occlusion during inference. We implement this approach for audiovisual emotion recognition, achieving state-of-the-art performance. Second, we present a multimodal framework for dealing with noisy or less discriminative scenarios. This situation is commonly observed in audiovisual automatic speech recognition (AV-ASR) with clean speech, where the performance often drops compared to a speech-only solution due to the variability of visual features. The proposed approach is a deep
learning solution with a gating layer that diminishes the effect of noisy or uninformative visual features, keeping only useful information. The approach improves, or at least maintains, performance when visual features are used. Third, we discuss alternative strategies to leverage unlabeled multimodal data. A promising approach is to use multimodal pretext tasks carefully designed to learn better representations for predicting given task, leveraging the relationship between acoustic, lexical, and facial features. These models offer principled solutions to increase the generalization and robustness of common speech-processing tasks when using multimodal architectures.

—

Carlos Busso is an incoming Professor of the Language Technologies Institute at Carnegie Mellon University. He is currently a Professor at the University of Texas at Dallas’s Electrical and Computer Engineering Department, where he is also the director of the Multimodal Signal Processing (MSP) Laboratory. His research interest is in human-centered multimodal machine intelligence and application, with a focus on the broad areas of speech processing, affective computing, and machine learning methods for multimodal behavior modeling for socially interactive agents, and robust multimodal speech processing.

He is a recipient of an NSF CAREER Award. In2014, he received the ICMI Ten-Year Technical Impact Award. His students received the 2015 third prize IEEE ITSS Best Dissertation Award (N. Li), and the 2024 AAAC Student Dissertation Award (W.C. Lin). He also received the Hewlett Packard Best Paper Award at the IEEE ICME 2011 (with J. Jain), and the Best Paper Award at the AAAC ACII 2017 (with Yannakakis and Cowie). He received the Best of IEEE Transactions on Affective Computing Paper Collection in 2021 (with R. Lotfian) and the Best Paper Award from IEEE Transactions on Affective Computing in 2022 (with Yannakakis and Cowie). In 2023, he received the Distinguished Alumni Award in the Mid-Career/Academia category by the Signal and Image Processing Institute (SIPI) at the University of Southern California. He received in 2023 the ACM ICMI Community Service Award. He is currently serving as an associate editor of the IEEE Transactions on Affective Computing and as a member of the IEEE Speech and Language Processing Technical Committee (2024-2027). He is a member of AAAC and a senior member of ACM. He is an IEEE Fellow, and ISCA Fellow.

Event Type: Colloquium
Room Number: In Person
Building: Doherty Hall A302
Speaker's Name: CARLOS BUSSO
Speaker Website: personal.utdallas.edu…
Speaker's Professional Title: Professor, Language Technologies Institute, Carnegie Mellon University
Talk Title: Addressing Robustness in Multimodal Machine Learning
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: jclark2@andrew.cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: lti.cmu.edu…

View Details

Thursday, November 14, 2024, 3 – 4pm

The homeomorph Turán problem is the extremal hypergraph problem to determine the maximum number of facets in a pure d-dimensional simplicial complex on n vertices that does not contain a subcomplex homeomorphic to some fixed d-dimensional topological space. The d = 1 case of this problem (i.e. subdivisions of a fixed graph) was settled decades ago by Mader, and in the last few years there has been substantial progress for the d = 2 case by many different researchers. In this talk I will outline some of this recent progress and then turn attention to joint work with Marta Pavelka in which we study the homeomorph Turán problem for the d-dimensional sphere and tie the problem to an important enumeration question of Gromov.

4:00 pm → Tea and Cookies sponsored by Jane Street, Math Lounge Wean 6220 (bring your own mug if you have one)

Event Type: Seminars
Room Number: In Person
Building: Wean Hall 8220
Speaker's Name: J. ANDREW NEWMAN
Speaker Website: sites.google.com…
Speaker's Professional Title: Postdoctoral Research and Instructor, Department of Mathematical Sciences, Carnegie Mellon University
Talk Title: The Turán problem for spheres
For More Information: bbukh@math.cmu.edu
Affiliations: Computer Science Department (CSD), Tepper School of Business
Organization(s): Department of Mathematical Sciences
Event Website Title: Event Website
Event Website URL: aco.math.cmu.edu…

View Details

Wednesday, November 13, 2024, 3 – 4pm

If anyone would like to present a paper for discussion, please be in touch.

Event Type: Talks
Room Number: In Person
Building: Mehrabian Collaborative Innovation Center 2201
For More Information: ericzeng@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): CyLab

View Details

Friday, November 15, 2024, 2:30 – 3:30pm

Robotic assisted surgery (RAS) systems incorporate highly dexterous tools, hand tremor filtering, and motion scaling to enable a minimally invasive surgical approach, reducing collateral damage and patient recovery times. However, current state-of-the-art telerobotic surgery requires a surgeon operating every motion of the robot, resulting in long procedure times and inconsistent results. The advantages of autonomous robotic functionality have been demonstrated in applications outside of medicine, such as manufacturing and aviation. A limited form of autonomous RAS with pre-planned functionality was introduced in orthopedic procedures, radiotherapy, and cochlear implants. Efforts in automating soft tissue surgeries have been limited so far to elemental tasks such as knot tying, needle insertion, and executing predefined motions. The fundamental problems in soft tissue surgery include unpredictable shape changes, tissue deformations, and perception challenges.

My research goal is to transform current manual and teleoperated robotic soft tissue surgery to autonomous robotic surgery, improving patient outcomes by reducing the reliance on the operating surgeon, eliminating human errors, and increasing precision and speed. This presentation will discuss our novel strategies to overcome the challenges encountered in soft tissue autonomous surgery. Presentation topics will include a robotic system for supervised autonomous laparoscopic anastomosis, ultra-minimally magnetic surgery, and end-to-end imitation learning of surgical tasks and procedures.

—

Axel Krieger, PhD, joined the Johns Hopkins University in the Department of Mechanical Engineering in July 2020. He is leading a team of students, scientists, and engineers in the research and development of robotic systems for surgery and interventions. Projects include the development of a surgical robot called smart tissue autonomous robot (STAR) and the use of 3D printing for surgical planning and patient specific implants. Professor Krieger is a recipient of the NSF CAREER award and an inventor of over thirty patents and patent applications. Licensees of his patents include medical device start-ups Activ Surgical and PeriCor as well as industry leaders such as Siemens, Philips, and Intuitive Surgical.

Before joining the Johns Hopkins University, Professor Axel Krieger was Assistant Professor in Mechanical Engineering at the University of Maryland and Assistant Research Professor and Program Lead for Smart Tools at the Sheikh Zayed Institute for Pediatric Surgical Innovation at Children’s National. He has several years of experience in private industry at Sentinelle Medical Inc and Hologic Inc. His role within these organizations was Product Leader developing devices and software systems from concept to FDA approval and market introduction. Dr. Krieger completed his undergraduate and master’s degrees at the University of Karlsruhe in Germany and his doctorate at Johns Hopkins, where he pioneered an MRI guided prostate biopsy robot used in over 50 patient procedures at three hospitals.

Event Type: Seminars
Room Number: In Person
Building: Tepper Building 1403
Speaker's Name: AXEL KRIEGER
Speaker Website: imerse.lcsr.jhu.edu
Speaker's Professional Title: Associate Professor, Department of Mechanical Engineering, Johns Hopkins University
Talk Title: Autonomous Robotic Surgery: Science Fiction or Reality?
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: kbuss@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Wednesday, November 13, 2024, 12 – 1:30pm

We’ve seen an explosion of investment in AI infrastructure and foundation models, yet a commensurate surge of popular AI products has yet to follow. In this talk, we’ll explain the dearth of existing products, examine building blocks for AI native AI products, and present methodologies for exploring new AI product ideas. Attendees will leave armed with a sharper view of problems preventing progress and tools to start tackling those problems themselves.

—

Nick Hobbs is a designer creating new generative AI products at Google. Prior to his current work, Nick founded Brief — a subscription news service — which he sold to Twitter. His experience with early-stage product development extends to mobile — he led the iOS Google App team during the early days of the iPhone — and emerging technologies like self-driving cars as the first product manager on the Google self-driving car team.

Faculty Host: Nik Martelaro

Event Type: Seminars
Room Number: In Person
Building: Newell-Simon 4305
Speaker's Name: NICK HOBBS
Speaker's Professional Title: Designer, New Generative AI Products, Google
Talk Title: How to Build a New Generation of AI Native Products
For More Information: rpetaeiz@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Thursday, November 21, 2024, 5pm

This thesis explores diverse decision-making strategies for autonomous agents by examining knowledge-dependent and data-dependent approaches in stationary and dynamic data environments. We address five core research problems across three thematic areas: knowledge-dependent, stationary data-dependent, and evolving data-dependent decision-making.

We first investigate knowledge-driven decision-making within robotic swarms, characterizing vulnerabilities in systems governed by consistent rule-following behaviors. Here, adversarial agents exploit predictable patterns in swarm actions, demonstrating the risks inherent in structured decision protocols.

The second theme shifts to stationary data, where we leverage expert demonstrations to inform decision-making. Two studies illustrate this: one develops socially intelligent robotic interactions through imitation learning, while the other unifies multiple methods of reward parameter estimation in expert policies under a common optimization framework. Together, these studies highlight how structured, stable examples can enhance model precision and generalization in social and reward-driven contexts.

The final focus is on continual learning to mitigate catastrophic forgetting in dynamic environments. We propose methods for incremental object detection, first by incorporating new object classes in images without destabilizing existing knowledge, and then extending this framework to videos in a domain-incremental setting. By leveraging temporal context, this approach aims to support models capable of adapting to varied, evolving environments, crucial for applications like autonomous driving and surveillance.

Thesis Committee
Kris Kitani (Chair)
Maxim Likhachev
Deva Ramanan
Hironobu Fujiyoshi (Chubu University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3001
Speaker's Name: NAVYATA N. SANGHVI
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Knowledge and Data Dependence in Decision-Making
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, November 21, 2024, 9am

Robotic simulation, planning, estimation, and control, have all been built on top of numerical optimization. In this same time, modern convex optimization has matured into a robust technology delivering globally optimal solutions in polynomial time. With advances in differentiable optimization and custom solvers capable of producing smooth derivatives, convex modeling has become fast, reliable, and fully differentiable. This thesis demonstrates the effectiveness of convex modeling in areas such as Martian atmospheric entry guidance, nanosatellite space telescope pointing, collision detection, contact dynamics of point clouds, online model learning, and finally, a derivative-free method for trajectory optimization that leverages modern parallelized simulation. In all of these domains, the reliability and speed of differentiable convex optimization enables real-time algorithms that are rigorous, performant, and easy to understand and modify.

Thesis Committee
Zachary Manchester (Chair)
Zico Kolter
Changliu Liu
Tom Erez (Google DeepMind Robotics)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: KEVIN TRACY
Speaker Website: kevintracy.info
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Differentiable Convex Modeling for Robotic Planning and Control
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, November 18, 2024, 9am

In an age of autonomous driving and robotics, we are increasingly engaging with robots that deploy autonomous assistance. Cognitive science and human-computer interaction literature tells us that, when we apply autonomy in assistive settings, we are often augmenting the user’s sense of agency over the system. Sense of agency is a phenomenon from cognitive science that represents the experience of being in control of one’s environment. If our assistive systems had some notion of the user’s sense of agency, we could deploy robotic assistance that maintains the user’s feeling of control over their assistive device, which is an important factor in technology adoption and use. To build systems that deploy assistance while maintaining a strong user sense of agency during the interaction, we must understand assistive robot users’ experiences of control in the context of the assistance the robot provides.

Although we know that sense of agency experiences are prevalent in human-machine interactions, assistive robotics literature rarely addresses this concept explicitly. Gap 1: The assistive robotics literature lacks an understanding of the subjective sense of agency experience among users with disabilities during assistive interactions. Gap 2: Additionally, we lack measures of sense of agency that can be computed online, which might enable us to monitor and maintain sense of agency throughout assistive interactions. Gap 3: Finally, assuming we had a real-time measure for sense of agency, it is unclear how we could use the measures to dynamically adjust the level of assistance throughout assistive interactions. This thesis addresses the above three research gaps in two assistance domains: physical and cognitive assistance.

To quantitatively address Gap 1, we run experiments in both assistance domains that investigate the effects of level of assistance on the user’s sense of agency. To qualitatively address Gap 1, we will employ semi-structured interviews and scenario-based design techniques to query participants with disabilities about common scenarios in which assistive technology violates their sense of agency. For Gap 2, we propose real-time proxy metrics for sense of agency based on cognitive science theories in both the physically and cognitively assistive domains. Finally, for Gap 3, we implement novel assistive paradigms that dynamically adjust the level of assistance based on an estimation of the user’s sense of agency, which is computed from our proxy metrics. These novel paradigms are then compared to conventional assistive systems which do not consider sense of agency.

In this thesis, we contribute a quantitative and qualitative understanding of the subjective experience of sense of agency among assistive robot users with disabilities in the cases of physical and cognitive assistance. Additionally, we contribute some of the first real-time proxy metrics for sense of agency in assistive robotics. Finally, we contribute novel assistance paradigms that consider the user’s sense of agency when providing assistance to the user, and we compare these paradigms to conventional assistance approaches. We hope to build assistive systems that respect the user’s sense of agency, potentially leading to improved assistive interactions that leave users feeling in control of their assistive devices.

Thesis Committee
Henny Admoni (Chair)
Aaron Steinfeld
Patrick Carrington
Elaine Short (Tufts University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305
Speaker's Name: MAGGIE COLLIER
Speaker Website: collierma.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Leveraging Sense of Agency to Improve the Experience of Control Over Assistive Robots
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, November 19, 2024, 1pm

Proteins are essential regulators of cellular processes. Intrinsically disordered proteins (IDPs), despite lacking stable tertiary structures under physiological conditions, play crucial yet often underexplored roles in biological processes. With recent experimental advances like DisP-seq for probing IDP-DNA binding, there is a pressing need for efficient, interpretable computational methods to identify sequence determinants of IDP-DNA interactions and analyze their cooperative effects on gene regulation. To address this, we develop U-DisCo, a novel deep learning model that predicts base-resolution IDP-DNA binding profiles directly from DNA sequences. Leveraging a U-Net architecture, U-DisCo captures both local base-level interactions and long-range dependencies up to 20 kilobases with high accuracy and computational efficiency, outperforming the baseline BPNet. By incorporating ATAC-seq data, U-DisCo enables robust cross-cell type predictions as a multimodal framework. U-DisCo identified key IDP-binding motifs, revealing distinct interaction patterns and cooperative behaviors across different IDPs. Interestingly, we observed short-range interactions for motifs like AP-2 and EWS-FLI1 (single GGAA motif), while others exhibited independent, enhancer-like functions. Further analysis revealed that some IDPs favored certain strand orientations, suggesting their involvement in specific regulatory mechanisms. Overall, U-DisCo is the first computational approach to explore multiple IDPs within a single cell type, offering a versatile framework for studying IDP-mediated gene regulation and genome-wide regulatory elements.

Thesis Committee
Jian Ma (Chair)
Lei Li

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: HONGWEI TU
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Master's Student, Computer Science Department, Carnegie Mellon University
Talk Title: Learning Genome-Wide Interactions of Intrinsically Disordered Proteins with DNA Using U-DisCo
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: amalloy@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Monday, November 18, 2024, 3:30pm

As a result of recent advancements in foundation models, including large vision-language models, several researchers have explored methods of combining multiple modalities of data as inputs for visual question answering. One key application of visual question answering in the context of the healthcare domain is automated medical report generation, where x-ray images and text-based symptom data for a patient might be provided as inputs, with the intention of generating a relevant medical report as an output. However, very few studies analyze the performance of these models alongside uni-modal encoder-decoder models, and even fewer compare the performance of these multimodal models depending on whether they are provided symptom information as an input. Furthermore, past studies often use simple evaluation metrics that look at n-gram overlaps, such as BLEU and ROUGE scores, which are not effective for generative foundation models that can generate different sentences with the same semantic meaning.

In this paper, we present two main contributions. First, we compare the performance of a variety of approaches for generating medical reports on a dataset of Chest X-Ray medical reports, including an encoder-decoder model, a multimodal model without symptom data, and a multimodal model with symptom data. Second, we design a new metric for evaluating the similarity between generated and reference medical reports using medical term transformation, sentence embeddings, and cosine vector similarity. Our results show that multimodal approaches to medical report generation far outperform encoder-decoder approaches, and providing symptom data slightly improves accuracy for generated medical reports. We also find that our evaluation metric more closely measures similarity between generated and reference medical reports than standard techniques, as evidenced by both quantitative and qualitative case-study comparisons.

This research pushes the frontier of medical report generation by further reinforcing the accuracy benefits of using multimodal models with symptom inputs and introducing a more comprehensive, customized scoring metric for evaluating generated medical reports.

Thesis Committee
Min Xu (Chair)
Martin Zhang
Bryan Wilder

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Newell-Simon 4305
Speaker's Name: ANKIT GUPTA
Speaker's Professional Title: Master's Student, Computer Science Department, Carnegie Mellon University
Talk Title: Analyzing Multimodal Machine Learning Model Performance and Evaluation Metrics for Medical Report Generation
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: amalloy@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Tuesday, November 19, 2024, 4:30 – 5:30pm

Join us for a distinguished lecture featuring

    Ashok Elluswamy, Vice President of AI Software, and
    Pete Bannon, Vice President of Hardware Engineering

Get insights from industry leaders about the latest in AI technology.

Event Type: Career Presentation
Room Number: In Person
Building: Rashid Auditorium, Gates Hillman 4401
Speaker's Name: TESLA DISTINGUISHED LECTURE
Event Poster Title: Posters - all Tesla Presentations
Event Poster URL: www.cs.cmu.edu…
For More Information: careers@cs.cmu.edu
Event Website Title: SCS Careers
Event Website URL: www.cs.cmu.edu…

View Details

Wednesday, November 20, 2024, 12 – 1pm

Intermittent computing is gaining popularity in applications that rely on batteryless energy-harvesting devices, which experience frequent and arbitrary power failures. To ensure progress, programs running on these devices rely on runtime support to save state and re-execute after a power failure.

In this talk, we study the logical underpinning of intermittent computing and model checkpoint, crash, restore, and re-execution operations as computation on crash types. We draw inspiration from adjoint logic to reason about the relationship between persistent and transient memories through (re-)execution, checkpointing, and restoration. Using crash types, we show that any correct intermittent execution can be simulated by a continuously-powered execution.

Presented as part of the PLunch Seminar Series
and in Partial Fulfillment of the CSD Speaking Skills Requirement

Event Type: Speaking Skills
Room Number: In Person
Building: Gates Hillman 8102
Speaker's Name: MYRA DOTZEL
Speaker Website: www.andrew.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Modal Crash Types for Intermittent Computing
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Wednesday, November 13, 2024, 12 – 1pm

In this talk I will survey (block) Krylov methods for solving systems of linear equations. Such methods have close connections with conjugate gradient, Lanczos method, Wiedemann's algorithm, and can be viewed as generalizations of the power method. What I know about these mostly occur over reals, so I'll mainly focus on the continuous setting. However, I will also attempt to discuss how such algorithms work over finite fields, and connections with matrix rank, eigenvalues, and minimum polynomials.

Event Type: Seminars
Room Number: In Person
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: RICHARD PENG
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Associate Professor, Computer Science Department, Carnegie Mellon University
Talk Title: Krylov Space Methods
For More Information: wrhe@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: www.cs.cmu.edu…

View Details

Tuesday, November 19, 2024, 3:30 – 4:30pm

Social networks scaffold the diffusion of information on social media. Much attention has been given to the spread of true vs. false content on social media, including the structural differences between their diffusion patterns. However, much less is known about how platform interventions on false content alter the diffusion of such content. In this work, we estimate the causal effects of a novel fact-checking feature, Community Notes, adopted by Twitter (now X) to solicit and vet crowd-sourced fact-checking notes for false content. An important aspect of this feature is its use of a bridging-based decision algorithm whereby fact-checking notes are shown only if they are seen as broadly informative and helpful by users from across the political spectrum. To estimate the causal effect of bridging-based fact-checking, we gather detailed time series data for 40,000 posts for which notes have been proposed and use synthetic control methods to produce counterfactual estimates of a range of diffusion-based outcomes. We find that attaching fact-checking notes significantly reduced the reach of and engagement with false content. In reducing reach, we observe that diffusion trees for fact-checked content are less deep, but not less broad, than synthetic control estimates for non-fact-checked content with similar reach. This finding contrasts notably with differences between false vs. true content, where false information diffuses farther, but with structural patterns that are otherwise indistinguishable from those of true information, conditional on reach.

—

Johan Ugander is an Associate Professor at Stanford University in the Department of Management Science & Engineering, within the School of Engineering. His research develops algorithmic and statistical frameworks for analyzing social networks, social systems, and other large-scale social and behavioral data. Prior to joining the Stanford faculty he was a postdoctoral researcher at Microsoft Research Redmond 2014-2015 and held an affiliation with the Facebook Data Science team 2010-2014. He obtained his Ph.D. in Applied Mathematics from Cornell University in 2014. His awards include a NSF CAREER Award, a Young Investigator Award from the Army Research Office (ARO), several Best Paper Awards, and the 2016 Eugene L. Grant Undergraduate Teaching Award from the Department of Management Science & Engineering.

Faculty Host:  Nihar Shah

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Gates Hillman 8102 and Zoom
Speaker's Name: JOHAN UGANDER
Speaker Website: web.stanford.edu…
Speaker's Professional Title: Associate Professor, Department of Management Science & Engineering, Institute for Computational & Mathematical Engineering, School of Engineering, Stanford University
Talk Title: Bridging-based fact-checking moderates the diffusion of false information on social media
For More Information: sharonw@cs.cmu.edu
Affiliations: Computer Science Department (CSD), Machine Learning Department (MLD)
Organization(s): School of Computer Science

View Details

Monday, November 18, 2024, 12 – 1pm

Reliability, security, and resiliency of the cyber-physical systems has been an active area of research, in one form or the other. With every advancement in enabling technology comes new challenges in terms of reliability and resiliency. The past few decades have seen a sharp rise in communication and computational technologies applied to large-scale systems. The cyber-physical layer grows exponentially as these systems undergo a transformation to an extensive dependence on distributed resources with associated digital control and communication interfaces, many of these located beyond the grid edge.

Consequently, the problem of securing cyber-physical systems has become technically more challenging on multiple fronts: (1) Ubiquity of internet of things (IoT) and industrial IoT (IIoT) devices increases the vulnerability and threat landscape exponentially; (2) The increased variability due to the introduction of distributed resources makes contingency analysis challenging; (3) Despite advancements in computation technology, the sheer size of the cyber landscape often renders computational processes infeasible; and (4) The tight coupling of the cyber and physical components makes pure information technology (IT) or operational technology (OT)-based security technologies less effective.

To address these challenges, current state-of-the-art approaches often incorporate AI/ML technologies to detect anomaly in the system operational data. This has shown significant promise with high true positive rates when the available data adequately captures the operating conditions of the system. However, performance degradation sets in when new operation regimes, not represented in the available dataset, are encountered. Unfortunately, this is well known and exploited by malicious attackers to mimic extreme event situations, forcing the system into a panic mode.

In this talk, I will discuss some recent results leveraging the cyber-physical nature to develop algorithms for holistic vulnerability assessment of these systems. This approach seamlessly merges data-driven machine learning models, for the cyber layer, with domain knowledge physics-based models, for the physical layer, to simultaneously achieve high accuracy and high generalizability for understanding the effects of both known and unknown extreme events. This knowledge promises to enable the development of resilient control algorithms to enable the systems to survive malicious or natural extreme events while sustaining critical functions.

—

Dr. Olugbenga Moses Anubi is an Associate Professor of Electrical and Computer Engineering at the Florida State University, with affiliations with the Center for Advanced Power Systems (CAPS) and the Center for Intelligent Systems, Controls and Robotics (CISCOR). He is the director of the Resilient and Autonomous Systems Lab (RASLab). He was a Lead Control Systems Engineer at the GE Global Research Center, NY. His work has resulted in more than 15 patents and several recognitions including the 2023 FAMU-FSU College of Engineering Faculty Rising Star Award, the GE Technology Award (Physical+Digital), the Connected Controls Technical Achievement Award, the Whitney Award and the Dushman Technology Award. His work is supported by DOE, ONR, DARPA, DOT and several industry partners. His research interests include control of autonomous systems and resilient cyber-physical systems with applications to energy, transportation and other critical infrastructures. He is an inducted senior member of the National Academy of Inventors (NAI) and a senior member of IEEE.

Host:  Michael Lisanti

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Panther Hollow Room 4105, Mehrabian Collaborative Innovation Center
Speaker's Name: OLUBENGA MOSES ANUBI
Speaker Website: eng-web1.eng.famu.fsu.edu…
Speaker's Professional Title: Associate Professor, and Director, Resilient and Autonomous Systems Lab, Department of Electrical and Computer Engineering, Florida State University
Talk Title: Generative Vulnerability Assessment of cyber-physical Systems
For More Information: bethbuch@andrew.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): CyLab, School of Computer Science
Event Website Title: Event Website
Event Website URL: www.cylab.cmu.edu…

View Details

Wednesday, November 13, 2024, 2 – 3pm

I’ll describe the design of Spanner’s new storage engine, Ressi, which replaced untyped sorted string tables (inherited from Bigtable) with a strongly typed SQL-native representation. Live migration of 6 exabytes of data and multiple billion-user products to the new engine posed unique challenges. Sound methodology from experimental computer science was the key to its success.

The simplicity and power of declarative queries combined with strongly consistent transactional semantics has scaled to many thousands of machines running an aggregate of over 2 billion queries per second for some of the largest applications in the world. While challenges emerge as we continue to scale, I argue that the dominant obstacle to achieving zettabyte scale databases is in experimental methodology rather than in the underlying technical problems themselves.

—

David F. Bacon leads Google’s Spanner storage engine team, responsible for over 70% of the total fleet-wide cost of Spanner. His current work includes compression, RAM efficiency, ASIC support for databases, protection against “mercurial cores”, and tools for predicting fine-grained impact of software and hardware changes. Prior to Google, he worked at IBM Research on programming language design, optimization, and hardware synthesis. He was named an ACM Fellow for pioneering work on real-time garbage collection. He holds a Ph.D. from UC Berkeley, and his thesis work on optimizing virtual functions is used in most modern C++ and Java compilers. He has published over 80 papers and holds 29 patents.

Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: DAVID F. BACON
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Principal Engineer, Leads, Spanner Storage Engine Team, Google
Talk Title: Evolution of the Storage Engine for Spanner, an Exabyte-scale Database System
For More Information: db-www@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: db.cs.cmu.edu…

View Details

Tuesday, November 12, 2024, 12:30 – 1:50pm

The patent process can be confusing at best and overwhelming at worst. This presentation aims to cut through the confusion and provide a clear, actionable understanding of how to protect your innovations. In this talk Frank will walk users through one of his patents from ideation to issuance.

Key Takeaways:

• Understanding the types of patents
• Steps to go from Idea to patent submission
• Pitfalls to avoid during the Patent submission process

View Details

Wednesday, November 20, 2024, 3:30pm

Large text-to-image models learn from training data to synthesize “novel” images, but how the models use the training data remains a mystery. The problem of data attribution is to identify which training images are influential for generating a given output. Specifically, removing influential images and retraining the model would prevent it from reproducing that output image. Unfortunately, directly searching for these “ground truth” influential images is computationally infeasible since it would require repeatedly retraining from scratch.

My research aims to develop effective and scalable attribution methods and evaluation schemes for large text-to-image models. First, I present a computationally feasible attribution benchmark for large text-to-image models. Through “customization” methods, we define ground truth attribution labels by creating synthetic images computationally influenced by exemplar images. This scheme allows efficient evaluation by avoiding retraining repeatedly. Next, I will present a new data attribution approach for general text-to-image models. We simulate unlearning the synthesized image, find training images that are forgotten after the unlearning process, and label these as influential.

Finally, I will present ongoing work on improving the efficiency of attribution algorithms and propose a future research direction for developing interpretable attribution algorithms.

Thesis Committee
Jun-Yan Zhu (Chair)
Deva Ramanan
Ruslan Salakhutdinov
Alexei A. Efros (University of California, Berkeley)
David Bau (Northeastern University)

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: SHENG-YU WANG
Speaker Website: peterwang512.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Data Attribution for Text-to-Image Models
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, November 14, 2024, 4:30 – 5:30pm

We construct a quantum oracle relative to which BQP = QCMA but quantum-computation-classical-communication (QCCC) key exchange, QCCC commitments, and two-round quantum key distribution exist. We also construct an oracle relative to which BQP = QMA, but quantum lightning (a stronger variant of quantum money) exists. This extends previous work by Kretschmer [Kretschmer, TQC22], which showed that there is a quantum oracle relative to which BQP = QMA but pseudorandom state generators (a quantum variant of pseudorandom generators) exist.

We also show that QCCC key exchange, QCCC commitments, and two-round quantum key distribution can all be used to build one-way puzzles. One-way puzzles are a version of "quantum samplable" one-wayness and are an intermediate primitive between pseudorandom state generators and EFI pairs, the minimal quantum primitive. In particular, one-way puzzles cannot exist if BQP = PP.

Our results together imply that aside from pseudorandom state generators, there is a large class of quantum cryptographic primitives which can exist even if BQP = QCMA, but are broken if BQP = PP. Furthermore, one-way puzzles are a minimal primitive for this class. We denote this class "CountCrypt".

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Blelloch-Skees Conference Room, Gates Hillman 8115 and Zoom
Speaker's Name: ELI GOLDIN
Speaker Website: eligoldin.com
Speaker's Professional Title: Ph.D. Student, Department of Computer Science, New York University
Talk Title: CountCrypt: Quantum Cryptography between QCMA and PP
For More Information: qvd@andrew.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): CyLab, School of Computer Science
Event Website Title: Series Website
Event Website URL: sites.google.com…

View Details

Friday, November 15, 2024, 12:30pm

As a researcher who has been working on speech recognition for more than 10 years, I'd like to introduce three main projects I was involved in, and share the lessons I learned from it with the students.

—

Yu Zhang is currently a research scientist at OpenAI. Previously, he was a research scientist at Google Brain working on speech processing. Yu received his Ph.D degree in computer science from Massachusetts Institute of Technology in 2017. During his Ph.D, he worked on improving speech recognition performance. Yu is a main contributor to Google's next generation RNNT ASR model and Tacotron based text-to-speech system. Yu led Google's Universal Speech Model efforts. After joining OpenAI, Yu is now a key contributor to GPT-4o that can reason across audio, vision, and text in real time.

Event Type: Colloquium
Room Number: In Person
Building: Doherty Hall A 302
Speaker's Name: YU ZHANG
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Research Scientist, Open AI
Talk Title: Hearing the AGI: from GMM-HMM to GPT-4o
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: jclark2@andrew.cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: lti.cmu.edu…

View Details

Saturday, November 16, 2024, 9am – 3pm

Join us for the finals of Carnegie Mellon University's annual Hack-a-Startup Pitch Competition!

Come support 15 talented finalist teams as they showcase the innovative startups they’ve developed in just two weeks, all competing for nearly $10,000 in cash prizes. Don’t miss this inspiring day of creativity, ambition, and entrepreneurial spirit.

Tentative Schedule

•   9:00 AM  →  Registration
•   9:30 AM  →  Welcome & Event Logistics
•   9:40 AM  →  Pitches Begin
• 12:00 PM  →  Judge Deliberation
• 12:15 PM  →  Top 3 + Patient Safety Technology Award Winners Announced
• 12:30 PM  →  Open Networking + Food
•   2:00 PM  →   Event End

View Details

Monday, November 18, 2024, 2pm

Resource-bound analysis aims to infer symbolic bounds of worst-case resource usage (e.g., running time, memory, and energy) of programs as functions of program inputs. Resource analysis has numerous applications, including job scheduling in cloud computing and prevention of side-channel attacks. Various resource analysis technique have been developed, and they have unique strengths and weaknesses that complement each other. (Automatic) static resource analysis, which analyzes the source code of programs, is sound: if it successfully infers a cost bound, it is guaranteed to be a valid bound. However, every static analysis technique is incomplete: there exists a program that the analysis technique cannot handle. Meanwhile, data-driven analysis, which statistically analyzes cost measurements obtained by running programs on many inputs, can infer a candidate cost bound for any program. However, it does not guarantee soundness of inference results.

To overcome limitations of individual analysis techniques, I propose hybrid resource analysis, which integrates two complementary analysis techniques to retain their strengths while mitigating their respective weaknesses. The user first specifies which analysis techniques are used to analyze which code fragments and quantities. Hybrid analysis then performs its constituent analysis techniques on their respective code fragments and quantities. Finally, their inference results are combined into an overall cost bound.

The development of hybrid resource analysis has been driven by the desire to go beyond Automatic Amortized Resource Analysis (AARA), a state-of-the-art type-based static resource analysis technique. I start by proving polynomial-time completeness of AARA. I next introduce Bayesian data-driven analysis, which conducts Bayesian inference on cost measurements to infer a posterior distribution of symbolic cost bounds. I then present the first hybrid resource analysis, Hybrid AARA, followed by a discussion of its limitations. To overcome these limitations, I introduce the second hybrid resource analysis, resource decomposition. I additionally describe Swiftlet, which instantiates the resource-decomposition framework with AARA and Bayesian resource analysis. Finally, for proposed work, my collaborators and I plan to develop data-driven-analysis for statistically inferring not only a worst-case symbolic cost bound but also a worst-case input generator, which is a program generating worst-case program inputs of various sizes.

Thesis Committee
Jan Hoffmann (Chair)
Feras Saad
Matt Fredrikson
Nadia Polikarpova (University of California, San Diego)

Additional Information

Event Type: Thesis Proposals
Room Number: In Person
Building: ASA Conference Room, Gates Hillman 6115
Speaker's Name: LONG PHAM
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Hybrid Resource-Bound Analyses of Programs
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Friday, November 15, 2024, 12:30pm

Autonomous robots—including self-driving vehicles, sidewalk delivery robots, and more—must navigate among humans in a safe and socially-compliant manner. Current approaches for building and evaluating such autonomous systems rely on data-driven techniques; however, a generalization gap emerges, as methods trained in these traditional paradigms are unable to cope with unexpected real-world scenarios. Therefore, this thesis aims to develop improved evaluation settings and methodologies to increase and assess robustness in autonomous robot navigation against these challenges.

This thesis proposal describes several completed works that assess and improve different facets of robustness in autonomy:

• For robustness against perception errors affecting downstream motion prediction, we construct a framework for converting top-down pedestrian trajectory datasets into a more challenging first-person view perspective. We then develop a correction module to account for the resulting errors, trained end-to-end with trajectory prediction approaches.
• For robustness against out-of-distribution, safety-relevant scenarios, we create a hierarchical characterization method which leverages counterfactual probes to find hidden safety-relevant scenarios in large datasets. We then address the induced generalization gap by incorporating the characterizations into downstream trajectory prediction models' inductive biases.
• For robustness against adversarial, safety-critical scenarios, we develop a reactive, skill-based adversary policy which leverages a learned, multi-faceted criticality objective to perturb existing scenarios. We then train ego policies in a closed-loop manner against these generated scenarios, demonstrating improved downstream ego performance.

View Details

Monday, November 18, 2024, 2 – 3pm

In this talk, I will present the research and development efforts in computer vision and large language models (LLMs) at PAII, the U.S. research lab of PingAn Group. We focus on applying these technologies across diverse financial scenarios, such as autonomous car damage estimation, elderly care, and satellite image analysis for agricultural insurance. I will delve into the technical aspects of multi-modal LLM development for AI-enabled medical diagnostics and share insights into current research directions and concerns in multi-modal large language models (MLLMs).

—

Mei Han is the director of PAII Inc, US Research Lab of Ping An Group. Her research interests are in the fields of video analysis, geometric modeling, computer vision and multi-modality large language model. She has published more than 120 papers at top-tier conferences/journals and filed 80 patents. Her research has made impacts to Ping An Property Insurance, Car Insurance, Smart City, Ping An Bank, Investment and many other internal and external business partners. Prior to joining Ping An, she held the position of research scientist at Google and research staff member at NEC Labs America. She’s made significant contributions to the products of Google Earth, StreetView, Photos, Image Search, YouTube, Cloud. Technologies developed by Dr. Han and her colleagues are at the core of the company Vidient's innovative surveillance products. Mei Han holds Doctorates in Robotics and Computer Science from Carnegie Mellon University and Tsinghua University. She has served as the Industry Chair for CVPR’22, CVPR’19, CVPR’17, CVPR’16, ACM MM’17, 3DV’16 and ICASSP’21, and Area Chair for CVPR'22, WACV'25, WACV'24, ICASSP'25, ICASSP'24. She was a Keynote Speaker at IEEE GlobalSIP’19.

Faculty Host:  Jian Ma

Event Type: Seminars
Room Number: In Person
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: MEI HAN
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Director, PAII Inc, US Research Lab, Ping An Group
Talk Title: AI/LLM Applications at PAII: AI-Enabled Medical Diagnostics at Scale
For More Information: aricarte@andrew.cmu.edu
Affiliations: Computational Biology Department (CBD), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI)
Organization(s): School of Computer Science

View Details

Tuesday, November 19, 2024, 9am

This dissertation addresses the problem of enhancing the efficiency and trustworthiness of neural models.

On the efficiency side, my work focuses on the problem of Hyper-parameter tuning (HPT), a crucial but expensive step to optimize the performance of Neural Networks (NN). In this dissertation, I propose two new HPT methods, HyperJump and TrimTuner. Both methods aim to maximize model’s quality while reducing training and optimization time. Despite using different techniques to solve the optimization problem, both rely on low-fidelity observations (e.g., training with sub-sampled datasets) to efficiently identify promising configurations to be then tested via high-fidelity observations (e.g., using the full dataset).

On the trustworthiness side, the focus of this work is on adversarial robustness and uncertainty estimation. On the adversarial robustness front, I investigate the challenges that arise when performing HPT for models that are adversarially trained, showing that, although the complexity of the HPT problem is exacerbated in adversarial settings, tuning the HPs independently for standard and Adversarial Training (AT) can improve accuracy. To reduce HPT costs, I propose leveraging cheap AT methods to obtain inexpensive, yet highly correlated, estimations of the quality of more robust methods. This approach, combined with a recent multi fidelity optimizer, enhances significantly the efficiency of the HPT process. On the uncertainty estimation front, I introduce Error-Driven Uncertainty Aware Training (EUAT), a method that strives to ensure that the model is highly uncertain when making inaccurate predictions and confident when making accurate ones. During training EUAT selectively employs two loss functions based on the model’s performance on training examples: i) it reduces uncertainty for correct predictions and ii) increases uncertainty for mispredictions, while preserving the accuracy.

This current proposal presents the research results already achieved so far and plans for future work in order to complete my dissertation.

Thesis Committee
David Garlan (Chair)
Paolo Romano (Co-Chair, Instituto Superior Técnico, University of Lisbon)
Matt Fredrikson
Steven Wu
Chrysoula Zerva (Instituto Superior Técnico, University of Lisbon)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: TCS Hall 358 and Zoom
Speaker's Name: PEDRO GONÇALO BRAVO MENDES
Speaker Website: web.tecnico.ulisboa.pt…
Speaker's Professional Title: Ph.D. Student, Ph.D. Program in Software Engineering, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Techniques for Enhancing the Efficiency and Trustworthiness of Neural Networks
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: aroudebu@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Thursday, November 21, 2024, 4:30 – 6pm

Large language models (LLMs) have demonstrated remarkable capabilities in generating coherent text and completing various natural language tasks. Nevertheless, their ability to perform complex, general reasoning has remained limited. In this talk, I will describe OpenAI's new o1 model, an LLM trained via reinforcement learning to generate a hidden chain of thought before its response. We have found that the performance of o1 consistently improves with more reinforcement learning compute and with more inference compute. o1 surpasses previous state-of-the-art models in a variety of benchmarks that require reasoning, including mathematics competitions, programming contests, and advanced science question sets. I will discuss the implications of scaling this paradigm even further.

—

Noam Brown is a research scientist at OpenAI investigating reasoning and multi-agent AI. He co-created Libratus and Pluribus, the first AIs to defeat top humans in two-player no-limit poker and multiplayer no-limit poker, respectively, and Cicero, the first AI to achieve human-level performance in the natural language strategy game Diplomacy. He has received the Marvin Minsky Medal for Outstanding Achievements in AI, was named one of MIT Tech Review's 35 Innovators Under 35, and his work on Pluribus was named by Science as one of the top 10 scientific breakthroughs of 2019. Noam received his PhD from Carnegie Mellon University.


About the Lecture:  The Katayanagi Lectures recognize the best and the brightest in the field of computer science and are presented by the School of Computer Science at Carnegie Mellon University in cooperation with the Tokyo University of Technology (TUT). The lectures recognize both senior and junior talent.  The series were established through a gift from Japanese entrepreneur and education advocate, Mr. Koh Katayanagi, who founded TUT and several other technical institutions in Japan over many multiple decades.
 

Event Type: SCS Distinguished Lectures
Room Number: In Person
Building: Rashid Auditorium, Gates Hillman 4401
Speaker's Name: NOAM BROWN
Speaker Website: noambrown.github.io
Speaker's Professional Title: Research Scientist, OpenAI
Talk Title: Learning to Reason with LLMs
For More Information: scs-dls@cs.cmu.edu
Affiliations: Computational Biology Department (CBD), Computer Science Department (CSD), Human-Computer Interaction Institute (HCII), Language Technologies Institute (LTI), Partnerships, Robotics Institute (RI), Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Thursday, November 14, 2024, 4 – 5pm

Summer Undergraduate Research Fellowships (SURF) are open to undergraduates interested in research in any discipline, whether in the arts, humanities, sciences, social sciences, engineering or business.  Awards are $4,500 for eight to ten weeks of full-time summer research on campus with a faculty mentor; students can join an existing project or they can propose one of their own. The fellowships are competitive and, as per the website description, include additional application materials beyond the requisite proposal. Students may work alone or in a group, but must apply for the SURF as an individual — no group awards are available. 

• Student Deadline for Proposal Submission ⇒  Wednesday, 19 February 2025. 
• Deadline for faculty mentors to approve the project and submit their letter of recommendation  ⇒  Sunday, 23 February 2025.
• Decisions will be announced in late-March 2025.

View Details

Friday, November 15, 2024, 1:30 – 2:30pm

Quantifying greenhouse gas sources and sinks from the atmosphere is essential for assessing the success of emission reduction efforts. Observations of atmospheric greenhouse gases have become major tools for tracking these fluxes. Because atmospheric transport links emissions to downwind concentration changes, this presents an inverse problem. In this presentation, I’ll introduce the general methodology for quantifying emissions, highlighting key successes and ongoing challenges. I’ll also provide an overview of diverse applications, from tracking emissions at individual facilities to monitoring trends on a global scale.

—

Brendan Byrne is a Senior Data Scientist at Qube Technologies, where he works on continuous monitoring of methane emissions. Previously, he was a Scientist at the NASA Jet Propulsion Laboratory (JPL), where he led research on the global carbon cycle and its implications for climate change. He is an expert on flux inversion analyses, where surface-atmosphere trace gas fluxes are estimated from atmospheric concentration data and holds a Ph.D. from the University of Toronto and both M.Sc. and B.Sc. degrees from the University of Victoria.

To Sign up or the Seminar

Event Type: Seminars
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: BRENDAN BYRNE
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Senior Data Scientist Qube Technologies
Talk Title: Quantifying greenhouse gas emissions through atmospheric inversion systems
For More Information: tsukiant@andrew.cmu.edu
Affiliations: Machine Learning Department (MLD)
Organization(s): Statistics and Data Science Department, Stamps
Event Website Title: Event Website
Event Website URL: www.cmu.edu…

View Details

Friday, November 15, 2024

At the Human-AI Integration Lab (HAL), we design, build, and study human-AI systems that amplify human potential by enhancing cognitive, creative, physical, and social capabilities. This talk will explore the challenges of developing these systems while addressing the unpredictability of AI and the complexity of human behavior. I will present our recent work on generative AI for learning and well-being, as well as AI systems that augment creativity, improve accessibility, and provide physical task guidance in mixed reality. The talk will conclude with my lab's vision to transform AI into a universal tool for augmenting human potential.

—

Misha Sra is the John and Eileen Gerngross Assistant Professor at the University of California, Santa Barbara where she directs the Human-AI Integration Lab in the Computer Science department. Misha received her PhD from the MIT Media Lab in 2018, advised by Prof. Pattie Maes in the Fluid Interfaces Group. She has published at the most selective HCI, VR, and machine learning venues such as CHI,UIST, IEEE VR, CVPR, ECCV and EMNLP where she received four best paper awards and honorable mentions. MIT selected her as an EECS Rising Star in 2018. In 2023, she was awarded an NSF CAREER Award for her work in Human-AI Interaction design. Her research has received extensivecoverage from leading media outlets (e.g., from MIT News, The Verge, PCMag) and has drawn the attention of industry research, such as Toyota Research, Samsung Research, and Unity 3D.

Faculty Host: David Lindbauer

In Person and Zoom (Internal to CMU) and Livestream (All Guests)

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Mauldin Auditorium, Newell-Simon 1305 and Livestream
Speaker's Name: MISHA SRA
Speaker Website: sites.cs.ucsb.edu…
Speaker's Professional Title: John and Eileen Gerngross Assistant Professor, Computer Science Department, University of California, Santa Barbara
Talk Title: Design of Human-AI Systems to Augment Human Potential
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: rpetaeiz@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Thursday, November 14, 2024, 3pm

One reason for the widespread adoption of SAT solvers is that they are trustworthy: their answers can be checked with verified software. In particular, many SAT solvers can emit proof certificates of unsatisfiability that are efficient to check. However, the standard proof systems in use today struggle to succinctly express proofs for problem instances with a high degree of symmetry.

In this talk, we discuss our recent work on proof checking tools for the substitution redundancy (SR) proof system. We discuss a few problems that admit short SR proofs, as well as how we can express and check those proofs. Our verified proof checker was developed in the Lean theorem prover.

Presented in Partial Fulfillment of the CSD Speaking Skills Requirement

Event Type: Speaking Skills
Room Number: In Person
Building: Newell-Simon 3305
Speaker's Name: CAYDEN CODEL
Speaker Website: crcodel.com
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Verified Substitution Redundancy Checking for SAT Solving
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Tuesday, November 12, 2024, 12 – 1:15pm

The availability of large code corpora, coupled with advances in machine learning have made it possible to build software development tools grounded in statistical principles, paving way for capabilities that were not possible with technologies that relied on program semantics alone. In the last couple of years, large language models have taken the world by storm, and have shown remarkable ability to create, explain, critique and debug code. In this talk, I will provide an overview of this area both from a research as well as technology point of view. I’ll describe how at Google we have been working on weaving AI capabilities in developer workflows, how we collect data, how we prioritize our work, and the impact that this work is showing. I will discuss how model quality and user experience interact in interesting ways that have a material implication on the success of such tools. I will share some thoughts on how the field is evolving from its erstwhile focus on code completion to higher-level tasks. Finally, I will discuss the role of benchmarks in our community, and what we can do to push the state of the art together at a higher pace.

—

Satish Chandra is a researcher at Google, where he applies machine learning techniques to improve developer productivity. His work has spanned many areas of programming languages and software engineering, including program analysis, type systems, software synthesis, bug finding and repair, software testing and, of course, application of AI to software development. His research has been widely published in leading conferences in his field. Satish Chandra obtained a PhD from the University of Wisconsin-Madison, and a B.Tech from the Indian Institute of Technology-Kanpur, both in computer science. He is an ACM Distinguished Scientist and an elected member of WG 2.4

In Person and Zoom Participation. See announcement.

Event Type: Lecture
Room Number: In Person and Virtual - ET
Building: TCS Hall 358 and Livestream
Speaker's Name: SATISH CHANDRA
Speaker Website: sites.google.com…
Speaker's Professional Title: Principal Engineer, Core Developer Group, Google
Talk Title: AI in Software Engineering at Google
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: lv2c@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: s3d.cmu.edu…

View Details

Tuesday, November 12, 2024, 10am

Leading design in large organizations often feels like fitting a square peg into a round hole—it doesn’t always come together seamlessly. Yet, how we show up and deliver can determine whether we earn a seat at the table.In this session, we’ll explore key principles to help designers navigate the unique complexities of leadership in business environments. While not exhaustive, these principles offer actionable insights, especially for designers transitioning into leadership roles. Topics include:

• Aligning design with business priorities
• Partnering effectively in product decisions
• Thriving within organizational constraints
• Balancing collaboration and transformation

Come ready to share your experiences and engage in a lively discussion as we unpack what it takes to lead design successfully in a business context.

—

Aradhana Goel is a seasoned Product and Design leader with over 20 years of experience addressing complex business challenges, guiding diverse product and service portfolios, building high-performing teams, and driving meaningful impact for users and stakeholders. Most recently, she served as Head of Product Design and Behavioral Science for Bayer Pharmaceuticals’ Integrated Care Team, where she led the vision, strategy, development, and launch of digital solutions in Cardiovascular and Women’s Health. Previously, as Head of New Product Development at Capital One, she oversaw the growth and scale of the innovative Capital One Café concept.Aradhana has also held prominent consulting roles, including over 12 years with IDEO and MAYA (now BCG Platinion). An entrepreneur at heart, she co-founded IDEO’s office in India, establishing its core offerings, business model, and client relationships, while building and leading a local design team.

Currently, Aradhana is an adjunct faculty member at Carnegie Mellon University’s Integrated Innovation Institute, where she teaches Methods & Tools for Product Innovation. She also serves as a strategic advisor and leadership coach for organizations, helping leaders and teams work more effectively individually, collaboratively, and at scale.

Aradhana frequently speaks on design thinking and product innovation, and has presented at conferences and events such as the Digital Therapeutics Conference, Lesbians Who Tech, Penn State’s Stuckeman School, CMU’s HCI Institute, and TEDx India.

Faculty Host: Laura Vinchesi

In Person and Zoom. See announcement.

Event Type: Talks
Room Number: In Person and Virtual - ET
Building: Newell-Simon Hall 3305 and Zoom
Speaker's Name: ARADHANA GOEL
Speaker Website: www.cmu.edu…
Speaker's Professional Title: Adjunct Faculty, Integrated Innovation Institute, Carnegie Mellon University
Talk Title: The Business of Leading Design
For More Information: cbstumpf@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science

View Details

Tuesday, November 12, 2024, 9am

Foundation models have become central to modern machine learning, introducing new capabilities such as zero-shot and few-shot learning. These paradigms offer significant advantages in terms of generalization and robustness. However, as their applications expand, so do the associated risks and limitations, including potential privacy violations and intellectual property concerns.

This thesis proposal seeks to analyze foundation models through the lens of information theory, focusing on three interconnected areas: robustness, privacy, and copyright protection. First, we investigate how contrastive vision-language models respond to distribution shifts, where training and test data differ significantly, proposing information-theoretic measures to quantify and enhance model robustness. Second, we examine how existing unlearning algorithms fail to remove private information from models, and develop efficient auditing tools for machine unlearning. Third, we explore the memorization issues in large language models, advocating for a compression-based adversarial prompt method to measure memorization, which is essential in identifying copyright infringement.

For future work, we first extend our afore-mentioned memorization measurement to continuous soft token space. Previously, this measurement requires a computation in the discrete space, which is not computationally efficient. We aim to achieve a more efficient algorithm by searching in the continuous space, and develop corresponding information-theoretic measure. Second, we investigate the suboptimal out-of-distribution (OOD) tokenization issue in large language models (LLM), and propose to improve OOD tokenization via a sparse optimal transport token translation.

Thesis Committee
J. Zico Kolter (Chair)
Graham Neubig
Ruslan Salakhutdinov
Lester Mackey (Microsoft Research, New England)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates Hillman 4405 and Zoom
Speaker's Name: ZHILI FENG
Speaker Website: zhilif.github.io
Speaker's Professional Title: Ph.D. Student, Machine Learning Department, Carnegie Mellon University
Talk Title: Leveraging Information Theoretic Tools for Foundation Model Analysis
For More Information: stidle@andrew.cmu.edu
Affiliations: Machine Learning Department (MLD)
Organization(s): School of Computer Science

View Details

Tuesday, November 12, 2024, 8am

Deep learning has made significant progress to analyze an unprecedented amount of rich visual information from the real world to enable applications such as robotics, surveillance, and public safety monitoring. The successful deployment of deep learning techniques highly relies on the availability of large-scale domain-specific annotated data. However, these constraints are unlikely to be met in many real-world scenarios. In practice, various domain gaps exist between the training and test data. Test data are typically drawn from out-of-domain distributions, encompassing novel viewpoints, varied noise conditions, and diverse scenes. In addition to the diversity in visual representations, deep learning models trained on fixed, closed-set labels may not meet the query requirements of arbitrary text prompts from users. Additionally, novel vocabularies may not be accessible during training. To enable the deployment of a robust visual perception system, learning generalized feature representations during training is crucial.

In this thesis, with the goal of developing systems which can generalize to novel viewpoints, scenes and vocabularies, we explore different representation learning methods based on Siamese learning, masked visual modeling, and generatively pre-training. This thesis consists of three parts. The first part conducts robust semantic instance segmentation for videos and 3D data. We aim to learn feature representations that are invariant to various viewpoints and noise conditions via Siamese learning. We propose to leverage temporal consistency for videos and spatial consistency for 3D volumetric images, such that the learned feature representations have strong generalization ability. In the second part, we tackle the problem of human action analysis, which requires the model to learn from dynamic cues. We propose representation learning techniques based on masked visual modeling, such that the model can learn better spatial-temporal context. We also exploit both RGB videos and 3D human meshes for robust multi-modal action analysis.  Finally, in the third part, we leverage generatively pre-trained vision-language models and develop systems that can handle novel vocabularies and text prompts. Our final goal is to build a robust system that can generalize to novel viewpoints, scenes, and vocabularies.  

Thesis Committee
Alexander Hauptmann (Chair)
Teruko Mitamura
Yonatan Bisk
Junwei Liang (Hong Kong University of Science and Technology)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: ASA Conference Room, Gates Hillman 6115 and Zoom
Speaker's Name: XIAOYU ZHU
Speaker Website: zgzxy001.github.io
Speaker's Professional Title: Ph.D. Candidate, Language Technologies Institute, Carnegie Mellon University
Talk Title: Learning Generalizable Visual Representations Towards Novel Viewpoints, Scenes and Vocabularies
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: StaceyYoung@cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science

View Details

Thursday, November 14, 2024, 10am

This thesis takes a comprehensive network discovery and computational approach to understanding state-sponsored operations on social media. The focus is on mapping out China’s public diplomacy and covert information operations to find key players, algorithmic tactics, and the role of bots and influencers in amplifying messaging. A core component of my research focuses on narrative shifts within these campaigns in response to global events, such as Russia's invasion of Ukraine. By temporally analyzing transformer-based embeddings, I map out how China’s narratives towards regional neighbors have shifted, particularly regarding its territorial claims around Taiwan and the South China Sea. I then analyze the relationship between social media and kinetic offline activity, analyzing China’s military flights around Taiwan and its social media messaging around the 2022 Nancy Pelosi visit. The culmination of my research contributes to constructing a scenario design for synthetic social media campaigns within a digital training environment, aimed at understanding state actor behavior in manipulating public opinion and policy. Lastly, I discuss policy implications and the impact of state-sponsored activity in a climate of declining platform moderation. My work advances the understanding of how state actors leverage and manipulate the information environment through diverse information maneuvers to further strategic objectives, providing a comprehensive view of the complex landscape of state-sponsored operations on social media.

Thesis Committee
Kathleen M. Carley (Chair)
Patrick Park
John Chin
Col. Matthew Benigni (US Army)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: TCS Hall 360 and Zoom
Speaker's Name: CHARITY S. JACOBS
Speaker Website: charityking2358.github.io
Speaker's Professional Title: Ph.D. Candidate, Ph.D. Program in Societal Computing, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Characterizing State-Sponsored Information Campaigns through Network Discovery
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: aroudebu@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Wednesday, November 13, 2024, 6 – 8:30pm

Spark – Startups and Emerging Companies is an innovative in-person networking event designed to provide a platform for small businesses and early-stage start-ups to engage with Carnegie Mellon University students. Companies will have the opportunity to network, recruit our students and share information about their company story, mission, and products.

Spark is hosted by the CPDC in collaboration with the Swartz Center for Entrepreneurship.

Event Type: Career Center Event
Room Number: In Person
Event Website Title: SCS Careers
Event Website URL: www.cs.cmu.edu…

View Details

Friday, November 22, 2024, 1 – 2:30pm

This virtual panel features 5 leaders from industry, finance, academia, and the not-for-profit sectors discussing matters related to sustainability.

Panelists include

• Arik Levinson  —  Professor, Georgetown University Economics Department
• Wallas E. Wiggins (CIT ’80 MSIA ’82)   —  Vice President Global Supply Management & Logistics, John Deere
• Fran Moore  —  Associate Professor, University of California Davis
• Katharine Kelleman  —  Chief Executive Officer, Pittsburgh Regional Transit
• Dr. Pascal Lavoie  —  Vice President, Breakthrough Technology Strategy, Alcoa

View Details

Monday, November 18, 2024, 4 – 5pm

The current leading approach for building multilingual NLP systems, such as machine translation, involves fine-tuning large foundation models for specific tasks. While this often results in strong performance, it can also diminish some of the core capabilities of the original model. This issue becomes particularly problematic when there is a mismatch between training and testing conditions. In this talk, we will highlight two critical abilities of large language models: multilinguality and context. Although foundation models are trained on a wide range of languages, their multilingual abilities often degrade when adapted to specific tasks. We will present two strategies to retain these abilities while incorporating new knowledge: an attribute-specific translation system using an attribute classifier and a multilingual summarization method based on query-key (QK) fine-tuning. Additionally, we explore how foundation models' ability to process long contexts can be leveraged in machine translation. We will present methods for integrating local context, specialized terminology, and multimodal information into translation systems.

—

Jan Niehues is a professor at the Karlsruhe Institute of Technology, where he heads the "AI for Language Technologies" research group. He earned his doctoral degree from the Karlsruhe Institute of Technology in 2014, specializing in "Domain Adaptation in Machine Translation."  He has contributed significantly to the field of machine translation and spoken language translation through extensive research conducted at several institutions including Carnegie Mellon University, LIMSI/CNRS, and Maastricht University.

His expertise spans various facets of machine translation, and he has played pivotal roles in numerous international projects, including the German-French Project Quaero, the EU H2020 project QT21, EU-Bridge, and Meetween. Notably, he serves as one of the organizers of the International Conference on Spoken Language Translation (IWSLT), showcasing his commitment to advancing the forefront of language technology and fostering global collaboration in the field.

Faculty Host:  Lei Li

Event Type: Talks
Room Number: In Person
Building: Gates Hillman 4301
Speaker's Name: JAN NIEHUES
Speaker Website: ai4lt.anthropomatik.kit.edu…
Speaker's Professional Title: Professor, and Lead, Artificial Intelligence for Language Technologies (AI4LT), Karulsruhe Institute of Technology
Talk Title: When Fine-Tuning Isn't Enough: Preserving Core Abilities in Multilingual NLP Models
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: jclark2@andrew.cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science

View Details

Friday, November 15, 2024, 6:30 – 11pm

Imagine an evening of magic and mystery, with the chance to decorate your own masquerade mask, indulge in delicious food, sip on refreshing non-alcoholic mocktails, dance the night away, and much more!

SCS Undergraduate Students →the ticket portal  goes live on Halloween, October 31 @ 5:15 pm!

Event Type: Special Events
Room Number: In Person
Speaker's Name: SCS Undergraduates!
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: scsball2024.my.canva.site

View Details

Monday, November 18, 2024, 12 – 1:30pm

Join President Farnam Jahanian for his annual update to the university community and hear about key university achievements and priorities.

Event registration close November 14th at 12:00 AM EST

 

Event Type: Special Events
Room Number: In Person and Virtual - ET
Building: Rangos Ballroom, Cohon University Center
Speaker's Name: PRESIDENT FARNAM JAHANIAN
Affiliations: Carnegie Mellon University
Organization(s): School of Computer Science

View Details

Friday, November 15, 6pm – Saturday, November 16, 2024, 10:30pm

Demosplash is one of the largest and longest-running demoparties in North America, catering both to newcomers to the demoscene as well as established artists. We show demos live on original hardware as much as humanly possible, offering a viewing experience that can't be matched by emulators or recordings. In addition to screenings, we feature a variety of events, including tech talks, workshops, and celebrations of retro gaming.

Stay tuned for this year's full events lineup.  We will of course have a whole roomful of retro gaming computers (Sharp X68000, Apple ][, Sega Genesis/Mega Drive, NES, and much much more) for anyone who wants to take a break from the demos or talks and play some video games!

REGISTER  |   Learn More

Demosplash 2024 is presented by the Carnegie Mellon University Computer Club.

Event Type: Fun
Room Number: In Person and VIrtual - ET
Building: Cohon University Center and Livestream
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: organizers@demosplash.org
Organization(s): CMU Computer Club, School of Computer Science
Event Website Title: Event Website
Event Website URL: www.demosplash.org

View Details

Tuesday, November 12, 2024, 12pm

Machine learning (ML) models increasingly make or inform decisions in domains such as financial markets, e-commerce, and fintech lending. Due to the high-stakes nature of the decisions being made, ML-based systems operating in these domains need to be able to reliably reason about different outcomes in the presence of various forms of strategic behavior. In particular, the principal (i.e. the person/institution deploying the ML model) often needs to reason about the incentives of other individuals in the market, leverage the information discrepancy between themselves and others, and learn causal relationships between observable features and outcomes of interest, often under partial feedback about the underlying learning/decision-making problem. Moreover, these issues often compound in realistic decision-making scenarios, requiring the principal to handle two or more of them simultaneously. In this thesis proposal, I will highlight the research I have done so far to address the additional challenges which come with learning and decision-making in online markets, and I will overview exciting directions for future research.

Thesis Committee
Nina Balcan (Co-chair)
Steven Wu (Co-chair)
Tuomas Sandholm
Michael I. Jordan (University of California, Berkeley / INRIA Paris)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: KEEGAN HARRIS
Speaker Website: keeganharris.github.io
Speaker's Professional Title: Ph.D. Student, Machine Learning Department, Carnegie Mellon University
Talk Title: Foundations of Data-Driven Decision-Making in Online Markets
For More Information: stidle@andrew.cmu.edu
Affiliations: Machine Learning Department (MLD)
Organization(s): School of Computer Science

View Details

Wednesday, November 13, 2024, 8am – 5pm

Our researchers will detail how they are crafting connections for software engineering by building partnerships to execute the SEI’s technical strategy to deliver tangible results. They will highlight methods, prototypes, and tools aimed at solving the most important problems facing the DoD, industry, and academia and advancing the state of the practice. Each session will explore how we were able to rapidly move research into practice, working together to develop practical solutions.

Join us to learn about our collaborative research advancements designed to address the four big software challenges for national defense and security: capability, timeliness, affordability, and trustworthiness. Learn how you can integrate these tools, techniques, and processes into your own work contexts and hear about the positive impact of these technologies on the DoD’s mission.

While the SEI Research Review 2024 is a free event, we ask that guests register to assure enough space for all participants.

REGISTER    |    Questions?

There will be no virtual component to this event, but we do plan to record the presentations and upload them to our website at a later date.

Event Type: Colloquium/Forum
Room Number: In Person
Building: Software Engineering Institute Building, 4500 Fifth Avenue
Speaker's Name: Hosted by the Carnegie Mellon University SEI
Affiliations: Carnegie Mellon University, Partnerships, Software Engineering Institute (SEI)
Organization(s): School of Computer Science
Event Website Title: Event Website and Registration
Event Website URL: www.eventbrite.com…

View Details

Thursday, November 14, 2024, 4:30 – 6pm

Camille Fournier is a distinguished technology leader who most recently served as Global Head of Engineering and Architecture for the Commercial and Investment Bank at JPMorgan Chase, where she led an 800+ person engineering organization. An accomplished author of the bestselling book "The Manager's Path" and "Platform Engineering: A Guide for Technical, Product, and People Leaders", she previously led Platform Engineering at Two Sigma and was the CTO of Rent the Runway.

Camille is known for her expertise in distributed systems, engineering management, and scaling both technology and teams. She has contributed to open source including the Apache ZooKeeper project, served on the founding technical oversight committee for the Cloud Native Compute Foundation, and is currently on the board of the ACM Queue. She holds a BS in Computer Science from Carnegie Mellon University, and an MS in Computer Science from the University of Wisconsin-Madison.

Event Type: SCS Distinguished Lectures
Room Number: In Person
Building: Rashid Auditorium, Gates Hillman 4401
Speaker's Name: CAMILLE FOURNIER
Speaker's Professional Title: Distinguished Technology Leader, and Bestselling Author
Talk Title: Engineering Your Own Path: From University to Universal Impact
Affiliations: Computational Biology Department (CBD), Computer Science Department (CSD), Human-Computer Interaction Institute (HCII), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI), Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Thursday, November 14, 2024, 8:30am – 4:30pm

Through collaboration with our partners, we actively engage with our Deployment Partner Consortium to identify real-world transportation needs, research project development and deployment, technology licensing and commercialization, student recruitiment for jobs and internships, as well as class and capstone projects.

This one-day, one-of-a-kind event will include a keynote from PA Department of Transportation Secretary Michael Carroll, Pennsylvania Turnpike Commission Chief Technology Officer Bob Taylor, plenary sessions,featuring national policy and deployment experts from industry, labor, government and nonprofit sectors, a research showcase (including live demos!) and a networking reception. This event will be held in person at CMU's Mill 19 campus in Hazelwood Green. RSVP now to hold your spot.

Putting our research and development and deployment approach into action – the annual deployment partner symposium provides an opportunity for interaction and discussion among deployment partners, researchers, and students (including our six nation-wide academic partners). Conversations are held not only during the formal program, panels and poster sessions, but also during breaks and over lunch.

Over 100+ participants are expected to attend, including UTC faculty and students along with our community, government and industry partners. Plan to join us!

By Invitation - RSVP | Additional Information

Presented by: Carnegie Mellon University’s Safety21 US DOT National University Transportation Center (UTC) — Director Raj Rajkumar | Traffic21 Institute Director Chris Hendrickson

Event Type: Conference/Workshop
Room Number: in Person
Building: Mill 19, Hazelwood Green
Affiliations: College of Engineering, Heinz College, Partnerships
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: safety21.cmu.edu…

View Details

Thursday, November 14, 2024, 3:30 – 4:30pm

What do we need to do to polish the write-up of our research papers? Papers that are otherwise worth publishing often get rejected due to presentation pitfalls. After about 1,000 rejections over the past 30+ years, some patterns started to emerge. In this talk, we discuss the three most important of these patterns, their potential remedies, as well as a list of an additional dozen of patterns and remedies.

—

Christos Faloutsos is a Professor in the Computer Science and Machine Learning Departments at Carnegie Mellon University and an Amazon Scholar. He is the recipient of the Fredkin Professorship in Artificial Intelligence (2020); he has received the Presidential Young Investigator Award by the National Science Foundation (1989), the Research Contributions Award in ICDM 2006, the SIGKDD Innovations Award (2010), the PAKDD Distinguished Contributions Award (2018), 31 “best paper” awards (including 8 “test of time” awards), and four teaching awards.

Eight of his advisees or co-advisees have received KDD or SCS dissertation awards. He is an ACM Fellow, he has served as a member of the executive committee of SIGKDD; he has published over 500 refereed articles, 17 book chapters and three monographs. He holds 12 patents (and several more are pending), and he has given over 50 tutorials and over 25 invited distinguished lectures. His research interests include large-scale data mining with emphasis on graphs and time sequences; anomaly detection, tensors, and fractals.

REGISTER
→ open to all SCS PhD Students and Post-docs

Additional Information

Host by the Computer Science Department

Event Type: Talks
Room Number: In Person
Building: ASA Conference Room, Gates Hillman 6115
Speaker's Name: CHRISTOS FALOUTSOS
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Fredkin University Professor of Computer Science Computer Science Department, Carnegie Mellon University
Talk Title: Lessons Learned from 1,000 Research Paper Rejections
Event Poster Title: Poster with QR Code
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD), Machine Learning Department (MLD)
Organization(s): School of Computer Science

View Details

Sunday, November 17, 2024, 1 – 2pm

Join us for this illuminating session with CMU’s NSF-funded AI Institute for Societal Decision Making (AI-SDM) as we explore Artificial Intelligence (AI) and Generative AI in our surrounding communities.

• REGISTER →  registration suggested for in-person attendance
• REGISTER  - Virtual Attendees  →  must advance to receive the meeting link to participate.

View Details

Wednesday, November 20, 2024, 9am – 6pm

Join in at this free and open event showcasing cutting-edge robotics, automation, and artificial intelligence (AI) solutions that are driving innovation across multiple industries.  Pittsburgh Robotics Discovery Day will feature over 130 exhibitors with live demonstrations of groundbreaking technologies that are transforming the way humans live, move and work.

From business leaders looking to innovate, to tech enthusiasts exploring the latest trends, to job seekers searching for emerging career opportunities, to those simply curious about how robotics and AI are reshaping our world - this event has something for everyone.  Be part of the action, engage with industry influencers, and witness firsthand the power of robotics and advanced technology in the heart of Western Pennsylvania.

Your journey into the future starts here—mark your calendar and join the thousands attending Pittsburgh Robotics Discovery Day!

Among the programs

► 9:00 am - 6:00 pm  →  Exhibitor Floor
     —  Get to our robotics, autonomy, and AI companies. Check out live demonstrations of their cutting edge technology. Visit manufacturers that are leveraging autonomous solutions to grow their business. Get to know organizations that are building community and career opportunities inside the ecosystem.

►  9:15 AM - 11:30 PM  →  Humans Behind the Robots
      —  Meet the people that are building Pittsburgh robotics, autonomy, and AI companies. Explore their career trajectories, efforts to diversity the workforce, and learn how you can join this fast-growing workforce.

►  9:15 AM - 11:30 PM  →  STEM Programming for Students
      —  Join in for a variety of engaging activities designed to introduce high school students to the robotics, autonomy, and AI ecosystem.

►  12:00 PM - 1:00 pm  →  New Economy Collaborative Annual Meeting
      —  Join our partner and facilitator of the region's $62M Build Back Better award, the New Economy Collaborative for a review of accomplishments.

►  2:00 - 4:00 pm  →  The Robotics Factory Startup Showcase
      — Join main stage sponsor, Innovation Works, and the Robotics Factory for a look at this year's startups emerging from the Accelerate Program.

and much more...

Multiple Zones to Explore

• Robotics Zone
• Career Pathways Zone
• Community Zone
• Manufacturing Zone
• Robotics Factory Startup Showcase

View Details

Wednesday, November 20 – Thursday, November 21, 2024

Watch for details.

Event Type: Conference/Workshop
For More Information: alacovar@andrew.cmu.edu | ckissell@andrew.cmu.edu
Affiliations: Partnerships, Robotics Institute (RI)
Organization(s): School of Computer Science, XRTC
Event Website Title: Center Website
Event Website URL: www.cmu.edu…

View Details

Thursday, October 3, 8:30am – Friday, October 4, 2024, 5pm

By invitation only.

Event Type: Corporate Visit
Room Number: In Person
For More Information: lalford@cs.cmu.edu
Affiliations: Partnerships, Research

View Details

Monday, September 30, 2024, 3:30 – 5pm

► Courtney Miller — Ph.D. Student, Software Engineering
     — Supporting the Sustainable use of Open Source Software
          Thesis proposal practice talk

In this dissertation, I study how to support and improve the processes used by developers facing open source dependency abandonment. Open source software forms the digital infrastructure that most modern software is built on, and expectations regarding ongoing maintenance are a widespread norm despite the reality that many open source projects become abandoned, even widely-used ones. However, supporting downstream users facing potential or actual dependency abandonment is a topic that has been largely neglected by open source sustainability research. 

To address this, I shift the focus of sustainability research from maintainers to users through the identification of both the challenges developers face when dealing with dependency abandonment as well as how they currently react at scale, improving the resiliency and adaptability of open source digital infrastructure and the many software supply chains that depend on them.  I demonstrate the efficacy of information transparency in encouraging timely downstream responses, highlight the unmet need for tooling to automate the identification of dependency abandonment, and investigate when and how to inform developers while minimizing notification fatigue, thus facilitating the sustainable use of open source. 

► Chenyang Yang — Ph.D. Student, Software Engineering
     — Machine Learning in Production: A Case Study on Amazon’s Product Rankers

I will talk about my experience as an intern working on Amazon's product rankers. In the talk, I will first present my observations on what challenges industry practitioners are faced with, and connect my observations to existing software engineering research on collaboration and evaluation challenges for ML in production. Finally, I will discuss how my observations motivate the design and implementation of a well-received tool for industry practitioners.

► Jonathan Aldrich — Professor, Software and Societal Systems Department
     — The Meerkat Vision: Language Support for Live, Scalable, Reactive Web Apps

The reactive programming paradigm has become ubiquitous for modern web and mobile app development. But despite its many benefits, today reactive programming is limited to the client and to data updates, leaving to the programmer the tedious and error-prone tasks of managing updates to code and synchronizing data updates between reactive clients and a server database. In this paper, we lay out the vision for Meerkat, a multitier, reactive, and live programming language designed to scale to the needs of modern applications. We introduce the language through a chat application which runs on our prototype implementation. We then describe approaches for modularizing and scaling Meerkat programs, customizing tradeoffs between consistency and availability, supporting local-first software and rich data models, and scaling live updates to full DevOps in software organizations. The Meerkat research program will enable a new era of developing apps that are more responsive, reliable, and evolvable than ever before.
→ This is a preview / practice talk for an Onward! 2024 paper to be presented next month.  It's an example of a "vision talk/paper" for a new research project. 

In Person and Zoom/Livestream Participation.  See announcement.

Event Type: Talks
Room Number: In Person and Virtual - ET
Building: TCS Hall 358 and Livestream
Speaker's Name: COURTNEY MILLER, CHENYANG YANG, and JONATHAN ALDRICH
Speaker's Professional Title: Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Three Talks
For More Information: sssg@cs.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: s3d.cmu.edu…

View Details

Monday, September 30, 2024, 4 – 5pm

In this talk, I’ll share some behind-the-scenes career highs and lows that led to my current (very fun) role at Los Alamos National Laboratory. I’ll also offer thoughts on creating career resilience by expanding the definition of “communities” and bringing an intentionality to how you engage with them.

—

Kary Myers is a fellow of the American Statistical Association and currently leads a group of ~40 scientists and R&D engineers in the Space Remote Sensing and Data Science Group at Los Alamos National Laboratory (LANL). With support from an AT&T Labs Fellowship, she earned her PhD from Carnegie Mellon’s Statistics and Data Science Department and her MS from their Machine Learning Department before joining LANL in 2006. She spent 15 years as a scientist in the Statistical Sciences Group at Los Alamos, including a few years as their deputy group leader and as the Deputy Director for Data Science in LANL’s Information Science and Technology Institute. She also served as LANL’s Intelligence and Emerging Threats Program Manager for Data Science.

She’s been involved with a range of data-intensive projects, from analyzing electromagnetic measurements, to aiding large scale computer simulations, to developing analyses for chemical spectra from the Mars Science Laboratory Curiosity Rover. She served as an associate editor for the Annals of Applied Statistics and the Journal of Quantitative Analysis in Sports, and she created and organizes CoDA, the Conference on Data Analysis, to showcase data-driven research from across the Department of Energy.

Event Type: Seminars
Room Number: In Person
Building: Poster Hall 153
Speaker's Name: KARY MYERS
Speaker Website: en.wikipedia.org…
Speaker's Professional Title: Space Remote Sensing and Data Science Group, Department of Statistics, Los Alamos National Laboratory
Talk Title: Community Detection (the life kind, not the network science kind)
For More Information: cveselkl@andrew.cmu.edu
Affiliations: Machine Learning Department (MLD)
Organization(s): Department of Statistics and Data Science
Event Website Title: Series Website
Event Website URL: www.cmu.edu…

View Details

Friday, September 27, 2024, 11:30am – 12:30pm

NewLimit is developing epigenetic reprogramming medicines to treat diseases with significant unmet needs.

Our Discovery Engine leverages functional genomics and machine learning to test thousands of reprogramming interventions per experiment. In each experimental cycle, we deliver unique combinations of reprogramming factors to cells, profile them using single-cell multi-omics, and employ machine learning models to infer how reprogramming impacts cellular function. We validate the most promising interventions with a suite of functional assays.

Rather than randomly exploring all possible combinations, our Engine intelligently selects interventions by integrating our knowledge of reprogramming factor biology with experimental results. This iterative approach ensures that each experimental cycle informs and enhances the next. This presentation will provide an overview of NewLimit’s discovery process and detail how we use machine learning and computational biology to accelerate therapeutic development.

Faculty Host: David Koes (PITT)

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Frick Fine Arts Building/Library, Room 125, 650 Schenley Drive and Zoom
Speaker's Name: GREGORY JOHNSON - To Be Rescheduled
Speaker Website: www.newlimit.com…
Speaker's Professional Title: Head of Machine Learning, NewLimit
Talk Title: Accelerating Therapeutic Discovery with Epigenetic Reprogramming and Machine Learning at NewLimit
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: kmg120@pitt.edu
Affiliations: Computational Biology Department (CBD)
Organization(s): School of Computer Science, CMU , PITT
Event Website Title: Series Webiste
Event Website URL: www.compbio.cmu.edu…

View Details

Friday, September 27, 2024, 1 – 10pm

Family Weekend brings together undergraduate and graduate students, families and friends for several days to experience the vibrancy of the Carnegie Mellon campus! Attending Family Weekend is free, however some events require registration or purchased tickets.

NOTE:
→  Please include your CMU student in your family’s registration.
→  The full event schedule can be found on the CMU Events App.

Event Type: Special Events
Room Number: In Person
Building: Various Locations and Events on Campus
Affiliations: Carnegie Mellon University
Organization(s): School of Computer Science
Event Website Title: Event Website and Schedule
Event Website URL: www.cmu.edu…

View Details

Friday, September 27, 2024, 11am – 12pm

The AI Institute for Societal Decision Making (AI-SDM) — which brings together AI and social sciences researchers to develop human-centric AI for societal good -- hosts student-led brainstorming discussion sessions. Our goal is to foster interdisciplinary collaboration and generate ideas on how AI can help solve societal problems, particularly from an angle of decision-making.

This will be our next informal brainstorm session.

RSVP  → for rough headcounts

In Person and Zoom Participation.  See announcement.

Event Type: Meeting
Room Number: In Person and Virtual - ET
Building: Gates Hillman 6121 and Zoom
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: shcarter@andrew.cmu.edu
Affiliations: Computer Science Department (CSD), Dietrich College of Humanities and Social Sciences, Human-Computer Interaction Institute (HCII), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI)
Organization(s): School of Computer Science, AI-SDM
Event Website Title: Additional Information
Event Website URL: www.cmu.edu…

View Details

Friday, September 27, 2024, 1pm

A fundamental goal of cognitive neuroscience has been understanding how the human visual cortex supports perceiving and interpreting visual information in the world around us. Traditional approaches to mapping the visual cortex have relied on manually assembled stimulus sets, often employing isolated objects in artificial contexts with simplified backgrounds. These approaches do not fully capture the complexity and richness of real-world visual experience, potentially biasing results and limiting our understanding of visual processing. My thesis introduces a suite of computational approaches leveraging naturalistic image stimuli to identify and characterize the high-level organization of visual information in the human brain. Specifically, I present:

1. Brain Diffusion for Visual Exploration (BrainDiVE): A method utilizing gradient guidance from a differentiable image-to-fMRI encoder and a pre-trained image diffusion model to generate naturalistic “most-exciting-inputs” that maximally activate specific brain regions.
2. Semantic Captioning Using Brain Alignments (BrainSCUBA): A technique unifying the embedding spaces of CLIP image and text embeddings with fMRI encoder weights to drive a vision-language model. This enables the generation of natural language descriptions of voxel-wise selectivity within the visual cortex.
3. Semantic Attribution and Image Localization (BrainSAIL): An approach employing vision foundation models and dense semantic features to localize activating objects within complex naturalistic images across higher-level visual areas.

View Details

Friday, September 27, 2024, 1:30 – 2:45pm

Virtual Reality (VR) offers compelling ways for people to interact with digital content in various domains such as education, gaming, communication, and design. However, VR platforms can be partially or completely inaccessible to people with disabilities. In particular, the expectation that people can move their bodies to interact with virtual content can present many challenges to people with limited mobility. As VR technology matures, and as commercial VR systems continue to grow in popularity, we have an opportunity to incorporate accessibility as a fundamental component in the design of VR systems and applications. In this talk, I will describe ongoing research that identifies and eliminates accessibility barriers that prevent people with limited mobility from engaging with VR.

—

Dr. Martez Mott is a Senior Researcher in the Ability group at Microsoft Research where he conducts research in the fields of human-computer interaction, accessibility, mixed reality, and human-centered AI. His research focuses on designing, implementing, and evaluating novel intelligent interactive technologies that are guided by scientific understandings of people’s experiences with computers and information. He is best known for his research on improving accessibility for people with limited mobility by identifying and dismantling accessibility barriers found in a range of computing technologies, including touch screens, gaze-based text entry, and virtual reality hardware. Dr. Mott is passionate about improving diversity in Computer and Information Science. He co-chaired the 2020 and 2021 CHI Mentoring Workshops at the ACM CHI conference and co-founded the Black
Researchers @ Microsoft Research employee resource group. Dr. Mott received a M.S. and Ph.D. in Information Science from the Information School at the University of Washington and a B.S. and M.S. in Computer Science from Bowling Green State University. He is the recipient of the 2024 Skip Ellis Early Career Award from the Computing Research Association.

Faculty Host:  Patrick Carrington

In Person and Zoom (Internal to CMU) and Livestream (All Guests)

Event Type: Seminars
Room Number: In Person
Building: Mauldin Auditorium, Newell-Simon 1305
Speaker's Name: MARTEZ MOTT
Speaker Website: www.microsoft.com…
Speaker's Professional Title: Senior Researcher, Ability Team, Microsoft Research Redmond
Talk Title: Designing Accessible Virtual Reality Experiences for People with Limited Mobility
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: rpetaeiz@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Friday, September 27, 2024, 12:30 – 1:50pm

Robust and comprehensive evaluations of generative AI models play a critical role in ensuring safe, and beneficial technologies being deployed in society. These evaluations rely heavily on semi-structured data annotated by humans. Both the data and human perspectives involved in the process, thus play a key role in what is taken as ground truth by models, and who the models are eventually able to serve. Historically, this perspective has been Western-oriented which leads to a lack of representation of global contexts and identities in models as well as evaluation strategies, and the risk of disregarding marginalized groups that are most significantly affected by implicit harms. Accounting for cross-cultural differences in interacting with technology is an important step for building and evaluating AI holistically.  We will talk through different strategies on broadening the scope of GenAI evaluations to be more inclusive of global perspectives and challenges.

—

Sunipa Dev is a Senior Research Scientist at Google Research in its Responsible AI and Human Centered Technologies organization, working at the intersection of language, society, and technology. Previously, she was an NSF Computing Innovation Fellow at UCLA, before which she completed her PhD from the University of Utah. Her research strives to ground evaluations of generative AI, especially language technologies in real world experiences of people, and foster inclusion of diverse, cross-cultural, and marginalized perspectives into AI pipelines. Her work has been awarded with an Outstanding Paper award ACL 2023, the NSF CI Fellows Award and DAAD AINet Award 2021, and she has been named one of 100 Brilliant Women in AI Ethics in 2022. She is also an advisor for the Widening NLP at *CL conferences which argues the importance of diversity and inclusion in NLP for better technologies of the
future.

Zoom Participation.  See announcement.

Event Type: Colloquium
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: SUNIPA DEV
Speaker Website: sunipa.github.io
Speaker's Professional Title: Senior Research Scientist, Google Research
Talk Title: Geocultural Inclusion and Representation in GenAI Evaluations
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: jclark2@andrew.cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: www.lti.cs.cmu.edu…

View Details

Friday, September 27, 2024, 2:30 – 3:30pm

Living materials represent a new frontier in engineering materials for robotic systems, incorporating biological living cells and synthetic materials into their design. These bio-hybrid robots are dynamic and intelligent and potentially harness living matter’s capabilities, such as growth, regeneration, morphing, biodegradation, and environmental adaptation. Such attributes position bio-hybrid devices as a transformative force in robotics development, promising enhanced dexterity, adaptive behaviors, sustainable production, robust performance, and environmental stewardship. Nature’s musculoskeletal design can act as an inspiration for artificial and living robots. We will touch on the transformative potential of living systems in crafting sophisticated, intelligent machines and delve into our recent advances in culturing biohybrid actuators and engineering bio-interfaces to produce contracting muscles and biohybrid robots.

—

Robert Katzschmann is an Assistant Professor of Robotics at ETH Zurich, where he leads the Soft Robotics Lab. He is associated with the Center for Robotics (RobotX), the ETH AI Center, and the Center for Learning Systems, a collaboration between ETH and the Max Planck Institute (MPI). His research primarily focuses on developing musculoskeletal robots that effectively combine soft, rigid, and living materials to perform complex tasks in real-world scenarios. Before he started his tenure at ETH Zurich, he served as the CTO of Dexai Robotics and as a Senior Applied Scientist at Amazon Robotics in the USA. He earned his Ph.D. in Mechanical Engineering from the Massachusetts Institute of Technology (MIT) in 2018 and his Diplom from the Karlsruhe Institute of Technology, Germany, in 2013. His work has been published in leading journals and conferences such as Nature, Nature Communications, Science Advances, and Science Robotics, as well as at prominent robotics conferences including ICRA, IROS, CoRL, ICLR, ICML, and RoboSoft. In addition to his research, he contributes as an editor for the International Journal of Robotics Research (IJRR) and has organized several workshops for the RoboSoft conference. He also works as an associate editor for ICRA, IROS, RoboSoft, and RSS, and he is an editorial board. member of npj Robotics. His research has been featured in premier news outlets such as the New York Times, Wall Street Journal, and BBC.

Faculty Host: Zachary Manchester

Event Type: Seminars
Room Number: In Person
Building: Tepper Building 1403
Speaker's Name: ROBERT KATZSCHMANN
Speaker Website: srl.ethz.ch…
Speaker's Professional Title: Assistant Professor, Institute for Robotics and Intelligent Systems, ETH Zürick
Talk Title: Can Robots Based on Musculoskeletal Designs Better Interact With the World?
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: kbuss@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Friday, September 27, 2024, 12 – 1pm

Currently, there is a substantial push towards autonomous vehicles (AV) in the market. However, largely or autonomous vehicles, though using extensive fault-tolerance e.g., in x-by-wire functions, are still not quite safe from an accidental faults’ perspective, leave alone malicious faults.

The growing number of recent AV architectures hinging on the advancements in AI/ML models, given their functional power, must be put in context with an equally significant number of related serious or fatal accidents. I advance by presenting important misconceptions and pitfalls originating from the use of inappropriate AI/ML techniques in the AV area, which may be the cause of serious accidents.

Furthermore, AV present an even greater threat surface to combined accidental faults and malicious attacks. These pitfalls have been very slowly recognized by car makers, with potentially harming results.

However, simultaneously securing unavoidable properties of Safety and Security is indeed a hard problem. I raise a bit of the curtain on how to break this chicken and egg dilemma—closing the safety-security gap—presenting some recent solution avenues based on cyber resilience, a core subject of my research.

Namely, I will discuss how in-car architectures might evolve to achieve fault and intrusion resilience of ECUs at system level, as well as reconciling the data-level stochastic nature of AI/ML paradigms with the determinism of driving control theory at system level, leveraging the best from both worlds: trustworthiness and intelligence.

—

Paulo Esteves-Veríssimo is a professor of Computer Science at KAUST, founding director of the Resilient Computing and Cybersecurity Center , and research fellow at the University of Luxembourg (LU). He is Fellow of IEEE and of ACM, author of over 200 peer-refereed publications and co-author of 5 books. He is currently interested in resilient computing, in areas like: SDN-based infrastructures; autonomous vehicles; distributed control systems; digital health and genomics; or blockchain and cryptocurrencies.

Faculty Host: William Sanders

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: PAULO ESTEVES-VERÍSSIMO
Speaker Website: cemse.kaust.edu.sa…
Speaker's Professional Title: Paulo Esteves-Veríssimo, Professor of Computer Science, and, Director of the Resilient Computing and Cybersecurity Center (RC3), King Abdullah University of Science and Technology (KAUST)
Talk Title: Safe and Secure AI/ML-driven Autonomous Vehicles? Not anywhere near yet...
For More Information: bethbuch@andrew.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): CyLab, Department of Electrical and Computer Engineering, School of Computer Science
Event Website Title: Event Website
Event Website URL: www.cylab.cmu.edu…

View Details

Tuesday, September 3, 2024, 12:30 – 1:50pm

The first seminar of the term will be Part 1 of presentations from our current students on their Summer internships and research.   The second seminar on September 10, will be Part 2 of our student presenters.

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Hamburg Hall 1002 and Zoom
Speaker's Name: JINGXIN SHI, LIMIN GE, AMAN PRIYANSHU, SURIYA AYYAMPERUMAL
Speaker's Professional Title: Master of Privacy Engineering Students, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Summer Internship/Research Presentations
For More Information: adamm2@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Event Website Title: Series Website
Event Website URL: privacy.cs.cmu.edu…

View Details

Tuesday, September 3, 2024, 10:50am

In my research, I want to design and develop practically efficient and provably secure distributed systems. To this end, I combine applied cryptography and formal verification tools. The goal of this proposal is to provide primitives that can be used in distributed systems, as well as methodologies to verify consensus protocols, to ensure the security, efficiency, and correctness of distributed systems. Specifically, the research focuses on three main areas: practical implementations of oblivious algorithms for Trusted Execution Environments (TEEs); practical and verified implementations of Garbled RAM; and methodologies to verify safety and liveness of consensus protocols.

TEEs can be used to offer efficient crash fault nodes with confidential computations; however, most TEEs implementations leak the page-level memory access pattern to the host machine where the TEE is running. Therefore, to guarantee confidential computations, TEE programs need not only correct implementations but also to be memory trace-oblivious. There has been extensive theoretical research in oblivious algorithms; however, there is a gap with practical implementations, particularly in the TEE setting. In this proposal, we aim to close this gap, providing an oblivious data structure library akin to C++'s STL, and extending it with oblivious graph algorithms.

TEEs require trust in the hardware manufacturer and a certain level of hardware/software integrity. In scenarios where this isn't possible, Garbled Circuits can be used to provide equivalent secure processor guarantees based on cryptographic assumptions. To achieve an efficient Garbled Circuit processor, Garbled RAM is necessary, and recent theoretical advancements suggest using tristate circuits to implement it. In this proposal, we aim to achieve concretely more efficient Garbled RAM constructions, as well as provide methodologies to verify the correctness of tristate circuits.

Finally, while TEEs and Garbled RAM aim to achieve secure computations, ensuring the correctness of the underlying protocols that use them is critical. We have previously developed a python DSL to verify safety properties of distributed system protocols. In this proposal, we aim to extend this framework to verify liveness properties of distributed system protocols, focusing on proof automation and generalization. 

Thesis Committee:
Elaine Shi (Co-chair)
Rodrigo Rodrigues (Co-chair, University of Lisbon, Instituto Superior Técnico)
Pedro Adão (University of Lisbon, Instituto Superior Técnico)
José Fragoso (University of Lisbon, Instituto Superior Técnico)
Aayush Jain
Andrew Miller (University of Illinois Urbana-Champaign)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Gates Hillman 8102 and Zoom
Speaker's Name: AFONSO TINOCO
Speaker Website: cmuportugal.org…
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Towards Practical and Verifiable Distributed Systems: Applications of Oblivious Algorithms, Garbled Circuits and Formal Methods+
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Tuesday, September 3, 2024, 11am

With modern robotic technologies, seafloor imageries have become more accessible to both researchers and the public. This thesis leverages deep learning and 3D vision techniques to deliver valuable information from seafloor image observations.

Despite the widespread use of deep learning and 3D vision algorithms across various fields, underwater imaging presents unique challenges, such as lack of annotations, color distortion, and inconsistent illumination, which limit the effectiveness of off-the-shelf algorithms. This thesis first develops a method to learn consistent deep features from underwater observations at different scales by introducing an algorithm based on contrastive learning. Without any annotations but geo-tags, we show that such deep features are efficient in inferring the vehicle location in a 3D reconstructed map at a small map scale. This thesis further tackles the more fundamental problem of building 3D reconstruction from raw images with heavy effects from light sources and medium interference. The following algorithms are developed to achieve seafloor 3D reconstruction with photorealistic quality: (i) Deep water true color restoration with NeRF (ii) Camera-light calibration for robotic platforms (iii) Dark environment relighting with 3DGS (iv) Recurrent underwater caustic removal.

From the real-world data we collected from robot deployments, we learned that simply enforcing consistency in geometry, appearance, and embedding does not necessarily produce accurate results. In this thesis, we develop algorithms and systems that wrap around 3D representations, which learn useful information from real-world underwater robotic data. These systems improve the ability of robots to navigate, map, and understand the environment, with zero annotations from humans.

Thesis Committee:
Matthew Johnson-Roberson (Chair)
Matthew O’Toole
Shubham Tulsiani
Katie Skinner (University of Michigan)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305
Speaker's Name: TIANYI ZHANG
Speaker Website: tyz1030.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Towards Underwater 3D Visual Perception
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, August 15, 2024, 1pm

Intelligent vehicles have been proposed as one path to increasing traffic safety and reducing on-road crashes. Driving “intelligence” today takes many forms, ranging from simple blind spot occupancy or forward collision warnings to distance-aware cruise and all the way to full driving autonomy in certain situations. Primarily, these methods are outward-facing and operate on information about the state of the vehicle and surrounding traffic elements. However, another less explored domain of intelligence is cabin-facing modeling information about the driver’s cognitive states.

In this thesis, we investigate the utility of a signal that can help us achieve cabin-facing intelligence: driver eye gaze. Eye gaze allows us to infer driver internal cognitive states and we explore how this can improve both autonomous driving methods and intelligent driving assistance. To enable this research, we first contribute DReyeVR, an open-source virtual reality driving simulator, which was designed with behavioural and interaction research priorities in mind but exists in the same experimental environments used by vehicular autonomy researchers, effectively bridging the two fields. We show how DReyeVR can be used to conduct psychophysical experiments by designing one to characterize the extent and dynamics of driver peripheral vision. We make good on the promise of bridging behavioural and autonomy research by using similar naturalistic driver gaze data to provide additional supervision to autonomous driving agents trained via imitation learning to mitigate causal confusion. We then turn to the assistive domain. First, we study false positives in a real-world dataset of forward collision warnings deployed in vehicles during a longitudinal study in-the-wild. We find that deploying FCWs purely based on scene physics without accounting for driver attention leads to overwhelming them with redundant alerts. We demonstrate a warning strategy that accounts for driver attention to explicitly model their hypothesis of other vehicles’ behaviour. Finally, we propose the shared awareness paradigm, a framework for continuously supporting driver situational awareness (SA) with an intelligent perception system. We track dynamic objects (e.g. vehicles, pedestrians etc.) and reason about them on two simultaneous fronts — drivers’ situational awareness and importance to driving safety for each object. To build the driver situational awareness model, we first collect data using a novel SA labeling method, to obtain continuous, per-object driver awareness labels along with their gaze, driving actions and the simulated world state. We use this data to learn a model that predicts drivers’ situational awareness of traffic elements given a history of their gaze and scene context. In parallel, we reason about the importance of objects in a counterfactual fashion by studying the impact of perturbing it on the ego vehicle’s motion plan. Finally, we put it all together, in an offline demonstration on replayed simulated drives to show how we could alert drivers of important objects they are unaware of.

We conclude by reflecting on how eye gaze can be used to model the internal cognitive states of human drivers, in service of improving both vehicle autonomy and driving assistance.

Thesis Committee:
Henny Admoni (Chair)
David Held
Nik Martelaro
Chien-Ming Huang (Johns Hopkins University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: ABHIJAT BISWAS
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Robotics Institute, Carnegie Mellon University
Talk Title: Eye Gaze for Intelligent Driving
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, August 5, 2024, 12 – 1pm

Join in for a panel series built to inform, inspire, and explore!  Learn about our intern and early career hiring process and what it is like to work and find community at Lockheed Martin.  Panelists will share their experiences, insights, and advice and on navigating job searches, a career, and life after college.

• Monday, August 5 — Engineering and Computer Science Related Majors → 12:00-1:00 pm & 8:00-9:00 pm ET - DAILY
• Tuesday, August 6 — Finance and Business Operations Related Majors → 12:00-1:00 pm & 8:00-9:00 pm ET - DAILY
• Wednesday, August 7 — Employee Groups and Belonging at LM → 12:00-1:00 pm & 8:00-9:00 pm ET - DAILY
• Thursday, August 8 — Engineering and Computer Science Related Majors → 12:00-1:00 pm & 8:00-9:00 pm ET - DAILY

View Details

Wednesday, August 7, 2024, 12 – 1pm

Large GPU clusters are increasingly becoming more heterogeneous due to advances in GPU design and incremental deployment of a mix of GPU types over time. Deep learning (DL) training jobs running on these GPU clusters can see varying job completion times depending on the resources allocated by the cluster scheduler and job hyper-parameters configured by users at the time of job submission. Sia is a cluster scheduler that (1) efficiently assigns heterogeneous GPU resources to elastic resource-adaptive DL training jobs, and (2) configures the job hyper-parameters to maintain high training efficiency for all running jobs without sacrificing the quality of trained models.

We will discuss challenges in optimizing resource-adaptivity for deep learning training (DLT) jobs on large clusters with many GPU types, and introduce a new scheduling formulation that efficiently matches DLT jobs and their configurations to GPU types and counts, while adapting to changes in cluster load and job mix over time. On job traces derived from real datacenters, Sia improves job completion times by 30-93% while using 12-60% fewer GPU hours. Furthermore, its scheduling policy is quick to evaluate and easily scales to GPU clusters with many GPU types and 1000s of GPUs.

—

Suhas Jayaram Subramanya is a final-year PhD student in the CS Department, advised by Prof. Greg Ganger. His primary research area is deep learning systems.

Zoom Participation.  See announcement.

Event Type: Talks
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: SUHAS JAYARAM SUBRAMANYA
Speaker Website: suhasjs.github.io
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Sia: Heterogeneity-aware, goodput-optimized ML-cluster scheduling
For More Information: karenl@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): Department of Electrical and Computer Engineering, PDL
Event Website Title: Series Website
Event Website URL: pdl.cmu.edu…

View Details

Thursday, August 8, 2024, 2 – 4pm

With the Robotics Institute Summer Scholar (RISS) Program ending, it’s time to celebrate our scholars and show appreciation for the faculty, RISS team members, and sponsors who have made this summer possible.

Our 46 scholars will be presenting their final research posters at this event. 
Snacks and drinks will be provided; mill around, eat, and learn about the amazing work our students have been doing this summer. We’re proud to showcase their hard work! 

Meet this year's scholars and learn more about their work at this event.

Event Type: Project Presentations
Room Number: In Person
Building: Collaborative Commons, ASA Conference Room, Gates Hillman 6115, and Gates Hillman 6121
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: riss-info@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Program Website
Event Website URL: riss.ri.cmu.edu…

View Details

Monday, August 5, 2024, 12:15pm

Multi-view stereo omnidirectional distance estimation usually needs to build a cost volume with many hypothetical distance candidates. The cost volume building process is often computationally heavy considering the limited resources a mobile robot has. We propose a new geometry-informed way of distance candidates selection method which enables the use of a very small number of candidates and reduces the computational cost. We demonstrate the use of the geometry-informed candidates in a set of model variants. We find that by adjusting the candidates during robot deployment, our geometry-informed distance candidates also improve a pre-trained model's accuracy if the extrinsics or the number of cameras changes. Without any re-training or fine-tuning, our models outperform models trained with evenly distributed distance candidates. Models are also released as hardware-accelerated versions with a new dedicated large-scale dataset. The project page, code, and dataset can be found here.

Additionally, the field of reinforcement learning (RL) has transformed strategic game play, enabling AI agents to achieve superhuman performance in games like chess, Go, and StarCraft. These advancements underscore the potential of RL in handling complex, long-horizon planning tasks against intelligent adversaries with a large search space of potential winning strategies. This project introduces a new competitive multi-phasic strategy game with partial observability and specialized units, demonstrating the use of RL to achieve winning performance. Additionally, this project explores the dynamics of the new competitive strategy game, how certain mechanics lead to different dominant strategies, and how to properly incentivize RL agents to learn winning strategies in this environment.

Thesis Committee:
Sebastian Scherer (Advisor)
Jeff Schneider
Cherie Ho

In Person and Zoom Participation. See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: CONNER PULLING
Speaker Website: theairlab.org…
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: Learning for Perception and Strategy: Adaptive Omnidirectional Stereo Vision and Tactical Reinforcement Learning
For More Information: brobbins@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Friday, August 9, 2024, 10am

Motion planning for dynamically complex robotic tasks requires explicit reasoning within constraints on velocity, acceleration, force/torque, and kinematics such as avoiding obstacles. To meet these constraints, planning algorithms must simultaneously make high-level discrete decisions and low-level continuous decisions. For example, pushing a heavy object involves making discrete decisions about contact locations and continuous decisions about fine interactions. Existing methods either search on a graph constructed by sampling or systematically discretizing the planning space or optimize parameterized trajectories subject to differential constraints. However, these methods suffer from combinatorial complexity or lack of convergence over long horizons. We hypothesize that these discrete high-level decisions often lie in a lower-dimensional subspace of the full planning space. Consequently, this thesis argues that by interleaving searching in this lower-dimensional discrete space with selective reasoning in the full planning space, we can efficiently discover global, long-horizon, dynamically rich capabilities to complete a task.

The goal of this thesis is to enable the automatic discovery of well-reasoned long-horizon plans for complex dynamical systems. To this end, we present the INterleaved Search And Trajectory optimization (INSAT) algorithm. INSAT combines the benefit of graph search-based planning algorithms to find paths over non-convex state spaces and that of trajectory optimization to find dynamically feasible trajectories in high-dimensional spaces. We demonstrate the effectiveness of INSAT in two challenging domains 1) aggressive quadrotor flight in large environments (in simulation) and 2) contact-rich manipulation of heavy objects in confined spaces. We also show that, by interleaving graph search and trajectory optimization, INSAT solves planning problems that a naively initialized trajectory optimization or standalone graph search does not solve. We then present INSAT*, a provably optimal variant of INSAT, and demonstrate its ability to discover optimal solutions for canonical dynamical systems.

The second half of the thesis explores three different ways of accelerating the INSAT planner: (1) CPU Parallelization: By leveraging recent advances in parallelized graph search, we enable CPU acceleration for optimization-based edge computation in INSAT, (2) Richer Planning Representation: We use sparse and optimization-friendly planning space representation called graphs of convex sets to speed up planning with INSAT. We also demonstrate that using INSAT as a planner on graphs of convex sets (IxG) is superior to mixed-integer convex optimization in terms of theoretical properties and runtime efficiency, (3) Preprocessing: By computing a motion library offline we can guarantee constant-time solutions for any planning query to IxG. These methods collectively enhance the performance and efficiency of the INSAT planner, making it more capable of addressing complex planning tasks.

Thesis Committee:
Maxim Likhachev (Co-chair)
Howie Choset (Co-chair)
Zachary Manchester
Russ Tedrake (Massachusetts Institute of Technology)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual -ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: RAMKUMAR NATARAJAN
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Robotics Institute, Carnegie Mellon University
Talk Title: Interleaving Discrete Search and Continuous Optimization for Kinodynamic Motion Planning
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Friday, August 9, 2024, 9am – 5pm

By invitation only.

Event Type: Corporate Visit
For More Information: ckissell@andrew.cmu.edu
Affiliations: Partnerships
Organization(s): School of Computer Science

View Details

Wednesday, August 7, 2024, 9am – 5pm

By invitation only.

Event Type: Corporate Visit
For More Information: gdarakos@andrew.cmu.edu
Affiliations: Partnerships

View Details

Thursday, August 8, 2024, 12pm

Human beings sense and express themselves through multiple modalities. To capture multimodal ways of human communication, I want to build adaptable robot policies that infer task pragmatics from video and language prompts, reason about sounds and other sensors, take actions, and learn mannerisms of interacting with people and objects. Existing solutions for robot policies rely on visual environment observations and structured language as goals. However, these assumptions limit sensory observations of the environment and the expressivity of the desired task from a user’s perspective. In this thesis, I present learning approaches for adaptable robot policies using different modalities to explicitly and implicitly convey task constraints to a learned robot policy.

The thesis proposal is organized into two parts: (1) completed and ongoing work focusing on video, language, and audio modalities, and (2) proposed work combining multiple modalities to fast adaptation of Robotics Foundation Models like OpenVLA. First, I present how we can train robot policies to infer the underlying tasks and preferences from the visual demonstration. I show how to infer the implicit task shown with cross-attention transformers to perform simulated dish-loading tasks. Then, I apply the same philosophy to train policies that infer the underlying task semantics from raw pixels in a prompt video and execute it in the robot's own environment. Second, I challenge the strong assumptions about language-based goal conditioning in robot policies. In one of the main works, I present sample-efficient robot policies that use the hierarchical decomposition of language into a sequence of interaction points and their relative waypoints.

I am currently developing a learning algorithm that allows a robot to efficiently predict how loud the robot's action noise is at the listener’s location and plan its actions accordingly. Humans inherently understand how their actions impact the acoustic environment around them, and we need this ability in home robots too. We train our model to visually predict how loud a listener may perceive the robot's noise at different indoor locations. Through these works, I study each modality individually, particularly how each modality can be used to improve the diversity of tasks performed and the ease of use of robots at home. Moving forward, I propose to enhance state perception and task specifications for more rapid adaptation and versatile robot control. Having examined how robots can explicitly and implicitly understand the specified task using different sensing modalities, my aim is to develop fast adaptation algorithms that connect foundational models by adapting them for visual cues and pragmatic task instructions.

Thesis Committee:
Yonatan Bisk (Chair)
Oliver Kroemer
Henny Admoni
Dieter Fox (University of Washington / NVIDIA)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: VIDHI JAIN
Speaker Website: vidhijain.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Multimodal Representations for Adaptable Robot Policies in Human-Inhabited Spaces
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, August 6, 2024, 7pm

Events are crucial discourse elements in natural language, playing a vital role in semantic understanding due to their complex structures that interconnect various parts of discourse. Events interact with other discourse elements to form diverse structures. Extensive research has been conducted on analyzing them, primarily focusing on frame structures (examining semantic roles such as participants, time, and location) and various forms of anaphora involving multiple events and entities, such as event coreference, event schema (event sequence, script), and ellipsis. The rich interactive nature of events presents both challenges and opportunities. On one hand, predicting and analyzing event structures can be complex. For example, conducting a standard document level event slot filling, can involve multiple structure prediction tasks (e.g., event mention, coreference, arguments, and schemas). On the other hand, the interactions among these structures can be leveraged to enhance model predictions, or provide a lens to study the mechanisms of models. This thesis explores the complexities and benefits of such interactions across different data availability scenarios, developing prediction methods including direct supervised training, crowdsourced event datasets, and studying automatically formed mechanisms related to them.

In the first part, we present empirical results analyzing event semantics with expert-annotated task-specific annotated datasets. We start by introducing methods studying isolated structures, such as event mention prediction, pair-wise event coreference, and event sequencing. We then present approaches to solve problems involving multiple structures, using multi-step or joint learning methods, such as joint coreference and sequencing, slot filling and verb phrase ellipsis. Recognizing the high cost of scaling expert-annotated datasets, the second part of this thesis explores methods to increase data availability through crowdsourcing and indirect supervision signals. An intriguing outcome of these approaches is their ability to reveal previously unspecified interactions between different textual structures. A key contribution in this area is our LLM360 language model project, which shares intermediate checkpoints throughout a model’s training process. We demonstrate the project’s utility for interpretability analysis, using the complex anaphora task of Winograd schemas as a case study.

This thesis demonstrates methods to address and utilize the complex nature of events. We find that scaling up data size leads to more iterations of such structures “automatically” appearing during analysis. Looking ahead, this work opens avenues for developing more sophisticated models that better capture relationships between event structures and for exploring large language models’ potential in understanding complex event semantics.  Additionally, our interpretability analysis, particularly through LLM360, paves the way for investigating how these  models processevent semantics internally. This research could lead to more transparent and  explainable AI systems, advancing our understanding of complex language processing.

Thesis Committee:
Teruko Mitamura (Chair)
Eduard Hovy
Taylor Berg-Kirkpatrick (University of California San Diego)
Vicent Ng  (The University of Texas at Dallas)

Additional Information

Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: ZHENGZHONG (HECTOR) LIU
Speaker Website: hunterhector.github.io
Speaker's Professional Title: Ph.D. Candidate, Language Technologies Institute, Carnegie Mellon University
Talk Title: Diving Deep into Event Semantics
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: StaceyYoung@cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science

View Details

Monday, August 5, 2024, 3pm

Autonomous robot navigation in off-road environments currently presents a number of challenges. The lack of structure makes it difficult to handcraft geometry-based heuristics that are robust to the diverse set of scenarios the robot might encounter. Many of the learned methods that work well in urban scenarios require massive amounts of hand-labeled data, but the nuances of deciding where a robot can and cannot drive in off-road terrain make it difficult to label large-scale data the same way. Many state-of-the-art approaches instead leverage self-supervised methods in training, using either expert demonstrations or proprioceptive feedback, but often still require a lot of data and can be vulnerable to domain shifts.

We adopt a philosophy that learned methods for off-road driving should be both self-supervised and adaptive, such that the robot can learn online without a human in the loop. In this work we propose a method that leverages proprioceptive cues and pre-trained visual foundation models to rapidly adjust its understanding of its environment in real-time, eliminating the need for large-scale training data and hand-labels. Specifically, we introduce a framework that predicts costmaps, speedmaps, and uncertainty by associating incoming visual features with roughness experienced by the system. With just seconds of collected experience, our results demonstrate navigation performance with as few interventions as methods trained on 100-1000x more data, while travelling as quickly as possible within the constraints of rider comfort. Furthermore, we aim to reduce the barrier to entry to full-scale off-road driving research by presenting TartanDrive 2.0, a large multi-modal dataset geared towards self-supervised learning methods.

Thesis Committee:
Sebastian Scherer (Advisor)
Wenshan Wang
Samuel Triest

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: MATTHEW SIVAPRAKASAM
Speaker Website: matthewjsiv.github.io
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: Online-Adaptive Self-Supervised Learning with Visual Foundation Models for Autonomous Off-Road Driving
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Wednesday, August 7, 2024, 10am

Concurrent software is notoriously difficult to write correctly, so to increase confidence in it, it is often desirable to apply formal verification techniques. One technique that is especially promising for verifying concurrent software is concurrent separation logic (CSL), which uses reasoning principles based on resource ownership. However, even with CSL, verifying complex systems at scale (e.g., those with 1000s of lines of code) remains challenging. The reasons it remains challenging include:

1. The manual proof effort required by many existing CSL frameworks.
2. The inherent complexity of the target systems. Sophisticated systems may have custom, low-level synchronization logic, which may be deeply intertwined with domain logic, in the interest of performance.

View Details

Tuesday, August 6, 2024, 3pm

The training performance of a deep learning (DL) training job is determined by the number, type and arrangement of the allocated GPU resources, and the job parameters (like batch size and learning rate) used for execution. Modern clusters for DL training contain tens of thousands of GPUs of many types, and a cluster scheduler allocates GPUs to training jobs to maximize collective training progress in the cluster. Existing DL cluster schedulers cannot handle the large space of adaptivity choices (i.e., combined space of GPU allocations and job parameters) for large, heterogeneous GPU clusters — many are not heterogeneity-aware, few are adaptivity-aware, and none scale to large clusters without sacrificing allocation fidelity and cluster efficiency.

In this thesis, we introduce (a) a scheduler to facilitate efficient job-resource adaptivity for DL training jobs on large heterogeneous GPU clusters, and (b) a method to scale optimization-based scheduling to much larger cluster sizes without sacrificing allocation fidelity and resource efficiency. Our adaptivity-aware scheduler, Sia, uses GPU resources judiciously to learn a job's training performance across different GPU types, and continuously co-optimizes the GPU allocation and job execution parameters to maximize cluster-wide training progress in heterogeneous GPU clusters. We then scale Sia to large cluster sizes by modeling the scheduling policy as a continuous optimization problem. We show that it is possible to augment the interface between a scheduler and the optimization problem solver to efficiently track changes to the scheduling problem arising from changing cluster conditions like job arrivals, departures and phase changes. We develop a prototype solver with the augmented interface for the Sia scheduling policy that can efficiently recover allocations for very large clusters. As an additional contribution, we observe that many other resource-allocation problems can also be formulated as continuous optimization problems and can be solved both quickly and efficiently using our proposed solver.

Thesis Committee:
Greg Ganger (Chair)
Zhihao Jia
Virginia Smith
Amar Phanishayee (Meta)

Additional Information

Event Type: Thesis Proposals
Room Number: In Person
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: SUHAS JAYARAM SUBRAMANYA
Speaker Website: suhasjs.github.io
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Efficient job-resource co-adaptivity for deep learning workloads on large heterogeneous GPU clusters
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Monday, August 5, 2024, 1:30pm

A painting is more than just a picture on a wall; a painting is a process comprised of many intentional brush strokes, leading to a performance far richer than the final output. The shapes of individual strokes are an important component of a painting's style. This is especially true for sparse sketches, where individual strokes are likely to be visible. Prior work in modeling brush stroke trajectories either does not work with real-world robotics or is not flexible enough to capture the complexity of human-made brush strokes. In this work, we aim to develop a robotic drawing agent with controllable stroke-level style based on human trajectories.

To achieve this, we develop a framework to collect brush trajectories from human artists on a real canvas. We model these trajectories with an autoencoder. Finally, we incorporate the autoencoder into the planning pipeline in the FRIDA robotic painting system. We find that, off-the-shelf, FRIDA's brush stroke renderer struggles or fails to learn the complex trajectories from the human demonstration data, especially with narrow brushes or markers. We present a novel brush stroke renderer that is capable of generalizing to complex, human-made brush strokes while maintaining a small Sim2Real gap. Our code is open sourced along with the dataset of drawing trajectories collected from people using real-world drawing tools.

Thesis Committee:
Jean Oh (Chair)
Jim McCann

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: LAWRENCE CHEN
Speaker Website: lawrencedchen.com
Speaker's Professional Title: Master's Student, Computer Science Department, Carnegie Mellon University
Talk Title: Spline-FRIDA: Enhancing Robot Painting with Human Brushstroke
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Wednesday, August 7, 2024, 12pm

Isaac Newton famously said, “stand on the shoulders of giants,” to emphasize the importance of pre-existing synthesis for making new challenges tractable in a single human brain. Newton himself learned partial abstractions from Ptolemy, Copernicus, Kepler, and Galileo, as well as Descartes’ analytic paradigm, which he used as foundations for his calculus problem. However, rapidly accumulating knowledge makes it increasingly difficult to be aware of existing approaches and innovate upon them.

In this thesis, I argue that what we need are new tools to help people synthesize useful cross-cutting abstractions from knowledge, effectively organize knowledge with those abstractions, and use them to find novel cross-domain insights. I present four systems toward this goal, where I explore several kinds of abstractions to enable new interaction capabilities. These include research threads' for supercharging people's reading experiences with AI to enable seamless interaction with thread-level abstractions while reading, the purpose-mechanism schema and how AI can help users find cross-domain analogical, andactive ingredients,' a mechanism abstraction that helps designers engage with and transfer insights from biology to mobility design.

Through controlled laboratory studies, I demonstrate the value of these abstractions in elevating people's focus during reading and exploration to a higher level (e.g., from individual papers to how notable threads divide a research field; from individual species to active ingredients of mechanisms), thereby gaining efficiency and helping them broaden their pursuit of problem-solving strategies. The end result is more creative ideas.

In a world of abundant knowledge and large language models, the structuring and distilling of conceptual insights will be the defining characteristics of driving value in knowledge work. By putting powerful techniques that empower conceptual interaction with information into the hands of everyday people, I envision a future where innovators everywhere deeply engage with insights that overcome domain boundaries and develop novel ideas that address personal challenges they face to bring forth positive effects for the world.

Thesis Committee:
Aniket Kittur (Chair)
Sherry Tongshuang Wu
Nikolas Martelaro
Michael Terry (Google)

Additional Information

Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: HYEONSU BUTTWEILER KANG
Speaker Website: www.hyeonsukang.com
Speaker's Professional Title: Ph.D. Candidate, Human-Computer Interaction Institute, Carnegie Mellon University
Talk Title: Accelerating Innovation through AI-Powered Conceptual Abstraction and Interaction Design
For More Information: lbuffing@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Friday, July 26, 2024, 3pm

Interactive task learning (ITL) is a machine learning paradigm that envisions AI that can learn whole programs directly from non-programmers' natural instruction. In this dissertation I present a system called AI2T that improves upon an ITL sub-paradigm called authoring-by-tutoring, whereby highly adaptive educational technology known as intelligent tutoring systems (ITSs) are authored by teaching an agent with rapid human-like learning capabilities. In the course of about 20-30 minutes authors can tutor AI2T with demonstrations and interactive feedback instead of needing to program an ITS by hand; a process which typically requires 200-300 developer hours per hour of instruction.

Authoring-by-tutoring presents a significant opportunity to democratize the authoring of ITSs. The defining characteristic of an ITS is the automatic delivery of detailed step-by-step feedback and hints characteristic of human-to-human tutoring. ITSs are typically more effective than traditional instruction and in some cases even more effective than human tutors. Authoring-by-tutoring is a path toward building the cognitively focused, precisely engineered, and reliably accurate behaviors of traditional ITSs without needing to hand-program behaviors or rely upon costly pretrained AI systems like large language models (LLMs) that are prone to hallucinating incorrect solutions and feedback. Toward this aim, this work innovates on methods of machine-learning that robustly learns complex behaviors via rapid bottom-up induction, instead of by mimicking patterns in big-data.

In this dissertation, I present two novel machine-learning algorithms that enable a data-efficient and robust interactive task learning, whereby 100% complete and accurate rule-based programs can be induced from interactive instruction. First I present STAND, a highly data-efficient algorithm for inducing preconditions for rules from binary reward signals. STAND out-performs algorithms like random forests and XGBoost known for their data-efficient learning on tabular data. STAND also enables a measure called instance certainty, an estimate of prediction probability that is more highly correlated with actual increases in holdout set performance than methods that rely on weighted ensembles. I show in simulation and with users that instance certainty can help authors estimate when AI2T has induced 100% complete programs, and show that it can provide active-learning support, helping authors identify the most helpful problems to tutor AI2T on next. Second, I introduce a method for learning hierarchical task networks (HTNs) from action sequences that helps AI2T induce simpler and more robust hierarchical programs than past systems. This approach is agnostic to action sequence lesson ordering, and induces HTNs with features like unordered groups and conditional actions that are useful for ITS rules.

Thesis Committee:
Kenneth  Koedinger (Chair)
Brad Myers
Vincent Aleven
Erik Harpstead
Kurt VanLehn (Arizona State University)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: DANIEL WEITEKAMP
Speaker Website: www.hcii.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Human-Computer Interaction Institute, Carnegie Mellon University
Talk Title: Building Educational Technology Quickly and Robustly with an Interactively Teachable AI
For More Information: lbuffing@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Thursday, July 25, 2024, 3 – 4:30pm

The final poster session for the  Research Experiences for Undergraduates in Software Engineering (REUSE) program is a welcomed opportunity to learn about the great research that our students conducted this summer. Join in as your schedule permits.

Posters will be presented on the following topics:

• Geometric Adversarial Attacks to Test Robotics Algorithms
• Explainable RL w/ Influence Functions
• NaNofuzz: Fuzzing, Property-Based, and Example-Based Testing
• How Does an Open-Source Ecosystem React to External Shocks? A Case Study of Rust
• Fine-tuning language models for data defense generation
• Interactive Geometric Proof Tool
• Understanding Deployment Challenges for Learning-Based Traffic Signal Control
• Attitudes Towards Voice-Sensitive AI-enabled Software for Decision Making Among
• Southern U.S. English Speakers
• Explaining Black-Box Machine Learning Decisions
• Qualitative Analysis of TEE Explainability
• Categorizing datetime bugs and fixes in open-source Python projects
• Motivations for Maintaining Open Source Software
• A Web Interface for Audio Transformation
• Understanding Program Visualizations in the Wild
• Design of Privacy Infrastructure for Internet of Things
• A Cooperative and Empowering Gig Economy
• Robustness Testing for Self-Driving Cars
• Prompt Engineering to Generate Model Requirements
• Putting the Ethics in Ethical Hacking
• Robustness Testing for Self-Driving Cars
• Designing CTF Challenges for Blockchain Education: A Beginner’s Approach
• Generating Effective Answers to Everyday Cybersecurity Questions
• An Empirical Study of Suspicious Starring Activities in GitHub
• Fine-tuning Language Model for Data Defense Generation
• Improving Visualizations of Formal Models
• Meerkat: distributed semantics with causal consistency
• ROSDiscover2: A Multi-Language Static Analysis Tool to Detect Architecture Misconfiguration Bugs
• Empowering and Enhancing Workers Through Building a Community-Centered Gig Economy
• GradBench: Benchmarking for the Future of Differentiable Programming
• Quantifying Software Robustness
• Veri-Datalog: A Verified Datalog Interpreter
• Proving Security Properties with Type Systems
• Learning to Play an Adaptive Cyber Deception Game
• Meerkat: distributed semantics with causal consistency
• Exploring Language Models as Agentic Email Assistants
• ROSAPIAnatomy: A Large-Scale Empirical Analysis of the Architecture of Robotics Systems
• Ad-Blocker Experiment
• Generalizing proofs with polymorphic modules
• Whamm!: a DSL for Instrumenting WASM Bytecode
• (in)effective Use of LLMs in Medical Settings
• Automating Translation to Legal Requirements Specification Language
• Using Kubernetes to Scale picoCTF Infrastructure
• Exploring the Correlation between Age and Reported Online Scams Using Reddit Data
• Measuring Trust in OSS
• Threaded Recomp-Verify: Advancing Model Checking through Multithreading

View Details

Thursday, July 25, 2024, 3 – 4pm

Over the last 50 years, autonomous robots have made the leap from being novel research contributions in university labs to becoming the fundamental technology upon which companies are built. While they traditionally have belonged to the engineering and computer science disciplines, robots have now crossed into other areas of study and research – making impacts in oceanography, geology, archaeology, biomechanics and biology. To exemplify these crossovers, the speaker will discuss several interdisciplinary projects that his research team have contributed to: altruistic robotics, underwater archeology, autonomous shark tracking, and education. These projects not only showcase several technical aspects of traditional robotics including motion planning, machine learning, state estimation, systems integration, and control theory, but also highlight the impact of interdisciplinary research and education.

—

Christopher Clark is research scientist at Apple, and has been a Professor at Harvey Mudd College since 2012. Before joining HMC’s faculty, he served as a faculty member at the University of Waterloo and California Polytechnic State University, San Luis Obispo. Clark is a Fulbright Scholar and for the 2011-2012 academic year, he held the William R. Kenan, Jr. Visiting Professorship for Distinguished Teaching at Princeton University. In 2004, he was a first hire at the startup company Kiva Systems (now Amazon Robotics), which changed warehouse management via multi-robot systems. He earned his undergraduate degree in engineering physics from Queen’s University, Canada, a master’s in mechanical engineering from the University of Toronto and a PhD in aeronautics and astronautics with a minor in computer science from Stanford University. Clark’s research areas include multi-robot systems, underwater robot systems, applied ML, control theory, intelligent vehicles, state estimation and motion planning.

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Mehrabian Collaborative Innovation Center, Conference Room 1, Lower Level and Zoom
Speaker's Name: CHRISTOPHER CLARK
Speaker Website: www.lair.hmc.edu…
Speaker's Professional Title: Research Scientist, Apple, and Professor, Harvey Mudd College
Talk Title: Robots Crossing Boundaries
For More Information: asilwal@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, July 25, 2024, 10am

As robots become increasingly integrated into human society, ensuring their safety is a paramount concern that has not been thoroughly addressed. This thesis tackles crucial questions on providing formal safety guarantees for robots operating in complex real-world scenarios and enabling these robots to perform diverse tasks intelligently and safely. By developing novel methodologies that build upon safe control, model-based control, and large language models (LLMs), my research bridges the gap between theoretical safety guarantees and practical implementation. This thesis presents three significant contributions: creating reliable safety measures and strategies, extending safety assurances to complex robotic systems, and achieving reliable open-world task execution. The benefits of this research are substantial, offering a framework for deploying robots that can safely and efficiently interact with their surroundings, thereby enhancing their utility and trustworthiness in various applications and fostering their integration into human society.

Thesis Committee:
Changliu Liu (Chair)
Yuejie Chi
Yorie Nakahira
Andrea Bajcsy
Chuchu Fan (Massachusetts Institute of Technology)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual
Building: Newell-SImon 3305 and Zoom
Speaker's Name: TIANHAO WEI
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Department of Electrical and Computer Engineering, Carnegie Mellon University
Talk Title: Safeguarding and Empowering General Purpose Robots
Affiliations: Robotics Institute (RI)
Organization(s): Department of Electrical and Computer Engineering, School of Computer Science

View Details

Friday, July 26, 2024, 10 – 11am

When outputting information to our senses, almost all wearable interfaces follow the same principle: externally generating stimuli (e.g., lights, sounds, vibrations) and then presenting them via devices placed at the endpoints of our sensory system, such as head-mounted displays in front of the eyes and vibration motors on the skin. While this intuitive approach of stimulating the endpoints might be sufficient for audiovisual interfaces, I argue that when it comes to engaging the sense of touch (i.e., haptics), it will fail to support a wide variety of interactive experiences. Even a single haptic device on the hand will obstruct users from touching or grabbing objects in the real world, making it undesirable for mixed reality. Let alone scaling this approach to a larger portion of the body, which would restrict the user’s whole body.

My research introduces an alternative approach to haptic output: instead of stimulating endpoints with external stimuli, I explore interactive devices that “internally” send electrical signals to the nervous system — intercepting the user’s nervous system. My approach creates haptic sensations beyond the point where the device is worn, establishing a basis for enabling haptic feedback while keeping the user’s body free, and scaling haptic interfaces to work for the entire body. In this talk, I will detail: (1) how intercepting the nerves can provide touch (CHI’23) and force (CHI’22) feedback without obstructing the user’s body with actuators; (2) how this approach can integrate into practical wearable devices such as a smartwatch (UIST’24); and (3) its potential to eventually generalize to a full-body interface by intercepting the user’s brain (CHI’24).

—

Yudai Tanaka is a PhD student in the Department of Computer Science at University of Chicago advised by Prof. Pedro Lopes. In his research, Yudai explores computer interfaces that output sensory feedback by intercepting the user’s brain or nervous system. These interfaces free up the user’s body from hardware in touch interactions; or even envision a new form of interactive experiences by presenting sensations directly to the brain. Yudai has published work at top Human-Computer Interaction (HCI) conferences including ACM CHI & UIST, with Best Paper Award (CHI’23), Best Paper Honorable Mention (CHI’24), and Best Demo Award (CHI’22).

Faculty Host:  Alexandra Ion

In Person and Zoom (internal CMU audiences only) and Livestream (All Guests)

Event Type: Talks
Room Number: In Person
Building: 407 S. Craig Street, 2nd Floor
Speaker's Name: YUDAI TANAKA
Speaker Website: yudai-tanaka.com
Speaker's Professional Title: Ph.D. Student, Department of Computer Science, University of Chicago
Talk Title: Intercepting User's Brain and Nervous System: A New Paradigm for Haptic Output
For More Information: alexandraion@cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Friday, July 26, 2024, 9am

The Internet has become the central source of information and communication in modern society. Congestion control algorithms (CCAs) are critical for the stability of the Internet: ensuring that users are able to fairly and efficiently share the network. Over the past 30 years, researchers and Internet content providers have proposed and deployed dozens of new CCAs designed to keep up with the growing demands of faster networks, diverse applications, and mobile users. Without tools to understand this growing heterogeneity in CCAs deployed in the Internet, the fairness of the Internet is at stake.

Towards understanding this growing heterogeneity, we develop CCAnalyzer, a tool to determine what CCA a particular web service deploys, outperforming previous classifiers in accuracy and efficiency. With CCAnalyzer, we show that new CCAs, both known and unknown, have widespread deployment in the Internet today, including a recently proposed CCA by Google: BBRv1. Next, we develop the first model of BBRv1, and prove BBRv1 can be very unfair to legacy loss-based CCAs, an alarming finding given the prolific deployment of BBRv1.

Consequently, we argue the need for a better methodology for determining if a new CCA is safe to deploy in the Internet today. We describe how the typical methodology testing for equal-rate fairness (every user gets the same bandwidth) is both an unachievable goal and ultimately, not the right threshold for determining if a new CCA is safe to deploy alongside others. Instead of equal-rate fairness, we propose a new metric we call, harm, and argue for a harm-based threshold. Lastly we present RayGen, a novel framework for evaluating interactions between heterogeneous CCAs. RayGen uses a genetic algorithm to efficiently explore the large state space of possible workloads and network settings when two CCAs compete. With a small budget of experiments, RayGen finds more harmful scenarios than a parameter sweep and random search.

Thesis Committee:
Justine Sherry (Co-Chair)
Srinivasan Seshan (Co-Chair)
Theophilus A. Benson
Jim Kurose (University of Massachusetts Amherst)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates Hillman 4405 and Zoom
Speaker's Name: RANYSHA WARE
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Battle for Bandwidth: On The Deployability of New Congestion Control Algorithms
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Thursday, July 25, 2024, 4pm

Online peer support groups provide a conducive environment, allowing members to get connected with peers who share similar difficulties. There, members are able to provide informational and emotional support to each other without restrictions of time and geographic location. However, peer support also suffers from a variety of challenges, including members’ lack of commitment and expertise in providing support. Failure to address these challenges might lead to unwanted consequences such as volunteer burnout and mistreatment of helpees.  

In my work, I started off studying peer support in a health-related context, emphasizing two research questions, 1) how to keep members committed to providing support and 2) how to empower committed members with skills so that they can provide better support. It is not clear, however, the extent to which conclusions obtained from this research can be applied to support groups where members might have conflicts of interest. I then expand my work to peer support groups where members have competition by examining how gig workers provide and receive support online.In a specific case study, I explored how gig workers collectively make sense of algorithms that manage their work in online communities. 
 
Thesis Committee:
Bob Kraut (Co-Chair)
Haiyi Zhu (Co-Chair)
Sarah Fox
 Julia Ticona (University of Pennsylvania)
 
Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: Virtual Presentation - ET
Building: Remote Access - Zoom
Speaker's Name: ZHENG YAO
Speaker Website: zheng-yao.com
Speaker's Professional Title: Ph.D. Candidate, Human-Computer Interaction Institute, Carnegie Mellon University
Talk Title: Peer Support in Online Communities
For More Information: lbuffing@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Friday, July 26, 2024, 11am

This work investigates a constrained inverse kinematic (IK) problem that seeks a feasible configuration of an articulated robot under various constraints such as joint limits and obstacle collision avoidance. Due to the high-dimensionality and complex constraints, this problem is often solved numerically via iterative local optimization. Classic local optimization methods take joint angles as the decision variable, which suffers from non-linearity caused by the trigonometric constraints. Recently, distance-based IK methods have been developed as an alternative approach that formulates IK as an optimization over the distances among points attached to the robot and the obstacles. Although distance-based methods have demonstrated unique advantages, they still suffer from low computational efficiency, since these approaches usually ignore the chain structure in the kinematics of serial robots. This work proposes a new method called propagative distance optimization for constrained inverse kinematics (PDO-IK), which captures and leverages the chain structure in the distance-based formulation and expedites the optimization by computing forward kinematics and the Jacobian propagatively along the kinematic chain. Test results show that PDO-IK runs up to two orders of magnitude faster than the existing distance-based methods under joint limits constraints and obstacle avoidance constraints. It also achieves up to three times higher success rates than the conventional joint-angle-based optimization methods for IK problems. The high runtime efficiency of PDO-IK allows the real-time computation (10−1500 Hz) and enables a simulated humanoid robot with 19 degrees of freedom (DoFs) to avoid moving obstacles, which is otherwise hard to achieve with the baselines.

Thesis Committee:
Howie Choset (Advisor)
Guanya Shi
Chao Cao

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Traffic21 Classroom, Gates Hillman 6501 and Zoom
Speaker's Name: YU CHEN
Speaker Website: neuling-jpg.github.io…
Speaker's Professional Title: Master's Student, Robotics Institute, Carnegie Mellon University
Talk Title: Propagative Distance Optimization for Constrained Inverse Kinematics
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Friday, July 26, 2024, 10:30am

Quadruped robots offer a versatile solution for navigating complex terrain, making them valuable for applications such as industrial automation or search and rescue. Although quadrupeds are more complex than bipeds, they are easier to balance and control and require fewer joints to actuate compared to hexapods. Traditional quadruped designs, however, often feature complex leg mechanisms that are difficult to scale and require many actuators. We introduce the DeltaWalker, a novel quadruped robot that uses linear delta robots as legs. Delta robots offer advantages such as precision and scalability, while still being able to move with three degrees of freedom. This design is inspired by DeltaHands, a 4-fingered hand robot with soft, 3D-printed linear delta robots. We turn the DeltaHands upside down and adapt the design for locomotion. We explore various gait patterns, including manually designed and trajectory-optimized gaits, evaluated in both simulated and real-world environments. We also describe the system kinematics and investigate other capabilities, such as rotating in place. The findings from these evaluations demonstrate the potential of the DeltaWalker as a simpler, scalable, omnidirectional quadruped robot.

Thesis Committee: 
Prof. Zeynep Temel (Advisor)
Prof. Zachary Manchester
Zilin Si

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: JENNIFER YANG
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: DeltaWalker: A Soft, Linearly Actuated Delta Quadruped Robot
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Friday, July 26, 2024, 3pm

Achieving human and animal-level agility has been a long-standing goal in robotics research. Recent advancements in numerical optimization and machine learning have pushed legged systems to greater capabilities than ever before, enabling black flips, parkour, and manipulation of heavy objects. Despite these exciting developments, this thesis identifies two key limitations of current legged robot technology and aims to improve upon existing art.

First, legged robots today require manual specifications of desired behaviors and fail to learn from their human and animal counterparts. We introduce SLoMo, a first-of-its-kind framework for transferring skilled motions from casually captured videos of humans and animals to legged robots. From a monocular RGB video, SLoMo synthesizes physically plausible trajectories for downstream offline trajectory optimization and online predictive control of quadruped or humanoid robots. We demonstrate SLoMo by transferring cat and dog motions to quadruped robot hardware and human motions to a simulated humanoid robot.

Second, current model-predictive control (MPC) for legged systems often resort to simplified models due to computational limitations in real-time settings. This is due to the high dimensionality of these robots and the reliance of existing numerical optimization algorithms on fundamentally serial, CPU-friendly linear algebra routines. We leverage advancements in GPU parallelization by developing a quadratic programming (QP) solver that uses only GPU-friendly operations. We refer to our solver as ReLU-QP, thanks to its computational similarities to inferencing a deep neural network with rectified linear unit (ReLU) activation functions. Across benchmarks on solving random QPs and high-dimensional MPC tasks in simulation, including balancing a full-order Atlas humanoid robot on one foot under control limits, ReLU-QP shows an order-of-magnitude speed improvement over state-of-the-art CPU-based QP solvers and solves MPC for modern legged robots at kilohertz rates.

Thesis Committee:
Zachary Manchester (Advisor)
Deva Ramanan
Guanya Shi
Kevin Tracy

In Person and Zoom Participation. See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: JOHN ZHANG
Speaker Website: johnzhang3.github.io
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Institute
Talk Title: Advancing Legged Robot Agility: from Video Imitation to GPU Acceleration
For More Information: barbarjean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Sunday, July 14, 2024, 3 – 4pm

Are you are wondering what CMU Buggy is all about? 
Buggy Alumni Associate (BAA)  recently digitized video from Raceday 2001, and are holding a 2001 Raceday Rewatch,  on the CMUBuggy YouTube Channel. This was the first broadcast ever produced by cmuTV, and some of this footage (and commentary) hasn't been seen since that day in 2001 (or maybe ever). We'll be live streaming all 29 heats from those 2 rain-soaked days, along with some pre-produced packages and live interviews conducted by cmuTV at Raceday 2001.

We'll also have some Raceday 2001-related trivia, a live chat for people to reminisce and catch up with old and new friends, and hopefully we'll have some guests on the stream to talk about that Raceday and share some of their favorite buggy stories. If you were involved in Raceday 2001, whether as a participant, Sweepstakes, or a spectator, and want to join the broadcast as well, just let me know!

So grab your favorite Sunday afternoon beverage, your favorite Sunday afternoon people, and join us for an afternoon of fun, excitement, and nostalgia as we release the world broadcast premiere of Raceday 2001. And feel free to pass this along to anyone and everyone - it's a virtual event, so all are welcome!

WATCH!

Event Type: Fun
Room Number: Virtual Viewing - ET
Building: Remote Access - YouTube
Affiliations: Alumni, Carnegie Mellon University, Student Groups and Organizations, Traditions
Event Website Title: Event Website
Event Website URL: cmubuggy.org…

View Details

Monday, July 15, 2024, 2pm

The era of vision-language models (VLMs) trained on large web-scale datasets challenges conventional formulations of “open-world" perception. In this work, we revisit the task of few-shot object detection (FSOD) in the context of recent foundational VLMs. 

First, we point out that zero-shot VLMs such as GroundingDINO significantly outperform state-of-the-art few-shot detectors (48 vs. 33 AP) on COCO. Despite their strong zero-shot performance, such foundational models may still be sub-optimal. For example, trucks on the web may be defined differently from trucks for a target application such as autonomous vehicle perception. We argue that the task of few-shot recognition can be reformulated as aligning foundation models to target concepts using a few examples. Interestingly, such examples can be multi-modal, using both text and visual cues, mimicking instructions that are often given to human annotators when defining a target concept of interest. 

Concretely, we propose Foundational FSOD, a new benchmark protocol that evaluates detectors pre-trained on any external datasets and fine-tuned on multi-modal (text and visual) K- shot examples per target class. We repurpose nuImages for Foundational FSOD, benchmark several popular open-source VLMs, and provide an empirical analysis of state-of-the-art methods. Lastly, we discuss our recent CVPR 2024 Foundational FSOD competition and share insights from the community. Notably, the winning team significantly outperforms our baseline by 23.9 mAP!

Committee:
Prof. Deva K. Ramanan (Advisor)
Prof. Katerina Fragkiadaki
Neehar Peri

In Person and Zoom Participation. See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Rashid Auditorium, Gates Hillman 4401
Speaker's Name: ANISH MADAN
Speaker Website: anishmadan23.github.io
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: Automating Annotation Pipelines by leveraging Multi-Modal Data
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, July 11, 2024, 2pm

The use of unmanned aerial vehicles (UAVs) for time critical tasks is becoming increasingly popular. Operators are expected to use information from these swarms to make real-time and informed decisions. Consequently, detecting and recognizing targets from video is extremely pivotal to the success of these systems. At greater altitudes or with more vehicles, this process becomes increasingly difficult and requires significant scrutiny from an operator. Hence, it is important to try and optimize the information passed between a UAV and the operator. In this work, we present a system that can be used to recreate high-fidelity environments as well as simulate the behavior and data of a UAV swarm. This simulation extends AirSim, a flight simulator plugin for Unreal Engine, to deploy fixedwing aircrafts and other vehicles for the use of conspicuity measurement and other visual salience studies. Through the use of this simulator, we showcase the effects that various factors can have on an operator’s target recognition including altitude, velocity, and conspicuity.

Committee:
Katia Sycara (Advisor)
Henny Admoni
Renos Zabounidis

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Mauldin Auditorium, Newell-SImon 1305 and Zoom
Speaker's Name: BRANDON RISHI
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: Simulation as a Tool for Conspicuity Measurement
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Wednesday, July 10, 2024, 10:30am

Global localization is essential for the smooth navigation of autonomous vehicles. To obtain accurate vehicle states, on-board localization systems typically rely on Global Navigation Satellite System (GNSS) modules for consistent and reliable global positioning. However, GNSS signals can be obstructed by natural or artificial barriers, leading to temporary system failures and degraded state estimation. On the other hand, off-road driving presents unique challenges for ground vehicles due to irregular terrain, leading to unstable surfaces for traversal and affects state estimation accuracy. Additionally, visual odometry performance may suffer due to the lack of distinct and reliable features for accurate state estimation. To address these challenges, we propose a novel learning-based method that synthesizes a local bird's-eye-view (BEV) image of the surrounding area by aggregating visual features from camera images. The proposed model combines a deformable attention-structured network with an image rendering head to generate top-down BEV images. The synthesized images are subsequently matched with an aerial map for cross-view vehicle registration in GNSS-denied off-road environments. Extensive real-world experimentation validates our method's advancement over existing GNSS-denied visual localization methods, demonstrating notable enhancements in both localization accuracy and registration frequency. Our method effectively reduces visual inertial odometry (VIO) drifts when integrated with an on-board VIO system via factor graph optimization.

Committee: 
Prof. Michael Kaess (Advisor)
Dr. Wenshan Wang
Easton Potokar

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: LIHONG (LEAH) JIN
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: GNSS-denied Ground Vehicle Localization for Off-road Environments with Bird's-eye-view Synthesis
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, July 8, 2024, 11am – 12pm

Dr. Andy Kilianski will provide an overview of ARPA-H, a new U.S. government funding agency pursuing R&D for health challenges. He will review the unique niche occupied by ARPA-H within the Department of Health and Human Services and how ARPA-H is already partnering with academia and industry to transform health outcomes across the country. Discussion will include current funding opportunities, future research interests, and how to become a program manager and get involved.

Faculty Host:  Artur Dubrawski

Additional Information on ARPA-H

In Person and Zoom Participation.  See announcement.

Event Type: Talks
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: ANDY KILIANSKI
Speaker Website: arpa-h.gov…
Speaker's Professional Title: ARPA-H Program Manager, Health Science Futures Group
Talk Title: ARPA-H and America’s Health: Pursuing High-Risk/High-Reward Research to Improve Health Outcomes for All
For More Information: awd@cs.cmu.edu
Affiliations: Computational Biology Department (CBD), Computer Science Department (CSD), Human-Computer Interaction Institute (HCII), Language Technologies Institute (LTI), Machine Learning Department (MLD)

View Details

Tuesday, July 9, 2024, 12:30pm

The rapid integration of artificial intelligence (AI) into human life underscores the importance of studying human-AI interaction and complementarity. The vast majority of AI research has focused on high-performance systems, typically on objective performance measures. However, as AI permeates through all facets of our lives, it is crucial to understand how AI can enhance human experiences and capabilities. To this end, interest has increasingly shifted towards identifying ways to develop AI not only with the goal of building capable systems but with the goal of complementing human behavior and decision making. To make progress towards human-AI complementarity, it is critical to understand both human behavior and the capabilities of AI agents that take human preferences into account. This work proposes three contributions to advance work on human-AI complementarity in shared workspaces.  

First, in Chapter 1, we address the lack of generalized frameworks for human-AI interaction by developing an open-source framework for real-time shared workspace experiments. This framework allows us to explore how human-AI complementarity can incorporate subjective preferences and human skill learning. In Chapter 2 we investigate human preferences for AI teammates in a collaborative task using a novel method for constructing robust, controllable AI policies. This research provides insights into what humans value in AI partners. Lastly, in Chapter 3 we propose a learning perspective on human-AI complementarity, exploring how AI partners can complement human learning in multi-agent control settings. Together, these contributions expand the study of human-AI complementarity by providing new tools for the research community and deepening our understanding of human behavior, preferences, and learning in human-AI interactions.

Thesis Committee:
Cleotilde Gonzalez (Chair)
John Miller
Henny Admoni

Event Type: Thesis Proposals
Room Number: In Person
Building: Porter Hall 223D
Speaker's Name: CHASE C. McDONALD
Speaker Website: chasemcd.com
Speaker's Professional Title: Ph.D. Student, Department of Social and Decision Sciences, Carnegie Mellon University
Talk Title: On Human-AI Complementarity in Shared Workspaces
For More Information: jmarch@andrew.cmu.edu
Organization(s): Department of Social and Decisions Sciences, AI Institute for Societal Decision Making (AI-SDM)

View Details

Tuesday, July 9, 2024, 12 – 12:30pm

The need for fleet-wide power and data management has never been higher. Automation is soaring. Electrification is everywhere. But managing charging, batteries and fleet uptime cost effectively remains a significant challenge for growing fleets.

WiBotic provides infrastructure that every autonomous fleet needs at scale: power and data. Without reliable power and data, autonomous systems cannot provide the value they promise to businesses all over the world. In this tech talk, WiBotic CEO and Co-Founder Ben Waters will provide an overview of wireless charging for autonomous systems, and will discuss some of the decisions that robotics companies and end-users should consider when it comes to charging infrastructure.

—

Ben Waters is the CEO and co-founder of WiBotic, a company that develops wireless charging solutions and fleet-wide battery management software for industrial mobile platforms of all shapes and sizes. Their mission is to energize businesses with autonomous systems to grow and scale without limitation, while creating a workplace filled with brilliant people who are passionate about leadership. Ben has a PhD in Electrical Engineering from the University of Washington, where he worked on wireless power technologies for implanted devices and a Bachelors Degree in Electrical Engineering and Physics from Columbia.

At WiBotic, Ben leads a team of engineers, designers, and business developers who are dedicated to solving the challenges of powering autonomous robots. They have created a highly flexible wireless charging platform that can charge mobile robots, medical carts, floor cleaning scrubbers, space infrastructure and underwater vehicles, regardless of their battery type, size, or shape. WiBotic Commander Software optimizes battery charging and provides cost-saving insights into battery maintenance, fleet uptime and simplifies autonomous charging. The WiBotic platform has been used by customers across various industries, such as e-commerce, manufacturing, agriculture, defense, security, and logistics, to improve the efficiency, reliability, and safety of their robotic operations.

REGISTER

About ARM M2M Tech Talks: This member-exclusive series features a subject matter expert from the ARM Member Consortium providing a brief overview of some of the technology driving their business. Following the presentation, we will open the event up for some quick interactive Q&A, discussion, and networking. Unlike our traditional webinar series, attendees will be able to join using both video and voice functionality (if desired) to connect with our session speaker. Connecting with the speaker after the Tech Talk using the Member Community is highly encouraged!

The ARM Institute is the nation’s leading Robotics and AI Manufacturing Innovation Institute part of the Manufacturing USA Network. With our 380+ member organization partners, we advance critically needed robotics and workforce solutions to strengthen U.S. manufacturing.
 

Event Type: Talks
Room Number: Virtual Presentation - ET
Speaker's Name: BEN WATERS
Speaker Website: www.wibotic.com…
Speaker's Professional Title: CEO and Co-founder, WiBotic
Talk Title: How wireless opportunity charging saves time, efficiency and battery life
Affiliations: Carnegie Mellon University, College of Engineering, Partnerships
Organization(s): School of Computer Science

View Details

Tuesday, July 2, 2024, 3pm

Causal inference focuses on uncovering cause-effect relationships from data, diverging from conventional machine learning which primarily relies on correlation analysis. By identifying these causal relationships, causal inference improves feature selection for predictive models, leading to predictions that are more accurate, interpretable, and robust. This approach proves especially effective with interventional data, such as randomized control trials (RCTs), where deliberate changes in variables allow for observing their effects.

In this study, we begin by examining if existing tabular datasets contain interventional data, such as natural experiments. Natural experiments occur when events affect individuals or groups differently, akin to the varied impact of the COVID-19 pandemic on different populations. Our findings demonstrate that real-world datasets indeed contain natural experiments, which can be utilized to enhance classification performance through causal inference. We further extend this methodology to investigate lung ultrasound video datasets, aiming to glean additional insights and enhance diagnostic accuracy.

Committee:
John Galeotti
Deva Ramanan
Peter Spirtes
Gokul Swamy

In Person and Zoom Participation. See announcement.

Event Type: Speaking Skills
Room Number: In Person and Virtual - ET
Building: Mauldin Auditorium, Newell-Simon 1305 and Zoom
Speaker's Name: GAUTAM RAJENDRAKUMAR GARE
Speaker Website: ggare-cmu.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Enhancing Model Performance and Interpretability with Causal Inference as a Feature Selection Algorithm
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, July 2, 2024, 10am

Place Recognition is essential for achieving robust robot localization. However, current state-of-art systems remain environment/domain-specific and fragile. By leveraging insights from vision foundation models, we present AnyLoc, a universal VPR solution that performs across diverse environments without retraining or fine-tuning, significantly outperforming supervised baselines. We further introduce MultiLoc, and enable zero-shot cross modal place recognition across lidar and thermal modalities by distilling features from vision foundation models. Our comprehensive evaluation across structured and unstructured datasets demonstrates the feasibility of building universal place recognition systems that can operate anytime, anywhere, anyview and across any sensor.

Thesis Committee:
Prof. Sebastian Scherer (Chair)
Prof. Michael Kaess
Dr. Wenshan Wang
Zhao Shibo

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual -ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: JAY KARHADE
Speaker Website: jaykarhade.github.io
Speaker's Professional Title: Master's Student (MSR), Robotics Institute, Carnegie Mellon University
Talk Title: Towards Universal Place Recognition
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, July 1, 2024, 9am

Touch is an essential sensing modality for making autonomous robots more dexterous and allowing them to work collaboratively with humans. In particular, the advent of vision-based tactile sensors has resulted in efforts to design them for different robotic manipulation tasks. However, this design task remains a challenging problem. This is for two reasons: first, the design of the sensor itself requires the compact integration of multiple optical elements to improve optical signal fidelity during interaction with the environment; second, the successful integration of vision-based tactile sensors into robotic manipulation tasks requires the co-design of both the sensors and the robot structure itself for optimal sensing and control.

This thesis aims to alleviate these two challenges by creating a general design framework that allows a roboticist to quickly iterate on the design and evaluation of vision-based tactile sensors for designated robotic manipulation tasks. First, our framework uses an optical simulator, based on physics-based rendering, that can accurately and efficiently generate the images captured by arbitrary sensor designs upon tactile indentation. Second, our framework uses a procedural sensor shape generator and introduces novel objective functions (perceptual and geometric) to improve tactile sensor designs automatically. Thirdly, we develop a general modular and interactive pipeline for rapid sensor prototyping with focus on novice users, that can automatically generate parameterized designs and optimize optical components within minutes. We provide an implementation of our framework as a design toolbox, OptiSense Studio.

We showcase the sim2real comparisons of a range of GelSight-like tactile sensors, specifically sensors that include optical components like mirrors, curved sensing surface, fluorescence, and light piping. Furthermore, we demonstrate the utility of our procedural sensor shape optimization for a curved tactile sensor shape and compare the optimized design performance against sensor expert’s hand-optimized designs by manufacturing a real-world prototype. We show the design of a new tactile sensor using our interactive toolbox and improve existing ones completely virtually.

This thesis tackles a critical problem in enabling the development of vision-based tactile sensors and consequently their adoption in sensing for diverse applications. Through this thesis, we demonstrate the utility of our design framework for the design of vision-based tactile sensors and compliant tactile sensors. More broadly, we want to create a new point of convergence between disparate communities such as computer graphics (physics-based rendering and simulation), optics (optical lens and material design), and robotics, and foster new research directions within and across these communities.

Thesis Committee:
Wenzhen Yuan (Co-chair, University of Illinois Urbana-Champaign)
Ioannis Gkioulekas (Co-chair)
Nancy Pollard
Edward Adelson (Massachusetts Institute of Technology)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person
Building: Newell-Simon 4305
Speaker's Name: ARPIT AGARWAL
Speaker Website: www.andrew.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Robotics Institute, Carnegie Mellon University
Talk Title: A Modularized Approach to Vision-based Tactile Sensor Design Using Physics-based Rendering
For More Information: lyonsmuth@cmu.edu
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, July 1, 2024, 11am

In this thesis, we present a new method to solve algorithmic and combinatorial problems by (1) reducing them to bounding the maximum, over x in {-1,1}ⁿ, of homogeneous degree-q multilinear polynomials, and then (2) bounding the maximum value attained by these polynomials by analyzing the spectral properties of appropriately chosen induced subgraphs of Cayley graphs on the hypercube (and related variants) called "Kikuchi matrices".

We will present the following applications of this method.

1. Designing algorithms for refuting/solving semirandom and smoothed instances of constraint satisfaction problems;
2. Proving Feige's conjectured hypergraph Moore bound on the extremal girth vs. density trade-off for hypergraphs;
3. Proving a cubic lower bound for 3-query locally decodable codes and an exponential lower bound for 3-query locally correctable codes.

View Details

Monday, September 9, 2024, 4 – 6pm

President Farnam Jahanian invites you to save the date for a
     
                         Celebration of Life
                                Honoring 
                             Jared L. Cohon 
 
Jared L. Cohon served as Carnegie Mellon University’s eighth president from 1997 to 2013. Jerry was widely respected and immensely beloved, and his leadership and scholarship have shaped the trajectories of an untold number of Tartans over the years. His brilliant mind, unyielding energy and unimpeachable integrity have made our institution — and our society — better in innumerable ways.

Additional information to follow in the coming weeks.

Event Type: Special Events
Room Number: In Person
Building: Rangos Ballroom, Cohon University Center
For More Information: CMUevents@andrew.cmu.edu
Affiliations: Carnegie Mellon University
Organization(s): School of Computer Science

View Details

Thursday, July 18, 2024, 6 – 8pm

Join BNY and Innovation Works for the UpPrize 2024 Semi-finals at the Nova Place Concourse. During the event, we'll hear from a panel of our 2023 UpPrize Finalists to see where they are now and how the BNY UpPrize Social Innovation Challenge impacted their businesses.

Join in to celebrate social innovation in Southwestern PA - as they announce the ten UpPrize 2024 Semifinalists who will proceed in the challenge for a chance to win up to $125,000 for their business!

Finalists will be part of this inaugural cohort starting in January 2025.  Led by entrepreneurial experts, this module is specifically designed to provide insights into customer acquisition and retention.  Additionally, participants will receive access to Innovation Works' networking events and support.

REGISTER

Event Type: Career Presentation
Room Number: Nova Place Concourse
Building: 100 South Commons, Pittsburgh, PA 15212
Speaker's Name: Semi-Finalists Announcement, Panel Discussion, and Networking
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
Event Website Title: Program Website and Registration
Event Website URL: www.upprize.org

View Details

Friday, June 28, 2024, 2pm

After correctness, the most important properties of programs concern their resource requirements, like how much time they take to run or how much memory they need. It is therefore desirable to automate the
derivation of a program’s costs. One successful approach to such automatic derivation is the type system
known as Automatic Amortized Resource Analysis (AARA). AARA finds polynomial bounds on resource usage by using its types to apply the physicist’s method of amortized cost analysis. Type inference in AARA can be reduced to linear programming, thereby automating resource analysis. This balance of expressive bounds and efficient analysis has brought AARA success.

Unfortunately, deriving a program’s resource usage can be difficult — in fact it is generally not computable.
Thus, despite AARA’s success, it is not surprising that there are many natural program patterns that it cannot analyze well. Sometimes AARA finds loose resource bounds, other times it finds bounds slowly, and sometimes it cannot find any bounds at all.

This thesis addresses such shortcomings by developing a variety of upgrades to the AARA type system that
allow the efficient derivation of tight resource bounds for more programs. The key theme underlying these
upgrades is the leveraging of linear reasoning principles. These ideas integrate well with AARA because AARA exists in the intersection of various forms of linearity: the linear flavor its type system, the linear relations of its cost bound templates, and the linear physicality behind the physicist’s method of amortized cost analysis.

This work first upgrades the type system with remainder contexts to better reason about reusable resources like memory. Then the class of AARA’s bounding functions is enlarged to include, e.g., exponential bounds. This class of functions is further enlarged to be multivariate, allowing dependence on products of data structure sizes. Next, this work provides a more efficient, matrix-based approach to inferring the cost-free AARA types needed for, e.g., non-tail recursion. Finally the physicist’s method of amortized cost analysis is refined into the quantum physicist’s method, which provides an automatable framework for reasoning about resource reallocation, while also allowing resource bounds to depend on data structure height.

Thesis Committee:
Jan Hoffmann (Chair)
Frank Pfenning
Stephanie Balzer
Thomas Reps (University of Wisconsin)

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person
Building: Mauldin Auditorium, Newell-Simon 1305
Speaker's Name: DAVID KAHN
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Leveraging Linearity to Improve Automatic Amortized Resource Analysis
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: csd.cmu.edu…

View Details

Wednesday, July 3, 2024, 12pm

Industrial control systems (ICS) govern critical infrastructure and processes, such as power generation, chemical processing, and water treatment. Given their widespread impact and their critical nature, there is a strong incentive for adversaries to attack ICS. An adversary that gains access to an ICS network can manipulate its process values to cause physical damage and harm. Machine-learning-based anomaly detection can be used to detect such manipulated data and is a common proposal for defending ICS. To make anomaly detection more effective for ICS, this thesis investigates and proposes solutions to several challenges when applying anomaly detection to an ICS. First, it is unclear what models and methods are best for detecting ICS anomalies; we comprehensively evaluate prior approaches and compare their performance, identifying what strategies were most effective. Second, it is unclear if and how anomaly-detection outputs can be used to diagnose ICS anomalies; we evaluate a variety of approaches for attributing ICS anomalies to the underlying components that were manipulated. Third, we identify fundamental issues with prior anomaly-detection approaches for ICS, and we are investigating how incorporating domain knowledge through graphs can improve current detection and attribution approaches. Finally, to better understand if current anomaly-detection approaches appropriately match the needs of ICS in practice, we are conducting an interview-based study to understand the workflows and perspectives of practitioners that monitor ICS.

Thesis Committee:
Lujo Bauer (Chair)
Eunsuk Kang
Vyas Sekar (CSD/ECE)
Michael Reiter (Duke University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Mehrabian Collaborative Innovation Center 2101 and Zoom
Speaker's Name: CLEMENT FUNG
Speaker Website: clementfung.me
Speaker's Professional Title: Ph.D. Student, Ph.D. Program in Societal Computing, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Proposing Guidelines and Approaches to Make Anomaly Detection More Effective for Industrial Control Systems
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: aroudebu@andrew.cmu.edu | cherold@cs.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Saturday, September 7, 2024, 11am – 6pm

Entrepreneurship Bootcamp, a fall semester, one-day glimpse into entrepreneurship at CMU and Pittsburgh.

Sponsored by: Swartz Center for Entrepreneurship and Graduate Entrepreneurship Club

Event Type: Career Presentation
Room Number: In Person
Affiliations: Entrepreneurship
Event Website Title: Event Website and Updates
Event Website URL: www.cmu.edu…

View Details

Monday, July 29, 2024, 8:30am

Datathon is a data science competition series similar to hackathons tackling real-world social challenges in topics such as urban traffic, renewable energy, and education. Teams of students receive complex datasets, then analyze them, use real-world findings, and prepare a report of their discoveries to a panel of judges.

Hosted by Citadel | Citadel Securities in cooperation with Correlation One.

The Datathon will be held 29 July through 5 August 2024.

 → Location:  Virtual
 → Open to undergraduate and graduate students
      — Undergraduate and graduate students in good standing, eighteen years of age or older
           residing in the US or Canada
      — Must be pursuing a bachelor’s or master’s degree in Math, Engineering, Statistical Modeling,
          Calculus, Computer Science, Physics, Economics, or related disciplines.
 → Application deadline: 11:59:59 pm EST on 21 July 2024
 → Prizes!  $15,000 + recruiting networking opportunities

Additional Information |  APPLY

Event Type: Career Center Event
Event Website Title: SCS/ECE Careers
Event Website URL: www.cs.cmu.edu…

View Details

Wednesday, June 26, 2024, 1pm

Large pre-trained models and internet data sources are key to general and efficient robot task learning. However, learning contact-rich behaviors, semantic task constraints, and robust task planning from internet data sources remains an open challenge. This proposal seeks to make progress towards a general robot task learning system leveraging pre-trained models and internet data. We limit our study to tasks in the cooking and food preparation domain. This proposal discusses our work in two key areas: skill learning and task planning. To improve skill learning, we present an approach for selecting between basic robot behaviors to accomplish cooking skills. We then propose further work to learn these basic robot behaviors and reduce the need for real-world execution when performing behavior selection. To improve task planning we conduct a large-scale study on Large Language Models as task planners. We identify key deficiencies and propose a new planning framework to address them. Finally, we propose a method for learning task ordering constraints from human video based on our prior constraint-learning work. Ultimately, we seek to produce a robot task-learning system capable of quickly acquiring new tasks in a home environment.Thesis Committee:
Christopher Atkeson (Chair)
David Held
Oliver Kroemer
Ruta Desai (Meta FAIR)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: MRINAL VERGHESE
Speaker Website: mrinal.verghese.org
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Accelerating Robot Task Learning with Large Pretrained Models and Internet Data
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, June 27, 2024, 10am

A grand challenge in human-computer interaction (HCI) is constructing user interfaces (UIs) that make computers useful for all users across all contexts. Conventional UI development processes have approached this goal by iteratively converging towards a single “final” UI through prototyping, implementation, and testing. However, even when following best practices, this approach locks in a set of assumptions that often cannot accommodate the diversity of user abilities, usage contexts, or computing technologies, ultimately limiting how we can use computers. For example, UIs designed for one context might not perform well in another, and UIs can be inaccessible to users with different abilities and preferences. In this dissertation, I propose a new approach that uses machine-learning-driven systems that automatically understand and manipulate existing UIs. Using content and functionality inferred from the UI, combined with sensed usage context, a new interface can be synthesized that better meets the immediate needs of individual users.

My work represents the initial technical foundation for this vision. First, I describe approaches for understanding user ability and context (user understanding), which HCI suggests is the basis for building good interfaces. I describe a recommendation system that recommends device settings (e.g., accessibility features) based on sensed usage behaviors and user interaction logs. Results from a user study showed that the majority (74%) of predicted recommendations were rated as helpful. Nevertheless, this approach of adapting interfaces through configuration changes has traditionally been limited, since applications often do not properly expose their semantics to external services. To this end, I describe several projects in the area of UI understanding, which shows that it is possible to overcome this barrier using data-driven ML models that predict interface layout, structure, and functionality from visual information, which is how UIs are generally assumed to be used. These predicted semantics can enable many forms of existing computing infrastructure, such as accessibility and UI agents to work more reliably and robustly. Finally, I combine both user and UI understanding to dynamically generate and adapt UIs that meet the specific needs of users. I describe ML-driven systems that generate UIs by modifying existing application layouts and generating UI code based on personalized user profiles and design objectives. Ultimately, through my work, I show that computational understanding of user interfaces allows UIs to be transformed from static objects into malleable representations that can be dynamically reshaped for new devices, modalities, and users.

Thesis Committee
Jeffrey P. Bigham (Chair)
Jodi Forlizzi
Tom Mitchell (MLD)
Sherry Tongshuang Wu
Jeffrey Nichols (Apple)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Mauldin Auditorium, Newell-Simon 1305 and Zoom
Speaker's Name: JASON WU
Speaker Website: jasonwunix.com
Speaker's Professional Title: Ph.D. Candidate, Human-Computer Interaction Institute, Carnegie Mellon University
Talk Title: Computational Understanding of User Interfaces
For More Information: lbuffing@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Program Website
Event Website URL: www.hcii.cmu.edu…

View Details

Friday, August 2, 2024, 8am

Red Bull Basement empowers student innovators in all areas of study to kickstart their ideas using technology to drive positive change. The aim is to connect and inspire all students to constructively challenge the status quo and make positive impacts, now and for the future.

You don’t have to be an entrepreneur or an inventor to join – everyone is invited to the brainstorm. The climax is a Global Final (2-5 December 2024 in Tokyo), where finalist teams bring their ideas to life for a pitch to global judges. Along the way are opportunities for networking, expert mentoring and more.

Every idea is welcome – you can offer a solution for a pressing, unmet need affecting life on campus or a greater challenge in the community. Technology and Artificial Intelligence may be the whole solution, or one part of it. Topics are limited only by your imagination, but to get started, think of: AI, ESG Innovation, Extended Realities or Robotics & Automation.

Applications open in August 2024, and we've made the process smoother than ever with our newest innovation: an AI Chatbot assistant!  Be among the first to experience this cutting-edge tool that simplifies bringing your idea to life on paper.  The chatbot will guide you through each step, making it easier to craft a compelling application. Don't miss out—register to receive an alert as soon as applications go live. No matter your background, this is your chance to contribute to positive change with the power of your ideas.

• Individuals aged 18 and above, proficient in English and residing in any of the participating countries, are invited to apply—either solo or in pairs. Regardless of your educational background or professional field, we welcome you to present an idea with the potential for significant impact.
• Applications can be submitted individually or in groups of two participants. More team members can collaborate behind the scenes, but only up to two selected representatives will introduce and pitch the idea at the Global Final.
• The team honored as Global Winner at the Global Final will receive a custom package of resources to realize their unique idea.

View Details

Tuesday, June 25, 2024, 12:30pm

The value alignment problem considers how robots can learn to behave in accordance with human values. Today, robot learning paradigms enable humans to provide data (e.g., preference labels or demonstrations), which the robot uses to update its behavior (e.g., reward model or policy) to be closer to the human’s values. However, the current paradigm requires the user to constantly supervise, provide new feedback, and—more fundamentally—perfectly understand where the robot is misaligned. Even if the robot eventually learns a perfect model of how the user wanted it to behave, the overall human-robot interaction during alignment could have been demanding, confusing, or arduous for the person.

This dissertation proposes that alignment isn’t just about a robot’s understanding of the task; it must also account for the overall interaction during the alignment process. In other words, alignment shouldn’t be treated as just a destination; it’s a journey. To achieve this goal, we break down the value alignment process into two levels: task alignment (wherein the robot understands and behaves in accordance with the human’s goals and intents) and interaction alignment (wherein the robot communicates with and seeks feedback from the user in accordance with the user’s interaction preferences). In our completed work, we enable task alignment in multi-agent collaborative games via online strategy adaptation and propose a way for shared robot controllers to measure their misalignment via conformal prediction. We further demonstrate a first step towards interaction alignment via proactive robot strategy explanations. In our proposed work, we will (1) formalize how different types of uncertainty (ie. action uncertainty, ambiguous human instruction, skill failure) inform different interactive, in-the-moment queries to the user, and will (2) mathematically model interaction-alignment over both physical and communicative robot behaviors. Our ultimate goal is to enable robots that are proactive participants in the value alignment process: robots that seek feedback strategically when they are uncertain during task execution and are cognizant of the capacity for human feedback constrained by the human’s state and the nature of the task.

Thesis Committee:
Henny Admoni (Co-chair)
Reid Simmons (Co-chair)
Andrea Bajcsy
Anirudha Majumdar (Princeton University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: MICHELLE ZHAO
Speaker Website: mzhao98.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Aligning Robot Task and Interaction Policies to Human Values
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, June 25, 2024, 11am

Although agriculture is a highly mechanized industry, numerous sectors like horticulture and floriculture heavily depend on manual labor because they require safe handling of plants and produce that can only be left to humans. However, many research and commercial robots have succeeded in several challenging dexterous manipulation tasks like harvesting, pruning, and plant health monitoring. These robots are typically designed as monolithic systems that are expensive to develop and, as a consequence, are prohibitively costly to adopt for regular farm use.

Using modular, reconfigurable robots (MRRs) for mobile robot manipulation in agriculture enables the automation of complex tasks with versatile and robust systems that offer cost-effective designs tailored to specific requirements. Researchers also benefit from MRRs, as they reduce non-recurring engineering costs, freeing up resources for critical areas of investigation. However, using MRRs presents a significant challenge: given an MRR system, how can a user generate the optimal design for their task? This problem is difficult, even for experts, and becomes more complex for mobile manipulation robots operating in highly cluttered environments.

In this thesis, we introduce an automated robot reconfiguration pipeline designed to solve the challenge of obtaining the best robot design for the task when using modular reconfigurable robots. We adopted a robot optimization framework that breaks down the design problem into (a) the robot model, (b) task and environment representations, (c) the task performance metric, and (d) the optimizer. This approach allows the users to choose their preferred MRR system, task description, and environment.

We introduce new task metrics evaluation methods for manipulation and navigation to deal with highly cluttered environments. For manipulation, we develop a new inverse kinematics solver for redundant manipulators that can efficiently obtain a solution if it exists. We also introduce a new approach to measuring the robot’s capacity to navigate a crop field without colliding with obstacles. With the addition of better environment representations, we seek to obtain the best MRR configurations for mobile manipulation in real-world agricultural tasks.

The results of this work will enhance the adoption of mobile manipulation robots for challenging agricultural tasks that require specialized skills and often go understaffed. Many stand to benefit from the automated robot reconfiguration tools proposed in this work: farmers and agronomists will improve labor efficiency, experts will save time, and researchers will reduce R&D costs.

Thesis Committee:
George Kantor (Chair)
Oliver Kroemer
Zeynep Temel
Changying (Charlie) Li (University of Florida)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and VIrtual - ET
Building: Newell-Simon 3305
Speaker's Name: DOMINIC GURI
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Optimal Modular Robot Design for Mobile Manipulation in Agriculture
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Wednesday, June 26, 2024, 12 – 2pm

►  SHASANK CHAVAN
      Vice President, Data In-Memory and AI Technologies, Oracle
      —  Leveraging Generative AI with Oracle AI Vector Search

AI Vector Search in Oracle 23ai is a new, transformative way to intelligently search through your unstructured business data efficiently, and accurately, by using AI techniques to match on the semantics, or meaning, of the underlying data. With the inclusion of a new VECTOR datatype, new approximate search indexes, and new SQL operators and extensions, enterprise companies can quickly and easily leverage AI Vector Search to build modern, generative-ai applications with just a few lines of SQL! And with this simplicity comes power, as AI Vector Search is fully integrated with Oracle’s enterprise-grade functionality, such as transactions, RAC, and Exadata. This talk will dive into the mechanics of AI Vector Search, ensuring a solid understanding of its implementation and benefits.

Shasank Chavan is the Vice President of the Data, In-Memory and AI Technologies group at Oracle. He leads an organization of brilliant engineers working on the nexus between AI systems and modern databases. His team is currently hyper-focussed on developing the next-generation, AI-centric data storage engine, designed for in-memory OLTP, Analytics and Vector Search capabilities to power the AI and Generative AI revolution to come. Shasank earned his BS/MS in Computer Science at the University of California, San Diego. He has accumulated 50+ patents over a span of 25 years working on systems software technology.

►  GREG GANGER
      Jatras Professor of Electrical and Computer Engineering, Carnegie Mellon University
      Director, Parallel Data Laboratory (PDL)
      —  Cluster Storage Systems Need Declarative IO Interfaces

Storage systems continue to be built around decades-old imperative interfaces, like read/write and get/put. Although this low-level interface can be used for any framework or application, it can lead to significant IO inefficiencies, especially in cases (e.g., data maintenance tasks like compaction, integrity checks, rebalancing, etc.), for which caches tend to be least effective. Although not a new fact, IO efficiency is reaching emergency status, as the IOPS/TB (or BW/TB) available from each storage device in large-scale cluster storage drops with each increase in device capacity...new approaches are needed to more efficiently use the IOPS/TB available.

It's time to augment cluster storage with declarative interfaces, whereby data maintenance tasks and data management applications can register need for sets of data items and allow the storage system to orchestrate the corresponding IO. So, rather than converting order-flexible and time-flexible needs into an arbitrary ordering of "do this now" imperative IO, the flexibility can be exposed to and exploited by the storage system. With this flexibility, significant opportunities arise for eliminating redundant IO (e.g., data read for an integrity check could also be used for rebalancing), smoothing IO bursts, and coelescing IOs.

This talk will describe the declarative IO concept, argue for their importance, talk about our early exploration into them, and invite discussion and collaboration.
 
Greg Ganger is the Jatras Professor of ECE and CS (by courtesy) at Carnegie Mellon University (CMU). Since 2001, he has also served as the Director of CMU's Parallel Data Laboratory (PDL) research center focused on data storage and processing systems. He has broad research interests in computer systems, including storage/file systems, cloud computing, ML systems, distributed systems, and operating systems. He earned his collegiate degrees from the University of Michigan and did a postdoc at MIT before joining CMU. He still loves playing basketball..he's lost a step but developed a sweet 3-point shot. And, no, the surfing pictures are not photoshopped.

Zoom Participation.  See announcement.

Event Type: Talks
Room Number: Virtual Presentations
Building: Remote Access - Zoom
Speaker's Name: Two Talks: SHASANK CHAVAN, GREG GANGER
For More Information: karenl@andrew.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): Department of Electrical and Computer Engineering, SCS
Event Website Title: Event Website
Event Website URL: pdl.cmu.edu…

View Details

Thursday, June 20, 2024, 11am

Wheel odometry is not often used in state estimation for off-road vehicles due to frequent wheel slippage, varying wheel radii, and the 3D motion of the vehicle not fitting with the 2D nature of integrated wheel odometry. This paper proposes a novel 3D preintegration of wheel encoder measurements on manifold. Our method additionally estimates wheel slip, radii, and baseline online to improve accuracy and robustness. Further, due to the preintegration, many measurements can be summarized into a single motion constraint using first-order updates for wheel slippage and intrinsics, allowing for efficient usage in an optimization-based state estimation framework. While our method can be used with any sensors in a factor graph framework, we validate its effectiveness and observability of parameters in a vision-wheel-odometry system (VWO) in a Monte Carlo simulation. Additionally, we illustrate its accuracy and robustness in real-world off-road scenarios in both a VWO and visual-inertial-wheel odometry (VIWO) system.

Committee:
Michael Kaess (Advisor)
George Kantor
David Wettergreen
Dan McGann

In Person and Zoom Participation.  See announcement.

Event Type: Speaking Skills
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: EASTON POTOKAR
Speaker Website: eastonpots.com
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Robust Off-road Wheel Odometry with Slip Estimation
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, June 20, 2024, 1:30pm

The Frontiers of Knowledge Awards, established in 2008, recognize and reward contributions of singular impact in science, art and the humanities, privileging those that significantly expand the frontiers of the known world, open up new fields, or emerge from the interaction of various disciplinary areas.

SCS Founders University Professor Takeo Kanade will receive the BBVA (Banco Bilbao Vizcaya Argentaria) Foundation's Frontiers of Knowledge Award in Information and Communication Technologies (2023) for developing mathematical foundations that underlie the current capabilities of computers and robots to comprehend and interpret visual images and scenes.

The event will be streamed live on the BBVA Foundation website.

Additional Details

Event Type: Special Events
Room Number: Livestream
Building: 17:30 pm Spanish Standard Time | 1:30 pm Eastern Standard Time
Speaker's Name: Honoring TAKEO KANADE
Speaker Website: www.frontiersofknowledgeawards-fbbva.es…
For More Information: akohlhaa@andrew.cmu.edu

View Details

Thursday, June 20, 2024, 10am

In this proposal, I investigate the collaboration challenges between software engineers and data scientists in building machine learning (ML) products, and propose interventions to facilitate their collaboration by bridging the identified knowledge boundaries.

Despite significant advancements in ML algorithms and model development, integrating ML models into operational products remains challenging, with collaboration issues frequently cited as one of the major challenges. I identify collaboration challenges, and triangulate them with existing domain knowledge through a qualitative interview study with industry practitioners and a comprehensive meta-summary study of academic literature. I demonstrate principles or ideas of how those collaboration problems can be solved, illustrated with three interventions: (a) a novel approach for supporting data scientists and software engineers in deriving actionable model requirements, which aims to bridge gaps during the requirements elicitation process, (b) an innovative method to engage practitioners in responsible AI practices, fostering a culture of ethical awareness and compliance, and (c) a policy for guiding the development of explainable AI, ensuring transparency and understandability of ML models within products. These interventions are designed to address the syntactic, semantic, and pragmatic knowledge boundaries that hinder effective teamwork in ML product development. Lastly, I compile a comprehensive dataset of ML products from GitHub to further support research and education in the domain. The methodological approach combines various research techniques tailored to address the specific research questions in each study.

By systematically identifying and addressing collaboration challenges among practitioners, this proposal aims to support the successful development and deployment of ML products in real-world settings.

Thesis Committee:
Christian Kästner (Chair)
Jim Herbsleb
Claire Le Goues
Ken Holstein
Samir Passi (Microsoft Research)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: TCS Hall 360 and Zoom
Speaker's Name: NADIA NAHAR
Speaker Website: sites.google.com…
Speaker's Professional Title: Ph.D. Student, Ph.D. Program in Software Engineering, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Facilitating Collaboration in Building Machine Learning Products
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: aroudebu@andrew.cmu.edu | cherold@cs.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Friday, June 21, 2024, 1pm

Advances in artificial intelligence (AI) enable impressive new technical capabilities: computers can diagnose diseases, translate between languages, and drive cars. Interestingly, today nearly 90% of AI initiatives fail; few projects survive until deployment. I argue that a lack of effective ideation leads teams to select suboptimal innovations to pursue. In addition, AI product teams fail to see low-hanging fruit, situations where simple predictive models can generate value for users and stakeholders. Currently, data science teams propose innovations customers do not want, while product teams ask for things AI cannot do. As AI capabilities become more pervasive and commoditized, discovering the right human problems to solve while mitigating potential harm remains a great challenge.

My research addresses this breakdown in early stage ideation and problem formulation. I studied practitioners and observed that teams better at ideating are more effective in developing AI solutions that generate value and minimize risk. Based on the industry best practices, I created new innovation processes and resources for helping cross-functional product teams effectively explore the AI solution space before selecting what to implement. I developed a taxonomy of AI capabilities and examples of these in product forms. These resources sensitize stakeholders to what AI can do and search for opportunities where these might be valuable. I developed a hybrid ideation method that blends technology-centered development and human-centered design. I conducted a preliminary assessment of these resources and processes through case studies with innovation teams working in critical care, radiology, insurance, and accounting. Overall, this dissertation provides a glimpse into the future of human-centered AI innovation, where human needs and concerns are given equal importance as technical advances in deciding what to build with artificial intelligence.

Thesis Committee:
James McCann (Co-Chair, Robotics Institute)
John Zimmerman (Co-Chair, HCII)
Jodi Forlizzi
Kayur Patel (Meta)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: ASA Conference Room, Gates Hillman 6115 and Zoom
Speaker's Name: NUR YILDIRIM
Speaker Website: nuryildirim.github.io
Speaker's Professional Title: Ph.D. Candidate, Human-Computer Interaction Institute, Carnegie Mellon University
Talk Title: Discovering the Right Things to Design with Artificial Intelligence
For More Information: lbuffing@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Program Calendar
Event Website URL: www.hcii.cmu.edu…

View Details

Thursday, May 30, 12pm – Monday, June 24, 2024, 1pm

Capital One is excited to host a variety of virtual and in-person events aimed at supporting you as you continue to navigate the recruiting process. Please see below for the event details and registration information for our upcoming events.

A full list of virtual Capital One events can be found here.

Virtual Information Sessions:

• May 30 — 12:00 ET → Students & Grads Virtual Event: Learn About Capital One's 2025 Tech Programs & Recruiter Q&A   ⇒  Register
• May 31 — 12:00pm ET  → Students & Grads Virtual Event: Learn About Capital One’s 2025 Programs & Recruiter Breakouts    ⇒   Register
• June 3 — 4:00pm ET → Students & Grads Virtual Event: Learn About Capital One’s 2025 Programs & Recruiter Breakouts   ⇒   Register
• June 12 — 12:00pm ET → Students & Grads Virtual Event: Learn About Capital One's 2025 Tech Programs & Recruiter Q&A  ⇒  Register
• June 18 — 12:00pm ET → Students & Grads Virtual Event: Learn About Capital One's 2025 Tech Programs & Recruiter Q&A  ⇒  Register
• June 24 — 12:00pm ET → Students & Grads Virtual Event: Learn About Capital One's 2025 Tech Programs & Recruiter Q&A  ⇒  Register

• June 7 — 1:00pm ET → Students & Grads Virtual Event: Tech Case Workshop  ⇒  Register
• June 17 — 12:00pm ET → Students & Grads Virtual Event: Tech Case Workshop   ⇒  Register

• May  30 — 4:00 PM ET → #StartHere: Your Life at Capital One   ⇒  Register

View Details

Thursday, May 30, 2024, 5pm

In interactive computing devices (VR/XR headsets), users interact with the virtual world using hand gestures and body actions. Typically, models deployed in such XR devices are static and limited to their default set of action classes. The goal of our research is to provide users and developers with the capability to personalize their experience by adding new action classes to their device models continually. Importantly, a user should be able to add new classes in a low-shot and efficient manner, while this process should not require storing or replaying any of the user's sensitive training data. We formalize this problem as privacy-aware few-shot continual action recognition. 

Towards this end, we propose POET: Prompt-Offset Tuning. While existing prompt tuning approaches have shown great promise for continual learning of image, text, and video modalities; they demand access to extensively pre-trained transformers. Breaking away from this assumption, POET demonstrates the efficacy of prompt tuning a significantly lightweight backbone, pre-trained exclusively on the base class data. We propose a spatio-temporal learnable prompt tuning approach, and apply additive prompts to Graph Neural Networks. We demonstrate our method on two new benchmarks for 3D skeleton human activity recognition and hand gesture recognition.

Thesis Committee:
Prof. Fernando De La Torre (Advisor)
Prof. Deva Ramanan
Prof. Kris Kitani 
Russell Mendonca 

In Person and Zoom Participation. See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: PRACHI GARG
Speaker Website: prachigarg23.github.io
Speaker's Professional Title: Masters Student, Robotics Institute, Carnegie Mellon University
Talk Title: Continual Personalization of Human Actions with Prompt Tuning
For More Information: barbarajean@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, May 30, 2024, 3pm

Sketching refers to a wide variety of techniques to compress large datasets into much smaller forms that can be efficiently processed to answer questions about the original dataset. Over the past few decades, sketching has emerged as a key tool to efficiently handle large datasets in majorly three settings: (i) the Classic setting, in which the dataset is given to us and we want to solve a problem as quickly as possible, (ii) the Streaming setting, in which the underlying dataset is defined by a large stream of updates and we want to compute interesting properties of the dataset using a small amount of space, and (iii) the Distributed setting, in which the dataset of interest is split among multiple servers and we want protocols that use a small amount of communication among servers to solve problems of interest on the underlying dataset.

Each of the above settings presents a different challenge with regard to the measure of efficiency we are interested in. In this thesis, we study sketching algorithms in these three settings for a variety of problems. While the techniques required to obtain our algorithms differ across problems and settings, the underlying idea of (possibly randomized) data compression to convert the original large dataset into a much smaller form is a key ingredient behind all of the results in this thesis.

Thesis Committee:
David P. Woodruff (Chair)
Pravesh K. Kothari
Richard Peng
Rasmus Pagh (University of Copenhagen)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates HIllman 4405 and Zoom
Speaker's Name: PRANEETH KACHAM
Speaker Website: www.praneethkacham.com
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: On Efficient Sketching Algorithms
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, June 3, 2024, 11am

Optimal Control is a popular formulation for designing controllers for dynamic robotic systems. Under the formulation, the desired long-term behavior of the system is encoded via a cost function and the policy, i.e. a mapping from the state of the system to control commands, to achieve the desired behavior are obtained by solving an optimization problem. A fundamental challenge in scaling up policy optimization to complex systems is that the computational requirement scales exponentially with the dimensionality of the state-space. Owing to this curse of dimensionality simplifying hierarchies are employed to reduce the computational burden. Very often, these hierarchies are hand-designed based on intuitions about the system’s dynamics, and do not account for their effect on the system’s closed-loop behavior under the resulting policies. The systematic design of hierarchies to simplify controller synthesis is a critical and active area of research and is the focus of this work.

This thesis introduces Policy Decomposition, a framework that alleviates the curse of dimensionality by algorithmically reducing a complex policy optimization problem into a hierarchy of simpler subproblems that are much more tractable to solve. Two standout features of this framework are its ability to 1) automatically propose control hierarchies and 2) estimate a priori how the control performance under policies resulting from different hierarchies compares with the optimal policy. Additionally, we develop search methods based on Genetic Algorithm and Monte Carlo tree search to automatically discover promising hierarchies. Therefore, hierarchies that dramatically reduce the required computation in policy optimization while sacrificing minimally on control performance can be readily identified. The framework is agnostic to the choice of policy representations and optimization algorithms.

We demonstrate the generality of the Policy Decomposition framework by applying it towards finding hierarchies for several robotic systems, including the control of a simplified biped, and a quadcopter. Furthermore, we present results using Policy Iteration with look-up table based policy representations as well as more modern methods such as Proximal Policy Optimization with neural network policies. The discovered hierarchies either outperform heuristically constructed ones in closed-loop performance or provide dramatic reductions in required compute but marginally suboptimal control performance.

Thesis Committee:
Hartmut Geyer (Chair)
Christopher Atkeson
Zachary Manchester
Nikolai Matni (University of Pennsylvania)
Alex Gorodetsky (University of Michigan)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: ASHWIN KHADKE
Speaker Website: www.andrew.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Robotics Institute, Carnegie Mellon University
Talk Title: Policy Decomposition
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, May 28, 2024, 1pm

Transformer is a neural network architecture centered on the self-attention mechanism. In recent years, it has become the de-facto architecture for deep learning, e.g., Large Language Models (LLMs) and Vision Transformers (ViTs). However, these models, with millions to billions of parameters, remain largely opaque and their mechanisms are difficult to interpret. As their real-world applications grow, gaining a deep understanding of their internal representations is essential for effectively utilizing and improving these models.

In this work, we closely examine the activation landscape in Transformers. We demonstrate that understanding the intriguing activation phenomena in Transformers can have practical and meaningful implications. First, we identify a fundamental limitation of the well-established magnitude pruning method, where it fails to consider the existence of features with large activations in large-scale Transformers. Leveraging this key insight, we develop a simple and effective pruning approach. Second, we discover and study the presence of very few activations with extremely large magnitudes, which we call massive activations. We investigate the role of massive activations in Transformers and show how they are fundamentally connected to the self-attention mechanism. Last, we discuss our proposed extensions of this work, primarily focusing on developing a unified framework for LLM compression, through a principled investigation of existing works.

Thesis Committee:
J. Zico Kolter (Chair)
Graham Neubig
Aditi Raghunathan
Kaiming He (Massachusetts Institute of Technology)

Additional Information

Event Type: Thesis Proposals
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: MINGJIE SUN
Speaker Website: eric-mingjie.github.io
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Understanding and Leveraging the Activation Landscape in Transformers
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Wednesday, May 29, 2024, 2pm

Foundational Machine Learning (ML) models are trained on massive datasets collected and curated over months and years and largely deployed in hyperscale datacenters.  As a result, such collection and training are strongly decoupled in which learning is not urgent. This is optimal when the cost of an incorrect response to a query is not catastrophic. However, ML systems that operate in adversarial or hostile physical environments must learn in a unique, edge computing paradigm on limited a priori training data.  This proposal introduces Survival-Critical Machine Learning (SCML), a new ML framework that defines optimal system design and operation that maximizes an ML system’s survivability in adversarial environments. In such scenarios, threats can evolve and morph continuously and an incorrect inference can potentially be catastrophic. In classical ML, the key performance indicator (KPI) of a system is some measure of output accuracy or quality with respect to the input.  In SCML, the KPI is defined by the time delay between the initial arrival of new data and when the derived knowledge from it is embedded in a newly deployed model, when learning is urgent.

In order to maximize survivability by adapting to evolving threats, SCML systems leverage Live Learning. Live Learning tightly couples data collection, inference, transmission, labeling, and training.  It is feasible to implement SCML, which is a new model of continuous learning, for improved survival of edge-based systems in adversarial environments. This work defines critical enhancements to Live Learning and its integration into a scalable and distributed SCML system. It explores the optimization of both traditional system-level design and properties as well as ML-focused improvements.  Combined, these minimize an SCML system’s exposure to threats, thus maximizing its survivability in adversarial or hostile environments. We seek to inform and influence future SCML system design by modeling and evaluating the behaviors of a team of distributed SCML systems that perform Live Learning.  

Thesis Committee:
Mahadev Satyanarayanan (Chair)
Rashmi Vinayak
Jeff Schneider
Babu Pillai (Intel Labs)

Additional Information

Event Type: Thesis Proposals
Room Number: In Person
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: ERIC STURZINGER
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Survival-Critical Machine Learning
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Tuesday, June 4, 9am – Wednesday, June 5, 2024, 5pm

SME is excited to host the Smart Manufacturing Experience on June 4-5, 2024, in Pittsburgh. This event will showcase the latest advancements in smart manufacturing technologies, with a focus on adoption and implementation to transform your manufacturing operations to digital technologies. Career development and training opportunities will also be a major focus to support small to mid-sized businesses in finding and retaining a skilled workforce.

The Smart Manufacturing Experience is the premier event focused on the disruptions transforming manufacturing, including:

• Additive Manufacturing (AM)
• Artificial Intelligence (AI) and Machine Learning
• Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR)
• Automation and Robotics
• Cybersecurity
• Data Analytics
• Industrial IoT: Sensors to Platform
• Business and Workforce Transformation

View Details

Tuesday, May 21, 8:30am – Friday, May 24, 2024, 5pm

Invitation Only

Event Type: Corporate Visit
Affiliations: Partnerships
Organization(s): School of Computer Science

View Details

Monday, May 20, 2024, 11am – 12:15pm

Before AI showed up, software engineers spent only 20-30 percent of their time coding, with the remaining time dedicated to discussing their code. With the AI revolution, stakeholders expect that ratio to decrease even more. Therefore, developers will be increasingly expected to excel in communication, writing, presenting, articulating a technical vision, resolving conflicts, and collaboration. Regarding developer productivity, it's crucial to remember that the ultimate goal is not to produce code but instead deliver outcomes that delight our customers. This shift in perspective from "output" to "outcome" means we must hone our communication skills and foster creativity, as these tools will help us build products that truly satisfy our customers. Investing in your personal growth and developing skills beyond coding allows you to gain more recognition, unlock numerous opportunities, and effectively advocate for your ideas. By attending this talk, you can gain valuable insights and practical tips on fostering better collaboration, practicing active listening, and building meaningful relationships with stakeholders and colleagues, all of which will enhance your professional development. In this talk, Chris will explore creativity, productivity, and communication. If you are an engineer or work with engineers, you will find this talk valuable and entertaining.

—

Chris Laffra has been a staff+ engineer at Uber, Google, IBM, and JPM. He is the author of the C4E book and is currently developing a product, to innovate in the Python Data Analytics space.

Faculty Host: Brad Myers

In Person and Zoom (Internal to CMU only) and Livestream (All Guests). See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Livestream
Speaker's Name: CHRIS LAFFRA
Speaker Website: chrislaffra.com
Speaker's Professional Title: Founder, PySheets
Talk Title: Creativity, Productivity and Communication
For More Information: rpetaeiz@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science

View Details

Monday, May 20, 2024, 12pm

Automatic speech recognition (ASR) is an essential technology which facilitates effective human-computer interaction. With the rapid progress in deep learning techniques, end-to-end (E2E) neural network-based ASR has brought significant advancements with remarkable performance. The success of ASR models have inspired various applications such as virtual assistants and automatic transcription services. Despite these achievements, recognizing conversational speech remains a challenging task, especially in the presence of environmental noise, room reverberations and speech overlaps.

This thesis aims to address the challenges of recognizing everyday conversation speech in ASR systems using E2E neural networks. The proposed research will explore techniques and methodologies to enhance the performance of ASR in challenging real-word conversational scenarios. We divide the problem into several sub-problems focusing on speech overlaps, noise, and reverberations, where each of these factors will be individually analyzed and addressed. In addition, we conduct diverse investigations on E2E neural network architectures to leverage the benefits of joint training to handle these challenges. Specifically, we build E2E ASR models by integrating ad-hoc modules, including speech enhancement, feature extraction and speech recognition.

We begin with the fundamental task of speech recognition using single-channel input containing a single speaker. Environmental noises and room reverberations significantly degrade speech recognition performance in such scenarios. To address this challenge, we propose a novel model architecture, integrating speech enhancement, self-supervised learning, and ASR models into a single neural network with an efficient training strategy. This integration has led to notable performance improvements, demonstrating the feasibility and effectiveness of employing end-to-end (E2E) neural networks for speech recognition with complex acoustic and linguistic properties. We then extend our approach to accept multi-channel speech input with a single speaker. Inspired by recent advancements in large speech foundation models, we expand the capabilities of a model trained on thousands of hours of single-channel speech data to handle multi-channel input. This extension significantly enhances performance, particularly evident in real meeting transcription data. Furthermore, we address the challenge of speech overlaps, an area that has been underexplored. Overlapping speech poses difficulties in accurately decoding and aligning individual utterances. To tackle this, we propose several end-to-end (E2E) models designed specifically to recognize overlapping speech within single-channel input. Finally, we turn our attention to multi-channel speech input with speech overlaps present in the signal. We introduce a model capable of processing multi-channel input from multiple speakers, leveraging spatial information for improved performance. We also integrate various approaches proposed earlier, further enhancing its effectiveness in challenging scenarios.

Thesis Committee:
Shinji Watanabe (Chair)
Bhiksha Ramakrishnan
Rita Singh
Naoyuki Kanda (Microsoft Research)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: ASA Conference Room, Gates Hillman 6115 and Zoom
Speaker's Name: XUANKAI CHANG
Speaker Website: www.xuankaic.com
Speaker's Professional Title: Ph.D. Candidate, Language Technologies Institute, Carnegie Mellon University
Talk Title: Everyday Conversation Speech Recognition with End-to-End Neural Networks
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: StaceyYoung@cmu.edu
Affiliations: Language Technologies Institute (LTI)
Organization(s): School of Computer Science

View Details

Thursday, May 9, 2024, 5:30 – 6:30pm

At AMD we push the limits of innovation to solve the world’s most important challenges!
 
Please join us for the Discover AMD Speaker Series: An Hour with the University Relations Team. This is an opportunity for you to ask questions and learn more about the AMD intern/co-op program and timelines, the application process, and our company culture.

REGISTER  ⇒  5:30 pm ET    [4:30 pm CT]

Event Type: Career Presentation
Room Number: Virtual Presentation
Speaker's Name: An Hour with the University Relations Team
Event Website Title: SCS/ECE Careers
Event Website URL: www.cs.cmu.edu…

View Details

Thursday, May 9, 2024, 12:30 – 1:30pm

Production agriculture plays a critical role in our lives, providing food security and enabling sustainability. Despite its immense importance, it currently faces many challenges including shortage of farmworkers, increasing production costs, excess use of herbicides just to name a few. Robotics and artificial intelligence-based technologies have the potential to increase the efficiency, long-term sustainability, safety, and profitability of agricultural production methods. Application of these advanced technologies often leads to proof-of-concept robotic systems that demonstrate novel concepts to do things that have never been done before. In this talk, I will showcase some of my recent research efforts towards robot enabled agricultural systems ranging from imaging systems that enable robust perception in the outdoor, autonomous robots for time critical tasks such as weeding, pruning, phenotyping etc., to robot systems for vertical farming. I will discuss key lessons learned implementing these systems in commercial farms, while highlighting its complexity and challenges it presents to robotics, but also the societal benefit it offers through technological advancements. Finally, I will provide some thought on key research challenges looking forward and envision the next generations of agricultural robot systems that we should build focusing on low-cost designs, general purpose use, and entrepreneurship.

—

Abhisesh Silwal (Abhi) is a Senior Project Scientist at the Field Robotics Center, Robotics Institute, Carnegie Mellon University. Before this appointment, he also completed his post-doctoral fellowship from CMU, received his Ph.D. from Washington State University and holds a Master’s degree in Mechatronics Engineering, and an Undergraduate degree in Electrical Engineering. His research interest includes systems research at the intersection of multi-disciplines of robotics including perception, navigation, manipulation and Artificial Intelligence. His research aim is to bring the state of the art from all disciplines of robotics to provide robust robotic solutions to agricultural and field robotics problems.

Faculty Host: David Wettergreen

In Person and Zoom Participation. See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Newell-Simon Hall 3305 and Zoom
Speaker's Name: ABHISESH SILWAL
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Senior Project Scientist, Field Robotics Center, Robotics Institute, Carnegie Mellon University
Talk Title: Robotics and AI for Sustainable Agriculture
For More Information: hhp@andrew.cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science

View Details

Tuesday, May 7, 2024, 10am

The widespread use of large language models has resulted in a multitude of tokenizers and embedding spaces, making knowledge transfer in prompt discovery tasks difficult.

In this work, we propose FUSE (Flexible Unification of Semantic Embeddings), an inexpensive approach to approximating an adapter layer that maps from one model's textual embedding space to another, even across different tokenizers.

We introduce a third-order tensor-based representation of a model's embedding space that aligns semantic embeddings that have been split apart by different tokenizers, and use this representation to derive an approximation of the gradient of one model's outputs with respect to another model's embedding space.

We show the efficacy of our approach via multi-objective optimization over vision-language and causal language models for image captioning and sentiment-based image captioning.

Presented in Partial Fulfillment of the CSD Speaking Skills Requirement

Event Type: Speaking Skills
Room Number: In Person
Building: ASA Conference Room, Gates Hillman 6115
Speaker's Name: JOSHUA WILLIAMS
Speaker Website: jnwilliams.github.io
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Tuesday, May 7, 2024, 3:30pm

We investigate two modifications to the traditional rules of Wythoff Nim, a combinatorial game. In Wythoff Nim, players take turns removing stones from a pair of piles. In each turn, a player chooses to either (1) take any number of stones from one pile or (2) take an equal number from both. The player removing the last stone wins. A seminal result of Wythoff states that at any point in the game, the current player is in a P-position — that is, guaranteed to lose assuming the other player plays optimally — if and only if the pair of pile sizes is of the form (nø, nø^2) for some natural number n, where ø = 1.618… represents the golden ratio.

This thesis introduces a variant of Wythoff Nim, which we call W(a,b), characterized by positive integers a and b, where players' moves are constrained to either removing a multiple of a from the first pile, a multiple of b from the second, or an equal number from both. We prove that the P-positions of this variant also follow sloping “beams”, but with slopes determined by a and b. Additionally, we explore the consequences of modifying Wythoff's Nim by introducing additional fixed winning and losing positions. Our empirical observations suggest a convergence of the P-positions in the long run, resembling those of the regular Wythoff game. Together, these results shed light on the applicability of Wythoff's theory to variants of his original game.

Thesis Committee:
Danny Sleator (Advisor)
Klaus Sutner

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: MIRABEL HU
Speaker's Professional Title: Masters Student, Computer Science Department, Carnegie Mellon University
Talk Title: Exploring Variations of Wythoff Nim
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, May 6, 2024, 10am

Large language models pre-trained on extensive web corpora demonstrate remarkable performance across a wide range of downstream tasks. However, a growing concern surrounds data contamination, where evaluation datasets may unintentionally influence the pretraining corpus, potentially inflating model performance. Despite these concerns, there remains a lack of comprehensive understanding regarding how such contamination impacts the performance of language models on downstream tasks, highlighting the necessity to investigate and mitigate this issue for accurate model evaluation. In this thesis, we present a taxonomy that categorizes the various types of contamination encountered by LLMs during the pretraining phase and identify which types pose the highest risk.  We analyze the impact of contamination on two key NLP tasks—summarization and question answering—revealing how different types of contamination influence task performance during evaluation. Our findings yield concrete recommendations for prioritizing data decontamination for pretraining.

Thesis Committee:
Matt Gormley (Chair)
Lori Levin

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: MEDHA PALAVALLI
Speaker's Professional Title: Masters Student, Computer Science Department, Carnegie Mellon University
Talk Title: A Taxonomy for Data Contamination in Large Language Models
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Friday, May 3, 2024, 10am

The approximation of matrices by smaller, simpler, or structured matrices is a fundamental problem in various fields of mathematics and computer science including numerical linear algebra, graph algorithms, computational geometry, signal processing, statistics, machine learning, and optimization. Recently, matrix approximation has been particularly important in modern computing as a key technique for efficiently processing enormous datasets in running time and memory scaling linearly, or even sublinearly, in the size of the dataset. In this thesis, we develop new and improved algorithms for a wide variety of matrix approximation tasks, drawing particularly heavily from sketching and sampling techniques from randomized numerical linear algebra, as well as sparse optimization techniques. We also utilize and develop connections of these problems with the literature of geometric functional analysis.

We develop and improve foundational tools for matrix approximation, and find novel applications of these building blocks to solve central questions in matrix approximation. Some of the basic tools that we develop and sharpen include nearly optimal constructions of oblivious and non-oblivious subspace embeddings, improved low rank approximation algorithms, and new properties of L1 regularization. Using our improved understanding of these primitives, we obtain a suite of applications such as the first polynomial space algorithms for high-dimensional computational geometry, nearly optimal algorithms for active linear regression, and the first nearly optimal coresets for multiple regression and subspace approximation. Many of our results have implications in big data computing settings, such as streaming, online, and distributed computation.

Thesis Committee:
David Woodruff (Chair)
Anupam Gupta
Richard Peng
Cameron Musco (University of Massachusetts Amherst)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: McWilliams Classroom, Gates Hillman 4303 and Zoom
Speaker's Name: TAISUKE YASUDA
Speaker Website: taisukeyasuda.github.io
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Algorithms for Matrix Approximation: Sketching, Sampling, and Sparse Optimization
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Thursday, May 2, 2024, 3pm

In many systems, servers do not turn on instantly; instead, a setup time must pass before a server can begin work. These “setup times” can wreak havoc on a system's queueing; this is especially true in modern systems, where servers are regularly turned on and off as a way to reduce operating costs (energy, labor, CO2, etc.). To design modern systems which are both efficient and performant, we need to understand how setup times affect queues.

Unfortunately, despite successes in understanding setup in the single server setting, setup in the multiserver setting remains poorly understood. To circumvent the main difficulty in analyzing multiserver setup, all existing results assume that setup times are memoryless, i.e. distributed Exponentially. However, in most practical settings, setup times are close to Deterministic, and the widely used Exponential-setup assumption leads to unrealistic model behavior and a dramatic underestimation of the true harm caused by setup times.

This thesis represents a comprehensive characterization of the average waiting time in a multiserver system with Deterministic setup times, the M/M/k/Setup-Deterministic. In particular, we derive multiplicatively-tight lower and upper bounds on the average waiting time, demonstrating that setup times, along with their distributions, can not be ignored; setup times can cause profound increases in waiting time, especially when the distribution of setup time has low variability. Our bounds are the first closed-form bounds on waiting time in any finite-server system with setup times, including the extensively-studied Exponential setup system. Furthermore, we use our bounds to derive a highly-accurate approximation, which we evaluate in a variety of settings. These results are made possible via our new method for bounding the expectation of a random time integral, called the Method of Intervening Stopping Times or MIST.

Thesis Committee:
Weina Wang (Chair)
Mor Harchol-Balter
Alan Scheller-Wolf
Jamol Pender (Cornell University)
Bill Massey (Princeton University)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates Hillman 4405 and Zoom
Speaker's Name: JALANI K. WILLIAMS
Speaker Website: jalaniw.github.io
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Setup Times in Multiserver Systems
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, April 29, 2024, 12:30 – 1:45pm

User-Centered Research and Evaluation is a core course for the bachelor and master programs at the Human-Computer Interaction Institute at Carnegie Mellon's School of Computer Science. The course provides students with real-world practice of UX research methods, including data analysis, usability testing, contextual inquiry, and lo-fi prototyping.

This past semester, about 70 undergraduate and graduate students across 14 teams tackled the questions of how we can empower and incentivize everyday users to uncover harmful algorithmic behaviors in a wide range of Generative AI systems (e.g., ChatGPT, Stable Diffusion) and assist AI/ML teams in addressing these issues. During this poster session, students will present the findings from their research and propose future directions for algorithmic auditing.

We will do this in a virtual poster session style, where students will be in breakout rooms, and you can move freely to the rooms you choose to discuss the research work. This allows you to come and go as you please, so you do not have to be present for the entire hour and fifteen minutes. You can visit the Breakout Rooms by Theme and Posters.

Faculty Hosts: Raelin Musuraca, Jason Hong

Zoom Participation. See announcement.

Event Type: Project Presentations
Room Number: Virtual Presentations - ET
Building: Remote Access - Zoom Meeting and Rooms
Speaker's Name: Project: Harnessing Everyday Users to Detect Harmful Behaviors in Generative AI
For More Information: rmusurac@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science

View Details

Thursday, May 2, 2024, 12 – 1pm

Historically, computer vision systems struggle with sensor limits, bandwidth usage and power draw when applied in the wild, limiting their practicality. In this thesis, we explore how light diffraction can be used to mitigate these challenges in real applications. Thanks to the unique coding of light that diffractive optics provide, we demonstrate compressive high-speed imaging that better encodes scene content into available camera bandwidth compared to past approaches. Leveraging the unique image formation model of diffracting light, we implement high-dynamic range illumination systems that enable faster, more programmable structured light systems and longer range time-of-flight cameras without increasing power usage. With these ideas in mind, we propose a technique for sensing the fine vibrations of objects over a large field-of-view via diffraction, an approach for probing light transport based on programming the coherence of a projector system, and a methodology for spatially controlling camera resolution by using a programmable diffraction grating.

Thesis Committee:
Matthew O'Toole (Chair)
Ioannis Gkioulekas
Aswin Sankaranarayanan
Mohit Gupta (University of Wisconsin-Madison)

Additional Information

Event Type: Thesis Proposals
Room Number: In Person
Building: Reddy Conference Room, Gates Hillman 4405
Speaker's Name: DORIAN CHAN
Speaker Website: dorianchan.com
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Diffraction and Computer Vision
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Wednesday, May 1, 2024, 2pm

Fully autonomous flight by low-cost, lightweight commercial off-the-shelf (COTS) drones could transform many use cases involving real-time computer vision. We show how such autonomy can be achieved using edge computing from a flight platform costing less than $800, and composed of a 320 g COTS drone with a 26 g COTS wearable device as payload. In spite of the extreme austerity of this platform and thermal limits on its LTE transmission, the system is able to perform tasks such as detecting and then tracking a target. It is also able to visually navigate around obstacles in its flight path. Such capabilities are only found on heavier and more expensive drones today.

Presented in Partial Fulfillment of the CSD Speaking Skills Requirement

Event Type: Speaking Skills
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: MIHIR BALA
Speaker Website: www.mihirbala.com
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Democratizing Drone Autonomy Via Edge Computing
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, April 29, 2024, 12pm

Current deep-learning models for object recognition are known to be heavily biased toward texture. In contrast, human visual systems are known to be biased toward shape and structure. What could be the design principles in human visual systems that led to this difference? How could we introduce more shape bias into the deep learning models?

In this talk, I report that sparse coding, a ubiquitous principle in the brain, can in itself introduce shape bias into the network. We found that enforcing the sparse coding constraint using a non-differential Top-K operation can lead to the emergence of structural encoding in neurons in convolutional neural networks, resulting in a smooth decomposition of objects into parts and subparts and endowing the networks with shape bias. We demonstrated this emergence of shape bias and its functional benefits for different network structures with various datasets.

For object recognition convolutional neural networks, the shape bias leads to greater robustness against style and pattern change distraction. For the image synthesis generative adversarial networks, the emerged shape bias leads to more coherent and decomposable structures in the synthesized images. Ablation studies suggest that sparse codes tend to encode structures, whereas the more distributed codes tend to favor texture.

Additionally, we also propose a new benchmark to evaluate shape bias and show that sparse code empowered compositional encoding schemes provide human level shape based generalization, pushing the machine vision towards humans one step further.
 
Presented in Partial Fulfillment of the CSD Speaking Skills Requirement

Event Type: Speaking Skills
Room Number: In Person
Building: Gates Hillman 8102
Speaker's Name: TIANQIN LI
Speaker Website: crazy-jack.github.io
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Thursday, May 2, 2024, 5:30 – 8:30pm

Come See Final Projects
Open-Source Nanofabrications Tools
Built entirely by CMU Students!

Course Instructors:  Elio Bourcart, Alexander Hakim

Event Type: Project Presentations
Room Number: In Person
Building: Bosch Room, Scott Hall 5201
Speaker's Name: Open-Source Nanofabrications Tools Built by CMU Students
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: ahakim@andrew.cmu.edu
Organization(s): Department of Electrical and Computer Engineering

View Details

Wednesday, June 5 – Thursday, June 6, 2024

The CMU WebAssembly Research Center Day brings together researchers from academia and industry, as well as participants in the Community Group (CG)  standards body, to discuss research and long-term topics related to WebAssembly.  The Research Day provides a unique opportunity for participants to discuss both ongoing standardization work and strategic directions for WebAssembly.

By invitation only.

Watch for Details.

Event Type: Conference/Workshop
Room Number: In Person
For More Information: wrc-team@cs.cmu.edu
Affiliations: Partnerships, Software and Societal Systems Department (S3D)
Event Website Title: Event Website
Event Website URL: www.cs.cmu.edu…

View Details

Tuesday, June 4, 2024, 9am – 5pm

The CMU WebAssembly (WASM) Research Center Day brings together researchers from academia and industry, as well as participants in the Community Group (CG)  standards body, to discuss research and long-term topics related to WebAssembly. 

Co-located with the June 5-6 Wasm Community Group Meetiing, this events offers a unique opportunity for participants to discuss both ongoing standardization work and strategic directions for WebAssembly.

By invitation only.

More details to come.

Event Type: Conference/Workshop
Room Number: In Person
Building: ASA Conference Room, Gates Hillman 6115
For More Information: wrc-team@cs.cmu.edu
Affiliations: Partnerships, Software and Societal Systems Department (S3D)
Organization(s): SCS
Event Website Title: Event Website
Event Website URL: www.cs.cmu.edu…

View Details

Tuesday, April 30, 2024, 4:30 – 5:30pm

Private Information Retrieval (PIR) enables a client to retrieve a database element from a semi-honest server while hiding the element being queried from the server. Maliciously-secure PIR (mPIR) [Colombo et al., USENIX~'23] strengthens the guarantees of plain (i.e., semi-honest) PIR by ensuring that even a misbehaving server:

• cannot compromise client privacy via selective-failure attacks, and 
• must answer every query consistently (i.e., with respect to the same database). 

View Details

Thursday, May 2, 2024, 10am

The ever-increasing scale of foundation models, such as ChatGPT and AlphaFold, has revolutionized AI and science more generally. However, increasing scale also steadily raises computational barriers, blocking almost everyone from studying, adapting, or otherwise using these models for anything beyond static API queries. In this talk, I will present research that significantly lowers these barriers for a wide range of use cases, including inference algorithms that are used to make predictions after training, finetuning approaches that adapt a trained model to new data, and finally, full training of foundation models from scratch.  For inference, I will describe our LLM.int8() algorithm, which showed how to enable high-precision 8-bit matrix multiplication that is both fast and memory efficient. LLM.int8() is based on the discovery and characterization of sparse outlier sub-networks that only emerge at large model scales but are crucial for effective Int8 quantization. For finetuning, I will introduce the QLoRA algorithm, which pushes such quantization much further to unlock finetuning of very large models on a single GPU by only updating a small set of the parameters while keeping most of the network in a new information-theoretically optimal 4-bit representation. For full training, I will present SWARM parallelism, which allows collaborative training of foundation models across continents on standard internet infrastructure while still being 80% as effective as the prohibitively expensive supercomputers that are currently used. Finally, I will close by outlining my plans to make foundation models 100x more accessible, which will be needed to maintain truly open AI-based scientific innovation as models continue to scale.

—

Tim Dettmers’s research focuses on making foundation models, such as ChatGPT, accessible to researchers and practitioners by reducing their resource requirements. This involves developing novel compression and networking algorithms and building systems that allow for memory-efficient, fast, and cheap deep learning. These methods enable many more people to use, adapt, or train foundation models without affecting the quality of AI predictions or generations. He is a PhD candidate at the University of Washington and has won oral, spotlight, and best paper awards at conferences such as ICLR and NeurIPS. He created the bitsandbytes library for efficient deep learning, which is growing at 1.7 million installations per month and received Google Open Source and PyTorch Foundation awards.

Faculty Host: Zico Kolter, Ameet Talwalkar

 In Person and Zoom Participation.  See announcement.

Event Type: Talks
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: TIM DETTMERS
Speaker Website: timdettmers.com
Speaker's Professional Title: Ph.D. Candidate, Paul G. Allen School of Computer Science & Engineering, University of Washington
Talk Title: Accessible Foundation Models: Systems, Algorithms, and Science
For More Information: dhyde@cmu.edu
Affiliations: Computer Science Department (CSD), Machine Learning Department (MLD)
Organization(s): School of Computer Science

View Details

Tuesday, April 30, 2024, 4 – 5pm

The European Repository of Cyber Incidents (EuRepoC) is an independent research consortium dedicated to better understanding the cyber threat environment in the European Union and beyond. Launched in November 2022, EuRepoC's key objectives are to promote data-driven discussions and policymaking within the field of cyber security and raise awareness of cyber security threats by providing an analytical framework for assessing and comparing the ‘life cycle’ of cyber incidents, focusing on technical, political and legal aspects. The presentation will introduce the Repository's approach and discuss the importance of a data-driven, open source approach to analyzing cybersecurity incidents and cyber threats.

—

Matthias C. Kettemann is Professor of Innovation, Theory and Philosophy of Law and head of the Department for Theory and Future of Law at the University of Innsbruck.  He leads the University’s Quantum Ethics Lab and heads research programmes at the Humboldt Institute for Internet and Society in Berlin and the Leibniz Institute for Media Research in Hamburg. After studying law in Graz, Geneva and at Harvard Law School he taught in Frankfurt, Heidelberg and Jena. He works and publishes on the power of rules and the rules of power in online spaces and has been engaged as an expert for the Council of Europe, UNESCO, OSCE, and national parliaments and ministries.

REGISTER → Attendees must register. Confirmation email will be provided upon receipt of registration.

Event Type: Seminars
Room Number: Virtual Presentation - ET
Building: Remote Access
Speaker's Name: MATTHIAS C. KETTEMANN
Speaker Website: leibniz-hbi.de…
Speaker's Professional Title: Professor of Innovation, Theory and Philosophy of Law, Head, Department for Theory and Future of Law, University of Innsbruck
Talk Title: Promoting Cybersecurity and a Consistent Cyber Policy through Analytics: Introducing the European Repository of Cyber Incidents
For More Information: centerforideas@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): IDeaS, School of Computer Science, Block Center for Technology and Society
Event Website Title: Series Website
Event Website URL: www.cmu.edu…

View Details

Thursday, May 2, 2024, 4pm

Retinal surgery procedures require surgeons to manipulate very delicate tissues with little room for error. During epiretinal membrane surgery, to reduce chances of recurrence, surgeons may have to remove the 10 µm thick internal limiting membrane from the retinal surface. An experimental procedure to treat retinal vein occlusion is retinal vein cannulation. During this procedure, surgeons are required to inject drugs into retinal veins that are less than 100µm thick. The difficulty of performing these procedures is exacerbated by natural human hand tremor, a constrained workspace, and limited visualization. Robotic surgery systems can aid surgeons during these demanding procedures by cancelling tremor at the surgical tooltip, providing improved visualization, implementing motion scaling, and constraining the tool to remain within certain regions.

An essential piece of information that is required to create and enforce safeguards using such robotic systems is the distance of the surgical tool tip from the retinal surface. Cameras attached to the microscope above the surgical workspace are accessible sources of information about the state of the tool and the retina in real time. However, the light path from the retinal surface to the camera is complex and pictures of the retina during surgery can be featureless and cloudy. Thus, many classical computer vision techniques that assume a pinhole model for the light path fail. OCT scans do provide high resolution scans of the underlying tissue, but they can be expensive to develop and have a limited range of around 2mm.

One remedy to the lack of features in retinal images is the use of a laser-aiming beam attached to the surgical tool. The laser spot emanating from the tool is quite visible in images and the spot itself is very easy to detect. Cases where the projected laser spot cannot be detected included obfuscation by light from the light pipe. However in this case, the tool’s shadow becomes a prominent indicator of retinal distance. This research aims to estimate retinal distance with acceptable accuracy using 3 metrics present in the camera image: (i) Projected laser area (ii) Distance of the laser from the tool tip (iii) Distance of tool tip from its shadow. The proposed framework to combine these 3 metrics is a dual Kalman filter that can update both states and parameters of a system. This is necessary as parameters estimated in lab conditions will inevitably be incorrect during the unpredictable conditions of in-vivo surgery.

Preliminary results include a dual Kalman filter formulation that uses the projected laser area for surface estimation. The method is shown to be independent of microscope magnification. The filter is able to predict retinal distance with errors less than 100 µm with controlled tool movement and also shows acceptable performance during freehand motion. Some preliminary results using the other two metrics, distance of the laser from the tool tip and distance of tool tip from its shadow are also explored.

This research also aims to use the developed retinal surface estimation method to implement virtual fixtures to reduce force applied during peeling and cannulation procedures. Some virtual fixture formulations are explored and their efficacy at reducing trauma to the retina and aid better tool movement is shown using phantoms in lab conditions. This research eventually aims to show the effectiveness of the surface estimation method and virtual fixtures in assisting retinal surgery procedures in in-vivo conditions like pig eyes.

Thesis Committee:
Cameron Riviere (Chair)
John Galeotti
Zeynep Temel
Iulian Iordachita (Johns Hopkins University)

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: 1305
Building: Mauldin Auditorium, Newell-Simon 1305 and Zoom
Speaker's Name: ARPITA ROUTRAY
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Improved Surface Estimation for use in Virtual Fixtures during Retinal Surgery
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website URL: www.ri.cmu.edu…

View Details

Wednesday, May 1, 2024, 10am

We study the problem of finding a trajectory for an agent to intercept a number of moving targets while avoiding obstacles. Applications include resupplying naval ships at sea and recharging aerial vehicles with a ground vehicle. We model the problem as an extension of the traveling salesman problem, which we refer to as the moving target traveling salesman motion planning problem with obstacles (MT-TS-MP-O). Prior approaches to this problem can find feasible solutions, but do not provide bounds on the suboptimality of these solutions, due to a lack of a lower bound on the optimal cost. We develop two methods of providing lower bounds. The first method relaxes the continuity requirement on the agent’s trajectory, and the second method relaxes the agent’s collision avoidance constraints. We present the benefits and drawbacks of each method, as well as future directions.Committee:
Howie Choset
Matt Travers
Maxim Likhachev
Charles Noren

In Person and Zoom Participation. See announcement.

Event Type: Speaking Skills
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: ANOOP BHAT
Speaker Website: www.ri.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Lower Bounds for Moving Target Traveling Salesman Motion Planning with Obstacles
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, April 29, 2024, 12 – 1pm

In recent years succinct zero knowledge proof systems have become a very active area of research with many commercial applications.  In this talk we will discuss a number of recent advances in the space, as well as an important application that is outside of the realm of decentralized systems. 

—

Dr. Dan Boneh is a Professor of Computer Science at Stanford University where he heads the applied cryptography group and co-directs the computer security lab.  Dr. Boneh's research focuses on applications of cryptography to computer security.  His work includes cryptosystems with novel properties, cryptography for blockchains, web security, and cryptanalysis.  He is the author of over 200 publications in the field and is a recipient of the 2014 ACM prize and the 2013 Godel prize, and is a member of the National Academy of Engineering.

Faculty Host:  Elaine Shi

In Person and Livestream Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Simmons Auditorium A , Tepper Building and Youtube
Speaker's Name: DAN BONEH
Speaker Website: profiles.stanford.edu…
Speaker's Professional Title: Professor, Computer Science Department, Stanford University
Talk Title: Recent Developments in Succinct Proof Systems and Their Applications
For More Information: bethbuch@andrew.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): CyLab, School of Computer Science
Event Website Title: Event Website
Event Website URL: www.cylab.cmu.edu…

View Details

Wednesday, May 1, 2024, 9:45am – 3pm

Our annual tradition of undergraduate research continues next Wednesday, May 1, at Meeting of the Minds. Please come to celebrate undergraduate research and maybe even find some new students with whom to work. 

Senior Thesis presentations
→  Rashid Auditorium, Gates Hillman 4401
→  9:45 am - 3:00 pm

Abstracts and Schedule of Presentations

• 9:45-10:00 am —  Updates (works in progress)
• 10:00 am-12:00 pm — Morning Session
• 1:00-3:00 pm — Afternoon Session

View Details

Tuesday, April 30, 2024, 11am

Navigating through unfamiliar environments is a challenging task. For people who are blind or have low vision (BLV), navigation can be particularly daunting. Guide robots are a type of service robot that can assist BLV people with navigation tasks. A significant amount of research related to guide robots has focused on technical contributions, while a dearth of research has centered on exploring the affordances BLV users desire from these robots. We conducted a two-phase user study with 16 BLV participants aimed at identifying features, functions, and behaviors of guide robots that enhance BLV users' sense of orientation, perceived safety, and comfort. The first phase centered on robot interaction, during which participants were guided by a robot along pre-computed trajectories (detailed in Shih & Boadi-Agyemang et al., under review). This work focuses on the second phase, which occurred after the participants experienced navigating with a guide robot that exhibited four different behaviors. In this phase, we conducted semi-structured interviews and participatory design (PD) sessions with each of the participants. The PD sessions utilized a scenario-based design (SBD) approach aimed at identifying ideal features and functionality of guide robots and a dialogue-simulated activity to verify these requirements. We identified three key themes that serve as design considerations for future robotic mobility aids for BLV people: (1) features that enhance the usability of guide robot systems, and factors that (2) influence BLV users' desires for shared robot control and (3) enhance BLV users' perceived safety in public settings. Our work reiterates the importance of interactive PD approaches, such as SBD, for facilitating accessible idea generation and iteration when working with BLV populations.

Committee: 
Dr. Aaron Steinfeld (Advisor)
Dr. Henny Admoni
Dr. Nikolas Martelaro
Michelle Zhao

In Person and Zoom Participation.  See announcement.

Event Type: Master's Thesis Presentation
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: ABENA BOADI-AGYEMANG
Speaker Website: abena.myportfolio.com
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Simulated Encounters of the Third Kind: Scenario-Based Approach to Designing Guide Robots
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Thursday, May 2, 2024, 9am

Augmented reality (AR) glasses have become increasingly discreet and capable given the advancements in their design, sensor technology, and processing power over the past decade. Equipped with egocentric video cameras, gaze trackers, and other sensors, AR glasses offer a unique opportunity to study human behavior in an unobtrusive and naturalistic manner. This thesis specifically focuses on predicting engagement in dyadic social interactions, as engagement is a key component of human communication. The ability to understand and model engagement can augment and improve human communication, as well as inform the development of increasingly user-centric and socially intelligent technologies.

As part of this work, we assembled a novel dataset featuring 17 pairs of participants engaged in casual conversations recorded on AR glasses. Engagement was measured by each participant providing a self-reported engagement rating accompanied by external raters’ scores. Inspired by successful applications of Large Language Models (LLM) in other domains, we also introduce a novel fusion strategy to convert behavioral data into text and combine participant information to predict user engagement using multiple pretrained models and the inference abilities of LLMs. To the best of our knowledge, this is the first approach to “reason” about human behavior using language representations of non-verbal cues and LLMs. We show this can be a powerful, simple, and flexible framework for future work on modeling human behavior and the development of socially intelligent technologies.

Thesis Committee
Fernando De La Torre (Chair)
Daphne Ippolito
Lori Holt (University of Texas at Austin)

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: CHENG CHARLES MA
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Masters Student, Computer Science Department, Carnegie Mellon University
Talk Title: Large Language Model Aided Modeling of Dyadic Engagement
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Wednesday, May 1, 2024, 1:30pm

This thesis develops Monte Carlo algorithms based on the walk on spheres (WoS) method to reliably solve fundamental partial differential equations (PDEs) like the Poisson equation on geometrically complex domains. Elliptic PDEs are a basic building block of algorithms and applications throughout science, engineering, and geometric computing. Yet despite decades of research on methods for solving such PDEs, conventional solvers still struggle to deal with the level of geometric complexity found in the natural world. A constant challenge is the need for spatial discretization, which traditionally involves dividing the domain into a high-quality volumetric mesh or grid to perform PDE-based analysis. Unfortunately, this approach does not scale well to modern computer architectures as it is inherently sequential and memory intensive. It also falters when dealing with imperfect data containing poorly-shaped elements or self-intersections. These shortcomings together hinder the ability of scientists, engineers and designers to analyze geometric data and iterate on designs.

Walk on spheres makes a radical departure from conventional PDE solvers by reformulating the problem in terms of recursive integral equations that can be solved using the Monte Carlo method, allowing it to avoid volumetric mesh generation and function space approximation altogether. Furthermore, since these integral equations closely resemble those found in light transport theory, one can leverage deep knowledge from Monte Carlo rendering to build new algorithms for solving PDEs.

In this work, we take inspiration from rendering to generalize WoS to solve a much broader set of linear elliptic PDEs on solid regions of ℝ^N. We develop complete “black box” solvers encompassing integration, variance reduction and acceleration. Our solvers share many benefits with Monte Carlo methods from rendering: no volumetric meshing, trivial parallelism, output-sensitive evaluation of the PDE solution and its gradient without the need to solve a globally-coupled system of equations, and the ability to handle geometric data of size and complexity that is essentially hopeless for grid-based techniques.

Thesis Committee:
Keenan Crane (Chair)
Ioannis Gkioulekas
Matthew O'Toole
Gautam Iyer
Matt Pharr (NVIDIA)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: McWilliams Classroom, Gates HIllman 4303 and Zoom
Speaker's Name: ROHAN SAWHNEY
Speaker Website: rohansawhney.io
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Monte Carlo Geometry Processing: A Grid-Free Approach to Solving Partial Differential Equations on Volumetric Domains
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Wednesday, May 1, 2024, 2pm

We study various aspects related to the efficient training of deep networks. In doing so, we also discuss and contribute to various theoretical facets of the field.

An unavoidable component in dealing with very large models is the need for distributing the training over many computational devices. This usually induces a considerable communication overhead that increases the risk of under-utilization of the system resources. To address this, we show that the entropy of the weights decreases during training, which thus become highly compressible; allowing for a considerable reduction in said overhead.

It is common practice to use squashing functions, like the softmax, at the output layer of neural nets. We study the effect these functions have on the gradient signal and argue that they may contribute to the well-known vanishing gradient problem. To this end, we introduce non-squashing alternatives and provide evidence that suggests, that they improve the convergence rate.

Our main contribution is in layer-wise training of deep networks. First, we make various useful observations on the properties of hidden layers and representations. We then show that layer-wise training can match the results of full-model backprop, while considerably reducing the memory footprint of the training process. We discuss the effect of implicit interlayer regularization and introduce new conjectures on its theoretical origin. Based on these, we show that interlayer regularization can be simulated in a few simple steps. Additionally, we discuss partition-wise training, which may speed up the optimization process by allowing for larger batch sizes and improved model parallelism.

Finally, we take a look beyond gradient descent. A novel solution to fitting multilayer perceptrons to training data is introduced. While it can outperform backpropagation with stochastic gradient descent on various toy problems, it tends to overfit and be capacity-hungry on more complex real data. We discuss why and point to future ways of addressing this. This solution can be expressed in closed form, albeit we expect that it will evolve into a hybrid iterative approach. Also, we suspect that our method might be a substantially better candidate for training deep nets on quantum computers than backprop.

Thesis Committee:
Roger B. Dannenberg (Co-Chair)
Bhiksha Raj (Co-Chair)
Zico Kolter
Ruslan Salakhutdinov
Douglas Eck (Google DeepMind)

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Gordon Bell Conference Room, Gates Hillman 5117 and Zoom
Speaker's Name: ANDERS ØLAND
Speaker Website: scholar.google.com…
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Efficient Deep Learning
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Tuesday, April 30, 2024, 12 – 1pm

Deep Learning is driving unprecedented progress in a wide range of Artificial Intelligence domains, including natural language processing, vision, speech, and multimodal. Sustaining this rapid pace of AI revolution, however, requires practical solutions to the extreme demands of model scaling on the compute, memory, communication and storage components of modern computing hardware. To address this challenge, we created a deep learning optimization library called DeepSpeed to make distributed model training and inference efficient, effective, and easy on commodity hardware. This talk will focus on DeepSpeed optimizations for improving memory, compute, communication, and data efficiency of extreme-scale model training.

—

Olatunji (Tunji) Ruwase is the lead and co-founder of the DeepSpeed project at Microsoft. His broad industry and research background spans compilers, operating systems, and hardware accelerators. His current focus is on systems and convergence optimizations, and frameworks for efficient distributed training and inference of deep learning models.  His research results on deep learning training, inference, and hyperparameter search are used in multiple Microsoft systems and products, such as Azure, Ads, Bing, Catapult, and HyperDrive. Tunji earned a PhD in Computer Science from Carnegie Mellon University under the guidance of Professor Todd Mowry.

In Person and Zoom Participation.  See announcement.

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: Newell-Simon 3305 and Zoom
Speaker's Name: OLATUNJI RUWASE
Speaker Website: www.microsoft.com…
Speaker's Professional Title: Lead and Co-founder, DeepSpeed Project, Microsoft Research
Talk Title: DeepSpeed: Enabling efficient trillion parameter scale training for deep learning models
For More Information: vakinwan@andrew.cmu.edu
Affiliations: Computer Science Department (CSD), Language Technologies Institute (LTI), Machine Learning Department (MLD), Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Series Website
Event Website URL: www.cs.cmu.edu…

View Details

Monday, April 29, 2024, 10:30 – 11:30am

Distributional Reinforcement Learning (RL) fits Q-functions by learning the whole conditional distribution of rewards-to-go and then taking its mean (e.g., C51, IQN). Empirically it often improves on analogous approaches that learn the conditional mean directly (e.g., DQN) even in risk-neutral RL where we only care about the mean, but a principled understanding as to why and when has been elusive. We resolve this by showing that distributional RL enjoys first- and second-order regret bounds in both online and offline RL in general MDPs with function approximation. In some cases these are the first bounds of their kind for any RL algorithm. First-order bounds scale with the cost of the optimal policy and, for example, establish fast regret rates in goal-based MDPs when a policy exists reaching the goal reliably. Second-order bounds scale with the variance of the return and, for example, establish fast regret rates in nearly deterministic systems. We explain this phenomenon in terms of sensitivity to heteroskedasticity and demonstrate the predictions of the theory empirically on real-world tasks. Beyond distributional RL, I will discuss the implications for choosing the right loss function for decision making.

—

Nathan Kallus is an Associate Professor at the Cornell Tech campus of Cornell University in NYC and a Research Director at Netflix. Nathan's research interests include the statistics of optimization under uncertainty, causal inference especially when combined with machine learning, sequential and dynamic decision making, and algorithmic fairness. He holds a PhD in Operations Research from MIT as well as a BA in Mathematics and a BS in Computer Science from UC Berkeley. Before coming to Cornell, Nathan was a Visiting Scholar at USC's Department of Data Sciences and Operations and a Postdoctoral Associate at MIT's Operations Research and Statistics group.

Papers:

• https://arxiv.org/abs/2305.15703
• https://arxiv.org/abs/2402.07198

View Details

Monday, April 29, 2024, 2:30pm

Peer review has been the method of choice for academics to ensure the overall quality of published research. Conferences in computer science, which are typically the primary venues to publish research, have grown tremendously with over 10,000 papers submitted to some conferences. Consequently, many parts of the review process are now automated, notably the assignment of expert reviewers to the submitted papers. These automated assignments depend on, among other things, similarities computed between reviewers and papers using natural language processing (NLP) techniques. A higher similarity between a reviewer and a submitted paper means that this reviewer is envisaged to have higher expertise for the paper.

A major problem in peer review is the existence of collusion rings, where groups of researchers try to get assigned each others' papers and then provide positive reviews to each other irrespective of the true qualities. It is commonly believed that the NLP-matching component of reviewer assignments is safe from manipulation by collusion rings. Consequently, most work on defending or investigating collusion rings focuses on other components of the review process, assuming the NLP-matching is not manipulated.

In this thesis, we show that the automated reviewer assignments used by OpenReview, the main peer review platform for conferences in machine learning and related fields, can be manipulated to ensure fellow colluders get assigned to other colluders' papers. We demonstrate this susceptibility via methods to modify colluders' abstracts, by both manual and algorithmic means, that can fool the automated assignments. Our findings have significant implications for defending against fraudulent behavior in peer review, for fairness and integrity of science.

Thesis Committee:
Nihar Shah (Chair)
Aditi Raghunathan

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: JHIH-YI HSIEH
Speaker Website: www.linkedin.com…
Speaker's Professional Title: Masters Student, Computer Science Department, Carnegie Mellon University
Talk Title: Automated Peer-Reviewer Assignment can be Manipulated to Secure Reviews from Colluders
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Friday, May 3, 2024, 9:30am

Two decades into the third age of AI, the rise of deep learning has yielded two seemingly disparate realities. In one, massive accomplishments have been achieved in deep reinforcement learning, protein folding, and large language models. Yet, in the other, the promises of deep learning to empower robots that operate robustly in real-world environments have yet remained unfulfilled. Vast diversity of objects, distribution shifts, long-tailed phenomena: outside of laboratories, real-world environments challenge modern statistical learning assumptions of the data.

Although such environments have generally been referred to as “unstructured,” this terminology belies their nature. Real-world environments are not “unstructured,” but arise because of structure: the underlying causal processes that generate the observed data. In this view, robots should not only reason and learn with respect to data, but also the data generating processes. Such processes can be formalized by the language of causality. Therefore, to learn and leverage the structure of these “open-world” environments, new causal-based robot learning algorithms are needed.

Towards this end, this thesis explores new avenues in causal learning and reasoning for a diversity of robot learning problems, from perception to control. First, we explore how causal features can be learned for perception models using sim-to-real transfer from synthetic data (DREAM, FormNet). We continue this theme in the introduction of structural sim-to-real transfer, where simulation can serve as a causal reasoning engine for the robot to select the relevant features for a control policy (CREST) or skill (SCALE). By construction, these policies are robust to irrelevant distribution shifts that would otherwise stymie correlation-based deep learning. Next, the rich interplay between control, dynamical systems, and causality is explored through the Learning By Doing (LBD) competition and LMeshNet, a methodology for constructing hybrid causal world models that integrate both latent and semantic information. Lastly, in causal curriculum learning, we first explore the commonalities and differences between human and agent curriculum learning (ACL). Then, we employ these lessons learned for CURATE: how agents can perform interventions to manipulate the sequencing of training data to efficiently learn a control policy to solve a desired, difficult task.

The advantages of adopting the principles of causal inference have been witnessed to date in fields such as biomedical sciences, economics, and genomics. In the machine learning community, it has recently been argued that such principles should be integrated to harness deep learning, towards causal learning of representations. Analogously, this thesis forwards that the robot learning community stands to gain by leveraging the principles of causality. In so doing, this new paradigm holds promise for robots to learn and leverage structure within the open world through causal robot learning for manipulation.

Thesis Committee:
Oliver Kroemer (Chair)
Shubham Tulsiani
Kun Zhang
Jonas Martin Peters (ETH Zürich)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Mauldin Auditorium, Newell-Simon 1305
Speaker's Name: TABITHA EDITH LEE
Speaker Website: www.andrew.cmu.edu…
Speaker's Professional Title: Ph.D. Candidate, Robotics Institute, Carnegie Mellon University
Talk Title: Causal Robot Learning for Manipulation
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Tuesday, April 30, 2024, 10am

From smartphones to microscopes and telescopes, optical lenses can be found in almost every part of modern society. Lens designs themselves span a large gamut of use cases, imaging microscopic phenomena, accommodating wide angle views, and viewing objects astronomically far away from us to name a few. Behind each of these lenses is an expert lens designer who has extensively used design tools that simulate the optics and optimize the designs using brute force methods like finite differencing or random sampling. Designers often need to intervene during the optimization process as these programs fail to produce better designs.

We present differentiable rendering techniques that efficiently calculates gradients of design objectives with respect to lens design parameters. Specifically, our contribution is twofold. First is a memory efficient method for calculating gradients of optical elements made of heterogeneous materials (so called, GRIN optics). Second is a method for calculating the gradient of light throughput with respect to lens parameters, a quantity that previous methods were not able to optimize directly. In this proposal, we seek to develop methods for searching for high performing designs over non-differentiable parameters in the lens design space, such as number of elements in the device.

Thesis Committee:
Ioannis Gkioulekas (Co-chair)
Matthew O'Toole (Co-chair)
Jim McCann
Bernd Bickel (ETH - Eidgenössische Technische Hochschule Zürich)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates Hillman 4405 and Zoom
Speaker's Name: ARJUN TEH
Speaker Website: www.cs.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Computational Lens Design
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, May 6, 2024, 10:30am

AI models have the potential to support and complement human decision-makers and users. And yet, the deployment of human-AI teams still faces practical challenges. The goal of my thesis is to (i) better understand how existing AI support fails to account for downstream user interactions, (ii) develop informative proxies to enable faster prototyping of AI support, and (iii) use insights from human studies to improve the design of human-AI teams. 

In the first part of my proposal, I will focus on predictive decision-making, where explanations have been hypothesized to help decision-makers make sense of AI predictions. I will overview user studies where we observe that this is not the case and present follow-up work to understand the failure modes of existing explanation methods. In the second part of my proposal, we explore one path to addressing the challenges of evaluating human-AI team designs in practice. I will present a use-case-grounded approach for predictive decision-making contexts called simulated evaluations. The final part of my proposal will cover our ongoing and proposed work to improve human-AI teaming in more complex, interactive settings, focusing on code co-pilots. I will present a recent evaluation and data collection effort to understand the effect of a model’s benchmark performance on downstream user helpfulness and proposed work on how insights and interaction data can be incorporated to build better AI pair programmers.

Thesis Committee:
Ameet Talwalkar (Chair)
Hoda Heidari
David Sontag (Massachusetts Institute of Technology)
Hal Daumé III (University of Maryland / Microsoft Research)
Dan Weld (University of Washington / AI2)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Gates Hillman 8102 and Zoom
Speaker's Name: VALERIE CHEN
Speaker Website: valeriechen.github.io
Speaker's Professional Title: Ph.D. Student, Machine Learning Department, Carnegie Mellon University
Talk Title: Towards a science of human-AI teams
For More Information: stidle@andrew.cmu.edu
Affiliations: Machine Learning Department (MLD)
Organization(s): School of Computer Science

View Details

Wednesday, May 1, 2024, 11am

Fine-tuned language models catastrophically forget tasks outside the fine-tuning distribution. On the flip side, fine-tuning is often used to remove unsafe behavior such as toxic content generation. Both this failure mode and success require that fine-tuning removes a capability from the model. We show that fine-tuning does not remove such capabilities, which is encouraging for reducing forgetting, and pessimistic for defending jailbreaks.

Via synthetic experiments, we hypothesize that language models implicitly infer the task of the prompt and that fine-tuning skews this inference towards tasks in the fine-tuning distribution. To test this, we propose Conjugate Prompting, which artificially makes the task look farther from the fine-tuning distribution while requiring the same capability, and we find that this recovers in-context learning abilities lost via instruction tuning and natural reasoning capability lost during code fine-tuning. More concerningly, conjugate prompting can recover harmful content generation suppressed by safety fine-tuning in chatbots like ChatGPT. Can algorithms like fine-tuning and input defenses reliably remove unwanted behavior? We find that the best fine-tuning and input defenses can not enforce one of the simplest, perfectly defined behaviors: do not output the word "purple".

Both forgetting and jailbreaking demonstrate that fine-tuning currently does not fully remove/change model capabilities. We propose future directions on improving capabilities by investigating length generalization and reliably removing capabilities via machine unlearning.

[1]  Understanding Catastrophic Forgetting in Language Models via Implicit Inference
[2]  Jailbreaking is Best Solved by Definition

Thesis Committee
Aditi Raghunathan (Chair)
Daphne Ippolito

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Wean Hall 5328
Speaker's Name: SUHAS KOTHA
Speaker Website: www.andrew.cmu.edu…
Speaker's Professional Title: Masters Student, Computer Science Department, Carnegie Mellon University
Talk Title: Fine-tuning Does Not Remove Language Model Capabilities
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Tuesday, April 30, 2024, 10:30am

Most large-scale storage systems employ erasure coding to provide resilience against disk failures. Recent work has shown that tuning this redundancy to changes in disk failure rates leads to substantial storage savings. This process requires code conversion, wherein data encoded using an [n^I,k^I] initial code has to be transformed into data encoded using an [n^F,k^F] final code.  Convertible codes are a class of codes that enable efficient code conversion while maintaining other desirable properties. In this thesis, we focus on the access cost of conversion (corresponding to the total number of symbols accessed in the conversion process) and on an important subclass of conversions known as the merge regime (corresponding to combining multiple initial codewords into a single final codeword).
   
In this setting, explicit constructions are known for systematic access-optimal Maximum Distance Separable (MDS) convertible codes for all parameters in the merge regime. However, the existing construction for a key subset of these parameters, which makes use of Vandermonde parity matrices, requires a very large field making it unsuitable for practical applications. In this thesis, we provide (1) sharper bounds on the minimum field size requirement for such codes, and (2) explicit constructions for low field sizes for several parameter ranges. In doing so, we provide a proof of super-regularity of specially designed classes of Vandermonde matrices that could be of independent interest.

Thesis Committee
Rashmi Vinayak (Chair)
Ryan O'Donnell

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Traffic21 Classroom, Gates Hillman 6501
Speaker's Name: SARANSH CHOPRA
Speaker Website: saranshchopra.github.io
Speaker's Professional Title: Masters Student, Computer Science Department, Carnegie Mellon University
Talk Title: Low Field Size Constructions of Access-Optimal Convertible Codes
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, May 6, 2024, 11am

Companies leverage personalization techniques to tailor user experiences. Personalization ap- pears in search engines and online stores, which include salutations and statistically learned correlations over search-, browsing- and purchase-histories. However, users have a wider variety of substantive, domain-specific preferences that influence their choices when they use directory services, and these have largely been overlooked or ignored. Specifically, users have preferences about what they are looking for, and are using services with varying levels of personalization to aid in discovering their things of interest. In the realm of requirements engineering, requirement analysts endeavor to gather, comprehend, and prioritize requirements, with an important fo- cus on stakeholder preferences and needs, employing diverse requirement elicitation techniques. Advances in Machine Learning (ML) and Natural Language Processing (NLP) have revolutionized the way people understand and interact with natural language, and opened up new opportunities to enhance and automate various facets of requirements engineering, including stakeholder preference elicitation.

The thesis aims to explore the potential of NLP techniques to enhance and automate stake- holder preference elicitation practices. Specifically, we demonstrate the following NLP-assisted preference elicitation methods: 1) we study the efficacy of extracting domain knowledge from user-authored scenarios using typed dependency parsing techniques, and from word embed- dings using a BERT-based Masked Language Model (MLM); 2) we research on how stakeholder preferences are expressed in text scenarios, whether elicited preferences represent missing re- quirements in existing systems, and how we may use named entity recognition techniques to build classifiers to label preference words in scenarios and link them to form preference phrases; 3) we build a tool to support and improve preference elicitation practices in interviews by using various NLP techniques, including MLM, transformers, speech-to-text transcription, and part- of-speech tagging. We also propose to build on and improve existing research findings with Large Language Models (LLM). The expected outcome of the thesis is to shed light on how NLP can be integrated into existing requirement elicitation practices to enhance and enrich them, while advancing our understanding about how stakeholder preference may be elicited more effectively and comprehensively.

Thesis Committee
Travis Breaux (Chair)
Christian Kästner
Bogdan Vasilescu
Fabiano Dalpiaz (Utrecht University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: TCS Hall 310 and Zoom
Speaker's Name: YUCHEN SHEN
Speaker Website: se-phd.s3d.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Ph.D. Program in Software Engineering, Software and Societal Systems Seminar, Carnegie Mellon University
Talk Title: NLP-Assisted Preference Elicitation in Requirements Engineering
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: cherold@cs.cmu.edu | aroudebu@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Tuesday, April 30, 2024, 9am

Before the ubiquity of social media, the information space was dominated by a small number of trusted originators (e.g., news organizations). This paradigm was shattered and replaced with a more diffused information environment where content originates from often unknown actors and is propagated nearly instantaneously. The need to rapidly identify factually inaccurate information (misinformation) in this environment is critical. Though some solutions exist to this problem, most current systems rely on human-driven fact-checking.

Existing fact-checking systems cannot operate at scale and are subject to significant human bias. In this thesis, I will describe a methodology for a fully automated misinformation detection pipeline that operates on multiple social media platforms. The pipeline leverages natural language processing (NLP) approaches to find, extract, and contextualize claims that could contain misinformation. Another set of NLP models and network science approaches assigns a likelihood to the truth value of these claims to arrive at a final misinformation likelihood score.

As part of this thesis, I will validate the misinformation detection pipeline in terms of its ability to accurately detect misinformation in labeled datasets as well as its research utility in the social cybersecurity domain by applying it in case studies on multiple social media platforms that focus on diverse topics/communities. The resulting system will serve as a valuable part of the social cybersecurity researchers' toolkit, to be used alongside the BEND framework to characterize the information environment and ultimately inform effective countermeasures.

Thesis Committee
Kathleen Carley (Chair)
Brandy Aven
Hong Shen
COL David Beskow (United States Military Academy)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: TCS Hall 460 and Zoom
Speaker's Name: IAN KLOO
Speaker Website: sc.s3d.cmu.edu…
Speaker's Professional Title: Ph.D. Student, Ph.D. Program Societal Computing, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Automated Misinformation Detection with Natural Language Processing and Network Models
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: cherold@cs.cmu.edu | aroudebu@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Friday, May 3, 2024, 11am

Software testing is an integral part of software development. However, testing faces challenges due to the time-consuming and challenging nature of writing high quality tests, leading to poorly maintained test suites and lower overall software quality. Prior work for automatically generating tests, like EvoSuite and Randoop can generate high-coverage tests, however often these tests are hard to read, unrealistic, or incorrect, necessitating additional effort from developers for verification. In contrast, language models have shown promise in generating human-like, high quality code functions, benefiting tools like Copilot in code generation.

However, language models are not as successful at generating tests, struggling with both hallucination and correctly invoking internal methods present in the code under test. This is because code generation language models only consider source code immediately before the generated code, and thus miss context in the file under test. To help overcome these limitations, I focus on how we can incorporate domain specific properties of testing such as the strong coupling between source and test files along with important test execution data to improve the application of language models to software testing. I also examine how we can better evaluate test generation approaches with metrics that are more meaningful to developers. My thesis statement is: We can exploit the structure of test code and close relationship between code and test files to enable the practical application of language models to software testing in both pretraining and fine-tuning. This approach can be used to a) generate useful unit test cases b) identify weaknesses in existing test suites and c) improve test suites to overcome found weaknesses.

My thesis will make the following contributions:

1. It presents a new method for pretraining models for test generation, that considers the relationship between source code and test code.
2. It provides an approach to automatically classify mutants as detected or undetected without executing the test suite by leveraging additional test context.
3. It demonstrates the effectiveness of adding execution context to test generation models, which enables us to generate mutant killing tests.
4. It evaluates all provided techniques with metrics and experiments that are practically meaningful developers, not considered in prior work.

Work I have already completed (ASE 2023) demonstrated that pretraining language models on dual objectives of code and test generation significantly improves unit test generation. I also leveraged the joint relationship between code and tests (FSE 2023) to improve predictive mutation testing techniques, modeling mutants at the token level, and incorporating both source and test methods during fine-tuning.

I propose to further apply these insights to a specialized case of mutation testing: generating tests that kill existing live mutants. I plan to include additional execution context into test generation models and use reinforcement learning. This will enable me to automatically generate test suites that are more comprehensive and similar to what an actual developer would write than current tools. I intend to complete this work by May 2025.

Thesis Committee:
Christian Kästner (Chair)
Daniel Fried
Claire Le Goues
Alex Groce (Northern Arizona University)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: TCS Hall 358 and Zoom
Speaker's Name: KUSH DAVE JAIN
Speaker Website: www.kushjain.com
Speaker's Professional Title: Ph.D. Student, Ph.D. Program in Softare Engineering, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Exploiting Test Structure to Enhance Language Models for Software Testing
For More Information: cherold@cs.cmu.edu | aroudebu@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Tuesday, April 30, 2024, 1:30pm

In most of the world's democracies, policy decisions are primarily made by elected political officials. However, under mounting dissatisfaction with representative government due to issues ranging from social inequality to public distrust, a new proposal is taking off: to augment representative democracy with mechanisms by which the public can directly participate in policymaking.

The guiding application of this thesis will be one particular model of participation, deliberative minipublics (DMs), though we will argue that our contributions may apply to many models of direct participation. In a DM, a panel of citizens is selected by lottery from the population; then, this panel convenes around a particular policy issue to study background information, deliberate amongst themselves, and then weigh in on the issue. DMs have been gaining momentum over the past decade, and they are now being used at national and supranational levels, and integrated into representative governments.

Motivated by this application domain, we make the following main contributions: In Part I, we design algorithms for performing the random selection of DM participants, a process known as sortition. Our sortition algorithms permit users to make optimal trade-offs between descriptive representation and other desirable properties conferred by randomness, and we characterize these tradeoffs using game theory, optimization, and empirics. In Part II, we use a novel social choice theory framework to investigate a notion of representation that departs from descriptive representation in a key way: it accounts for the political reality that people may be affected to widely varying degrees by any given policy decision. In Part III, we study a key potential impact of the background information/deliberation phase of a DM: increases in the extent to which participants consider how others in their society may be affected by different policy options. In Part IV, we discuss why our contributions can be useful regardless of how DMs ultimately fare in the political sphere, and we highlight how the enclosed research illustrates new ways to combine tools from political science and computer science.

Thesis Committee
Ariel Procaccia (Chair) (Carnegie Mellon University / Harvard University)
Nihar Shah 
Anupam Gupta (New York University)
Nika Haghtalab (University of California, Berkeley)
Ashish Goel (Stanford University)

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Orals
Room Number: In Person and Virtual - ET
Building: Reddy Conference Room, Gates Hillman 4405 and ZOom
Speaker's Name: BAILEY FLANIGAN
Speaker Website: sites.google.com…
Speaker's Professional Title: Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
Talk Title: Strengthening our Participatory Democracy Toolkit using Algorithms, Social Choice, and Social Science
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Tuesday, April 30, 2024, 2pm

Multiple-choice questions (MCQs) are the predominant form of assessment in educational environments, known for their efficiency and scalability. Traditionally, these questions are crafted by instructors, a method that despite its expertise often results in inconsistencies and errors. In response to these limitations and the need for scalability, learnersourcing has been leveraged, which involves students in the question creation process. Although this method capitalizes on the diverse perspectives of students, it also leads to significant variability in the quality of the questions produced. Additionally, while recent advances in artificial intelligence have facilitated more scalable and automated methods for generating MCQs, these AI-driven methods still suffer from many of the same shortcomings as those created by humans. Current evaluation methods for MCQs predominantly rely on human judgment, which introduces subjectivity and lacks scalability. While automated evaluation methods provide scalability, they fall short in adequately assessing the educational value of questions, focusing instead on surface-level features that do not match expert evaluation.

To improve the evaluation of MCQs across all creation methods, I propose the Scalable Automated Question Usability Evaluation Toolkit (SAQUET). This toolkit provides a domain-agnostic approach to evaluate the quality of educational MCQs with a focus on their pedagogical implications. Utilizing natural language processing techniques, SAQUET applies 19 criteria from the Item-Writing Flaws (IWF) rubric, classifies questions according to Bloom's Revised Taxonomy levels, and suggests a set of hypothesized skills that the question might require. This multifaceted approach allows for a more nuanced evaluation of MCQs, providing immediate and actionable feedback to instructors, students, and other educational stakeholders who engage in question creation.

In my research thus far, I have demonstrated that students are capable of generating high-quality assessments when given minimal scaffolding and support from technology. I have investigated the potential of involving students and crowdworkers in generating and evaluating the skills required to solve problems. I have shown how crowdworkers can leverage the IWF rubric to evaluate questions akin to human experts. Finally, through a preliminary study I  have indicated that the automated application of the IWF rubric to evaluate question quality yields results similar to those of human evaluations. In my proposed work, I plan to extend the capabilities of SAQUET, testing its effectiveness across various educational domains, integrating skill tagging, and refining Bloom’s Revised Taxonomy classifications within the toolkit. I will conduct case studies with educational practitioners to delve deeper into the challenges and opportunities in the MCQ evaluation process, aiming to enhance the reliability and pedagogical effectiveness of these assessments.

Thesis Committee
John Stamper (Chair)
Ken Koedinger
Sherry Tongshuang Wu
Christopher Brooks (University of Michigan)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Newell-Simon 4305 and Zoom
Speaker's Name: STEVEN MOORE
Speaker Website: stevenjamesmoore.com
Speaker's Professional Title: Ph.D. Student, Human-Computer Interaction Institute, School of Computer Science
Talk Title: Creating and Evaluating Pedagogically Valid Assessments at Scale
For More Information: lbuffing@andrew.cmu.edu
Affiliations: Human-Computer Interaction Institute (HCII)
Organization(s): School of Computer Science
Event Website Title: Event Website
Event Website URL: www.hcii.cmu.edu…

View Details

Sunday, April 28, 2024, 6pm

The program includes compositions by:

• Martin Baker
• Blaine Black
• Kate Hyun
• Liam Neely
• Jasmine Papineau Wood
• Cody Rong
• Cole Savo
• Harry Schneider
• Katherine Wang
• Allen Zhu

View Details

Monday, April 29, 2024, 1pm

Large GPU clusters are increasingly becoming more heterogeneous due to advances in GPU design and incremental deployment of a mix of GPU types over time. Deep learning (DL) training jobs running on these GPU clusters can see varying job completion times depending on the resources allocated by the cluster scheduler and job hyper-parameters configured by users at the time of job submission. Sia is a cluster scheduler that (1) efficiently assigns heterogeneous GPU resources to elastic resource-adaptive DL training jobs, and (2) configures the job hyper-parameters to maintain high training efficiency for all running jobs without sacrificing the quality of trained models.

We will discuss challenges in optimizing resource-adaptivity for deep learning training (DLT) jobs on large clusters with many GPU types, and introduce a new scheduling formulation that efficiently matches DLT jobs and their configurations to GPU types and counts, while adapting to changes in cluster load and job mix over time. On job traces derived from real datacenters, Sia improves job completion times by 30-93% while using 12-60% fewer GPU hours. Furthermore, its scheduling policy is quick to evaluate and easily scales to GPU clusters with many GPU types and 1000s of GPUs.

Presented in Partial Fulfillment of the CSD Speaking Skills Requirement

Event Type: Speaking Skills
Room Number: In Person
Building: McWilliams Classroom, Gates Hillman 4303
Speaker's Name: SUHAS JAYARAM SUBRAMANYA
Speaker Website: suhasjs.github.io
Speaker's Professional Title: Ph.D. Student, Computer Science Department, Carnegie Mellon University
Talk Title: Sia: Heterogeneity-aware, goodput-optimized ML-cluster scheduling
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: matthewstewart@cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Wednesday, May 1, 2024, 12 – 1:15pm

Conspiracy theories are a paradigmatic example of beliefs that, once adopted, are extremely difficult to dispel. Influential psychological theories propose that conspiracy beliefs are uniquely resistant to counterevidence because they satisfy important needs and motivations. Here, we raise the possibility that previous attempts to correct conspiracy beliefs have been unsuccessful merely because they failed to deliver counterevidence that was sufficiently compelling and tailored to each believer’s specific conspiracy theory (which vary dramatically from believer to believer). To evaluate this possibility, we leverage recent developments in generative artificial intelligence (AI) to deliver well-argued, person-specific debunks to a total of N = 2,190 conspiracy theory believers. Participants in our experiments provided detailed, open-ended explanations of a conspiracy theory they believed, and then engaged in a 3 round dialogue with a frontier generative AI model (GPT-4 Turbo) which was instructed to reduce each participant’s belief in their conspiracy theory (or discuss a banal topic in a control condition). Across two experiments, we find robust evidence that the debunking conversation with the AI reduced belief in conspiracy theories by roughly 20%. This effect did not decay over 2 months time, was consistently observed across a wide range of different conspiracy theories, and occurred even for participants whose conspiracy beliefs were deeply entrenched and of great importance to their identities. Furthermore, although the dialogues were focused on a single conspiracy theory, the intervention spilled over to reduce beliefs in unrelated conspiracies, indicating a general decrease in conspiratorial worldview, as well as increasing intentions to challenge others who espouse their chosen conspiracy. These findings highlight that even many people who strongly believe in seemingly fact-resistant conspiratorial beliefs can change their minds in the face of sufficient evidence.

—

David Rand is the Erwin H. Schell Professor and Professor of Management Science and Brain and Cognitive Sciences at MIT. Bridging the fields of cognitive science, behavioral economics, and social psychology, David’s research combines behavioral experiments and online/field studies with mathematical/computational models to understand human decision-making. His work focuses on illuminating why people believe and share misinformation and “fake news”; understanding political psychology and polarization; and promoting human cooperation. He has published over 200 articles in peer-reviewed journals such Nature, Science, PNAS, the American Economic Review, Psychological Science, Management Science, New England Journal of Medicine, and the American Journal of Political Science, and his work has received widespread media attention.

David regularly advises technology companies such as Google, Meta/Facebook, and TikTok in their efforts to combat misinformation, and has provided testimony about misinformation to the US and UK governments. He has also written for popular press outlets including the New York Times, Wired, and New Scientist. He was named to Wired magazine’s Smart List 2012 of “50 people who will change the world,” chosen as a 2012 Pop!Tech Science Fellow, awarded the 2015 Arthur Greer Memorial Prize for Outstanding Scholarly Research, chosen as fact-checking researcher of the year in 2017 by the Poyner Institute’s International Fact-Checking Network, awarded the 2020 FABBS Early Career Impact Award from the Society for Judgment and Decision Making, and selected as a 2021 Best 40-Under-40 Business School Professor by Poets & Quants. Papers he has coauthored have been awarded Best Paper of the Year in Experimental Economics, Social Cognition, and Political Methodology.

Additional Information:

• Durably reducing conspiracy beliefs through dialogues with AI
• Twitter | BlueSky

Event Type: Seminars
Room Number: In Person and Virtual - ET
Building: TCS Hall 358
Speaker's Name: DAVID RAND
Speaker Website: davidrand-cooperation.com…
Speaker's Professional Title: Erwin H. Schell Professor of Management Science and Brain and Cognitive Sciences, Institute for Data, Systems, and Society, and Director, Human Cooperation Laboratory, and Applied Cooperation Initiative, Massachusetts Institute of Technology
Talk Title: Durably reducing conspiracy beliefs through dialogues with AI
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: lv2c@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Wednesday, May 1, 2024, 2pm

The widespread adoption of time series machine learning (ML) models faces multiple challenges involving data, modeling and evaluation.Data. Modern ML models depend on copious amounts of cohesive and reliably annotated data for training and evaluation. However, labeled data is not always available and reliable, and can also be dispersed across different locations. We propose systematic solutions to making time series data ML-ready.

Modeling. Most current time series ML models are built, trained and evaluated on individual datasets from a specific application domain. Thus, to build an effective model for a particular application scenario, substantial effort, time, and domain expertise are required to develop a successful task-specific design. We propose to partially address this limitation by developing large pre-trained foundation models for time series, to ease development of useful models across diverse application domains with limited resources, data and labels.

Evaluation. Currently, time series models are commonly evaluated using relatively small, specific and highly tailored benchmarks, which may obfuscate assessment of their performance. We highlight the gaps in evaluation techniques and propose addressing the most important of them through comprehensive, multi-metric assessment.

In summary, this thesis aims to democratize time series artificial intelligence by simplifying and accelerating development of models, while improving their performance in real-world application scenarios facing resource constraints and imperfect data.

Thesis Committee:
Artur Dubrawski (Chair)
Jean Oh
Barnabás Póczos
Frederic Sala (University of Wisconsin-Madison)
Laurent Callot (Amazon)

Additional Information

In Person and Zoom Participation. See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Mauldin Auditorium, Newell-Simon 1305 and Zoom
Speaker's Name: MONONITO GOSWAMI
Speaker Website: mononitogoswami.github.io
Speaker's Professional Title: Ph.D. Student, Robotics Institute, Carnegie Mellon University
Talk Title: Towards Pragmatic Time Series Intelligence
For More Information: lyonsmuth@cmu.edu
Affiliations: Robotics Institute (RI)
Organization(s): SCS
Event Website Title: Event Website
Event Website URL: www.ri.cmu.edu…

View Details

Monday, April 29, 2024, 5pm

As traffic demand continues to increase globally, improving the efficiency and safety of the interconnected network of transportation systems around the world has become an increasingly critical priority. Various AI technologies have been designed to this end, and many have achieved good performance in simulation-based evaluations. However, a disconnect between theory and practice exists: few state-of-the-art AI technologies have been deployed to actually help resolve these challenges in the real world. One important cause of this disconnect is that these AI technologies have made unrealistic simplifying assumptions, which have made them unable to address the pain points of human stakeholders. In this thesis, I propose to answer research questions related to how AI technologies can be better designed for deployment by addressing four common challenges: uncertainty in underlying and observed levels of demand; coordination between individuals and systems; interpretability and controllability for complicated decision-making algorithms; and heterogeneity among end-users and deployment contexts. My completed, in-progress, and proposed work tackles these challenges through the lens of two key problem domains, traffic signal control and gig driving. Ultimately, the goal of this thesis is to design AI systems which are capable of being physically deployed and creating tangible impacts in these domains.

Thesis Commitee
Fei Fang (Co-chair)
Norman Sadeh (Co-chair)
Sean Qian
Matteo Pozzi
Peter Stone (The University of Texas at Austin)

Additional Information

In Person and Zoom Participation.  See announcement.

Event Type: Thesis Proposals
Room Number: In Person and Virtual - ET
Building: Traffic21 Classroom, Gates Hillman 6501 and Zoom
Speaker's Name: REX CHEN
Speaker Website: lythronaxargestes.github.io
Speaker's Professional Title: Ph.D. Student, Ph.D. Program in Societal Computing, Software and Societal Systems Department, Carnegie Mellon University
Talk Title: Rethinking the Design of Coordinated, Human-Centric AI Systems for Deployment in Transportation
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: aroudebu@andrew.cmu.edu
Affiliations: Software and Societal Systems Department (S3D)
Organization(s): School of Computer Science

View Details

Monday, April 29, 2024, 11:30am

Reconstructing 3D scenes from multi-view images has always been a challenging problem in computer vision and computer graphics. Traditional methods like structure from motion and multi-view stereo have been widely used for pose estimation and dense point cloud reconstruction. However, these methods have limited ability to reconstruct complex scenes with fine details. Recently, since the introduction of neural radiance fields (NeRF), volumetric neural rendering has shown great promise in reconstructing complex scenes with high fidelity. To accurately reconstruct scene geometry, other works have also proposed ways to directly model the signed-distance function or occupancy of a scene. However, these methods are often slow to train and cannot effectively leverage known scene information.

In this thesis, we propose a novel point-based representation that combines the efficiency of point clouds with the expressiveness of neural rendering. Point clouds are particularly appealing as a scene representation for rendering tasks, as they are the natural output of many 3D sensing modalities, including structure from motion, multi-view stereo, and lidar. They also come with a rich library of geometric queries. In our work, we utilize point clouds to efficiently reconstruct 3D scenes by using the generalized winding number as a proxy for the scene occupancy and by interpolating per-point neural features with appropriate kernels. We leverage the Barnes-Hut approximation and fast dipole sums to perform fast winding number queries and feature interpolation, as well as logarithmic complexity backpropagation for efficient differentiable rendering. We empirically show that our method consistently outperforms existing methods in both reconstruction quality and efficiency on a wide range of real-world scenes.

Thesis Committee:
Ioannis Gkioulekas (Chair)
Matthew O'Toole

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Gates Hillman 7501
Speaker's Name: HANYU CHEN
Speaker Website: hanyuc.com
Speaker's Professional Title: Master's Student, Computer Science Department, Carnegie Mellon University
Talk Title: 3D Reconstruction with Fast Dipole Sums
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science

View Details

Monday, April 29, 2024, 1pm

Large Language Models have been shown to perform well on natural language tasks, even those they were not explicitly trained to perform. Fine tuning these models on smaller datasets has become a popular technique to achieve high performance on specific tasks. However, fine tuning can lead to the memorization of training data, which may be a privacy concern. In this work, I investigated the extraction of training data from fine-tuned large language models. I conducted a series of experiments to determine how easily private training data can be extracted from fine-tuned models using different data extraction techniques. I also investigated how the amount of training data used for fine-tuning, the number of epochs, the length and content of each training sample, and the fine-tuning technique and parameters used affect the ease of data extraction. I found that data extraction is simple with direct access to the model if the model is not trained only on prompt completions. I also found that the proportion of data that can be extracted increased with the amount of data used for fine-tuning (for a constant number of epochs). This work has implications for the privacy of individuals whose data is used for fine-tuning, as well as for businesses or groups that use fine-tuned models in public facing software.

Thesis Committee:
Matt Fredrikson (Chair)
Yuvraj Agarwal

Additional Information

Event Type: Master's Thesis Presentation
Room Number: In Person
Building: Gates HIllman 9115
Speaker's Name: MIHIR DHAMANKAR
Speaker Website: mdkar.github.io
Speaker's Professional Title: Master's Student, Computer Science Department, Carnegie Mellon University
Talk Title: Extraction of Training Data from Fine-Tuned Large Language Models
Event Poster Title: Poster
Event Poster URL: www.cs.cmu.edu…
For More Information: tracyf@cs.cmu.edu
Affiliations: Computer Science Department (CSD)
Organization(s): School of Computer Science