CAR Guidance: Staying On-Manifold under Compositional Rewards
We introduce a plug-and-play module that corrects off-manifold drift when guiding flow models with multiple rewards at inference time.
Generative AI
Collaborative Learning and Adaptive Robots (CLeAR) Lab.
Below, you’ll find our contributions to generative modeling/AI and their applications (typically in robotics). At CLeAR, we have developed new deep generative models that are based on gradient flows and structured temporal models that can capture complex relationships, yet are interpretable and useful for planning and decision-making.
DISCO: Language-Guided Manipulation with Diffusion Policies and Constrained Inpainting
DISCO uses vision-language models to guide diffusion policies with optimized keyframe inpainting, enabling superior zero-shot open-vocabulary robotic manipulation.
Coupled Hierarchical Diffusion for Long-Horizon Tasks
We propose CHD, a unified diffusion-based framework that tightly couples sub-goal and trajectory generation to improve long-horizon task planning.
Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks
DOPPLER is a new framework that combines diffusion models and hierarchical reinforcement learning to let robots plan and replan complex, long-horizon tasks from offline data with robustness in the real world.
Out-of-Distribution Detection with a Single Unconditional Diffusion Model
We show that a single unconditional diffusion model performs competitively in out-of-distribution detection tasks by measuring the rate-of-change and curvature of diffusion paths connecting data samples to the standard normal distribution.
LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning
We develop a safe planning method for trajectory generation by sampling from diffusion model under different LTLf constraints.
Generative Modeling with Flow-Guided Density Ratio Learning
We extend gradient flow methods to a variety of high-quality image synthesis tasks using a novel density ratio learning method.
Don’t Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
We develop interpolant policies that leverage informative source distributions for imitation learning.
Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
We apply techniques from continual learning to the problem of selective forgetting in deep generative models. Our method, dubbed Selective Amnesia, allows users to remap undesired concepts to user-defined ones.
Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction
We construct a decomposed latent state space model for perspective-taking for human robot interaction.
Deep Explicit Duration Switching Models for Time Series
We propose a deep switching state space model that can capture both state-dependent and time-dependent switching patterns in time series data.
Learning Robust Latent Representation for Reinforcement Learning with Multi-Modal Observations
We construct a shared latent space from different sensory modalities via contrastive learning.
A characteristic function approach to deep implicit generative modeling
We present a new method for training GANs via characteristic functions
Generation meets recommendation: proposing novel items for groups of users
We present an approach to generate new items for groups of users based on their interaction history.