LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning
We develop a safe planning method for trajectory generation by sampling from diffusion model under different LTLf constraints.
Robot & Machine Learning
Collaborative Learning and Adaptive Robots (CLeAR) Lab.
Below, you’ll find our contributions to robot/machine learning. Much of our work is on learning from (or interacting with) humans.
Octopi: Object Property Reasoning with Large Tactile-Language Models
We introduce PhysiCLeaR, an annotated dataset of everyday objects and tactile readings collected from a Gelsight Mini sensor, as well as Octopi, a system that leverages both tactile representation learning and large vision-language models to perform physical reasoning and inference, given tactile videos of multiple objects.
Don’t Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
We develop interpolant policies that leverage informative source distributions for imitation learning.
GRaCE: Balancing Multiple Criteria to Achieve Stable, Collision-Free, and Functional Grasps
We propose an optimization-based grasp synthesis framework, GRaCE, to generate context-specific grasps in complex scenarios. We test GRaCE in a simulator and a real-world grasping tasks.
Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigation
We introduce a novel framework centered around the Probable Object Location (POLo) score, which allows the agent to make data-driven decisions for efficient object search.
Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
We apply techniques from continual learning to the problem of selective forgetting in deep generative models. Our method, dubbed Selective Amnesia, allows users to remap undesired concepts to user-defined ones.
The Best of Both Worlds in Network Population Games: Reaching Consensus and Convergence to Equilibrium
We study a model of multi-population learning with heterogenous beliefs.
Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction
We construct a decomposed latent state space model for perspective-taking for human robot interaction.
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
We explore the potential of LLMs to act as zero-shot human models for HRI. We contribute an empirical study and case studies on a simulated table-clearing task and a new robot utensil-passing experiment.
Refining 6-DoF Grasps with Context-Specific Classifiers
We propose a sampling-based grasp synthesis framework, GraspFlow, to generate context-specific grasps. We test GraspFlow in a real-world table-top grasping task.
Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
We develop a family of stable continuous-time neural state space-models.
Generative Modeling with Flow-Guided Density Ratio Learning
We extend gradient flow methods to a variety of high-quality image synthesis tasks using a novel density ratio learning method.
Translating Natural Language to Planning Goals with Large-Language Models
We contribute an empirical study into the effectiveness of LLMs, specifically GPT-3.5 variants, for the task of natural language goal translation to PDDL.
Safety-Constrained Policy Transfer with Successor Features
Transfer source policies to a target reinforcement learning task with safety constraints using Successor Features.
Observed Adversaries in Deep Reinforcement Learning
We examine the problem of observed adversaries for deep policies, where observations of other agents can hamper robot performance.
Fairness meets CMDPs
This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on CMDPs.
The Dynamics of Q-learning in Population Games
We develop an accurate physics-inspired model for describing how a population of Q-learning agents adapt as they interact.
MIRROR for Assistive Human-Robot Communication
Inspired by Social Projection Theory, we use the robot's self model to efficiently model humans.
Deep Explicit Duration Switching Models for Time Series
We propose a deep switching state space model that can capture both state-dependent and time-dependent switching patterns in time series data.
Embedding Symbolic Temporal Knowledge into Deep Sequential Models
We embed symbolic knowledge expressed as linear temporal logic (LTL) and use these embeddings to guide the training of deep sequential models.
Learning Robust Latent Representation for Reinforcement Learning with Multi-Modal Observations
We construct a shared latent space from different sensory modalities via contrastive learning.
Enhancing Deep Learning with Symbolic Knowledge
Bridging the gap between symbolic and connectionist paradigms via Graph Neural Network embeddings
A characteristic function approach to deep implicit generative modeling
We present a new method for training GANs via characteristic functions
Juiced and Ready to Predict Private Information in Deep Cooperative Reinforcement Learning
Training robots that can interactively assist humans with private information
Embedding Symbolic Knowledge into Deep Networks
Leveraging prior symbolic knowledge to improve the performance of deep models.
Generation meets recommendation: proposing novel items for groups of users
We present an approach to generate new items for groups of users based on their interaction history.
Efficiently Exploring Reward Functions in Inverse RL
Using Bayesian Optimization to address the ill-posed nature of Inverse Reinforcement Learning