The Dynamics of Q-learning in Population Games
We develop an accurate physics-inspired model for describing how a population of Q-learning agents adapt as they interact.
1 Posts
AAMAS
1 Posts
AI-HRI
Observed Adversaries in Deep Reinforcement Learning
We examine the problem of observed adversaries for deep policies, where observations of other agents can hamper robot performance.
1 Posts
AIES
Fairness meets CMDPs
This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on CMDPs.
1 Posts
CVPR
A characteristic function approach to deep implicit generative modeling
We present a new method for training GANs via characteristic functions
5 Posts
HRI
Getting to Know One Another Calibrating Intent, Capabilities and Trust for Human-Robot Collaboration
We address the problem of intention and capability calibration in human-robot collaboration with a decision-theoretic approach.
Modeling the Interplay of Trust and Attention in HRI an Autonomous Vehicle Study
We study and model how trust and attention affect how humans interact with autonomous vehicles.
Juiced and Ready to Predict Private Information in Deep Cooperative Reinforcement Learning
Training robots that can interactively assist humans with private information
Robot Capability and Intention in Trust-based Decisions across Tasks
We present results from a human-subject study designed to explore two facets of human mental models of robots - inferred capability and intention - and their relationship to overall trust and eventual decisions.
1 Posts
ICLR
1 Posts
ICML
Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
We develop a family of stable continuous-time neural state space-models.
6 Posts
ICRA
Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigation
We introduce a novel framework centered around the Probable Object Location (POLo) score, which allows the agent to make data-driven decisions for efficient object search.
Safety-Constrained Policy Transfer with Successor Features
Transfer source policies to a target reinforcement learning task with safety constraints using Successor Features.
Embedding Symbolic Temporal Knowledge into Deep Sequential Models
We embed symbolic knowledge expressed as linear temporal logic (LTL) and use these embeddings to guide the training of deep sequential models.
Learning Robust Latent Representation for Reinforcement Learning with Multi-Modal Observations
We construct a shared latent space from different sensory modalities via contrastive learning.
Enhancing Deep Learning with Symbolic Knowledge
Bridging the gap between symbolic and connectionist paradigms via Graph Neural Network embeddings
Towards Effective Tactile Identification of Textures using a Hybrid Touch Approach
We show that iCub robot classifies the surface textures with both sliding and touch movements under loose constraints with high accuracy.
1 Posts
IJRR
5 Posts
IROS
Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction
We construct a decomposed latent state space model for perspective-taking for human robot interaction.
Refining 6-DoF Grasps with Context-Specific Classifiers
We propose a sampling-based grasp synthesis framework, GraspFlow, to generate context-specific grasps. We test GraspFlow in a real-world table-top grasping task.
Extended Tactile Perception
We show that robots can extend their perception through grasped tools/objects via dynamic tactile sensing.
TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition
We propose a Spiking Graph Neural Network to take into account taxel geometry.
Fast texture classification using tactile neural coding and spiking neural network
We show that robots achieve fast classification of textures through Neural Encoding and Spiking Neural Network.
4 Posts
NeurIPS
Out-of-Distribution Detection with a Single Unconditional Diffusion Model
We show that a single unconditional diffusion model performs competitively in out-of-distribution detection tasks by measuring the rate-of-change and curvature of diffusion paths connecting data samples to the standard normal distribution.
Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
We apply techniques from continual learning to the problem of selective forgetting in deep generative models. Our method, dubbed Selective Amnesia, allows users to remap undesired concepts to user-defined ones.
Deep Explicit Duration Switching Models for Time Series
We propose a deep switching state space model that can capture both state-dependent and time-dependent switching patterns in time series data.
Embedding Symbolic Knowledge into Deep Networks
Leveraging prior symbolic knowledge to improve the performance of deep models.
1 Posts
RA-L
LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning
We develop a safe planning method for trajectory generation by sampling from diffusion model under different LTLf constraints.
6 Posts
RSS
Octopi: Object Property Reasoning with Large Tactile-Language Models
We introduce PhysiCLeaR, an annotated dataset of everyday objects and tactile readings collected from a Gelsight Mini sensor, as well as Octopi, a system that leverages both tactile representation learning and large vision-language models to perform physical reasoning and inference, given tactile videos of multiple objects.
Don’t Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
We develop interpolant policies that leverage informative source distributions for imitation learning.
Arena 3.0 - Advancing Social Navigation in Collaborative and Highly Dynamic Environments
We introduce the third iteration of the Arena platform - Arena 3.0, a platform to develop, train, and benchmark social navigation approaches!
GRaCE: Balancing Multiple Criteria to Achieve Stable, Collision-Free, and Functional Grasps
We propose an optimization-based grasp synthesis framework, GRaCE, to generate context-specific grasps in complex scenarios. We test GRaCE in a simulator and a real-world grasping tasks.
MIRROR for Assistive Human-Robot Communication
Inspired by Social Projection Theory, we use the robot's self model to efficiently model humans.
Event-Driven Visual-Tactile Sensing and Learning
Accurate, Fast, and Low-powered Multi-Sensory Perception via Neuromorphic Sensing and Learning
1 Posts
Social
Efficiently Exploring Reward Functions in Inverse RL
Using Bayesian Optimization to address the ill-posed nature of Inverse Reinforcement Learning
1 Posts
T-AFFC
Applying Probabilistic Programming to Affective Computing
We propose a probabilistic programming approach to affective computing.
6 Posts
award
Kaiqi wins the 2022 Research Achievement Award.
Kaiqi Chen is awarded the Research Achievement Award for his RSS 2022 paper on Differentiable Social Projection for Human Robot Communication.
Harold wins NUS ATEA 2022.
Harold has been awarded the University Annual Teaching Excellence Award (2022). Harold also won the Faculty Teaching Excellence Award and is now on Faculty H...
Sreejith wins the 2020 Research Achievement Award.
Sreejith Balakrishnan is awarded the Research Achievement Award for his NeurIPS 2020 paper on efficiently exploring reward functions in Inverse Reinforcement...
Tasbolat and team win IROS 2021 Best Paper Award.
We have won the Best Paper Award at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021 in October. IROS is a premier flagshi...
Yaqi wins the 2020 Research Achievement Award.
Yaqi Xie is awarded the Research Achievement Award for his NeurIPS 2019 paper on Embedding symbolic knowledge into deep networks.
Sreejith makes it to the 2019 Honor List of Student Tutors
Sreejith Balakrishnan is awarded the 2019 Honor List of Student Tutors for his contribution as a tutor to the module “CS4246: AI Planning and Decision Making”.
1 Posts
emotion
Applying Probabilistic Programming to Affective Computing
We propose a probabilistic programming approach to affective computing.
1 Posts
fairness
Fairness meets CMDPs
This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on CMDPs.
11 Posts
generative
Out-of-Distribution Detection with a Single Unconditional Diffusion Model
We show that a single unconditional diffusion model performs competitively in out-of-distribution detection tasks by measuring the rate-of-change and curvature of diffusion paths connecting data samples to the standard normal distribution.
LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning
We develop a safe planning method for trajectory generation by sampling from diffusion model under different LTLf constraints.
Generative Modeling with Flow-Guided Density Ratio Learning
We extend gradient flow methods to a variety of high-quality image synthesis tasks using a novel density ratio learning method.
Don’t Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
We develop interpolant policies that leverage informative source distributions for imitation learning.
Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
We apply techniques from continual learning to the problem of selective forgetting in deep generative models. Our method, dubbed Selective Amnesia, allows users to remap undesired concepts to user-defined ones.
Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction
We construct a decomposed latent state space model for perspective-taking for human robot interaction.
Deep Explicit Duration Switching Models for Time Series
We propose a deep switching state space model that can capture both state-dependent and time-dependent switching patterns in time series data.
Learning Robust Latent Representation for Reinforcement Learning with Multi-Modal Observations
We construct a shared latent space from different sensory modalities via contrastive learning.
A characteristic function approach to deep implicit generative modeling
We present a new method for training GANs via characteristic functions
Generation meets recommendation: proposing novel items for groups of users
We present an approach to generate new items for groups of users based on their interaction history.
1 Posts
icra
Harold delivers ICRA'24 Keynote
Harold gave a well-received Keynote at ICRA’24 on our group’s work on generative modeling and robotics! He spoke about two of our recent works on new kinds o...
2 Posts
knowledge
Embedding Symbolic Temporal Knowledge into Deep Sequential Models
We embed symbolic knowledge expressed as linear temporal logic (LTL) and use these embeddings to guide the training of deep sequential models.
Embedding Symbolic Knowledge into Deep Networks
Leveraging prior symbolic knowledge to improve the performance of deep models.
29 Posts
learn
Out-of-Distribution Detection with a Single Unconditional Diffusion Model
We show that a single unconditional diffusion model performs competitively in out-of-distribution detection tasks by measuring the rate-of-change and curvature of diffusion paths connecting data samples to the standard normal distribution.
LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning
We develop a safe planning method for trajectory generation by sampling from diffusion model under different LTLf constraints.
Generative Modeling with Flow-Guided Density Ratio Learning
We extend gradient flow methods to a variety of high-quality image synthesis tasks using a novel density ratio learning method.
Octopi: Object Property Reasoning with Large Tactile-Language Models
We introduce PhysiCLeaR, an annotated dataset of everyday objects and tactile readings collected from a Gelsight Mini sensor, as well as Octopi, a system that leverages both tactile representation learning and large vision-language models to perform physical reasoning and inference, given tactile videos of multiple objects.
Don’t Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
We develop interpolant policies that leverage informative source distributions for imitation learning.
GRaCE: Balancing Multiple Criteria to Achieve Stable, Collision-Free, and Functional Grasps
We propose an optimization-based grasp synthesis framework, GRaCE, to generate context-specific grasps in complex scenarios. We test GRaCE in a simulator and a real-world grasping tasks.
Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigation
We introduce a novel framework centered around the Probable Object Location (POLo) score, which allows the agent to make data-driven decisions for efficient object search.
Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
We apply techniques from continual learning to the problem of selective forgetting in deep generative models. Our method, dubbed Selective Amnesia, allows users to remap undesired concepts to user-defined ones.
The Best of Both Worlds in Network Population Games: Reaching Consensus and Convergence to Equilibrium
We study a model of multi-population learning with heterogenous beliefs.
Latent Emission-Augmented Perspective-Taking (LEAPT) for Human-Robot Interaction
We construct a decomposed latent state space model for perspective-taking for human robot interaction.
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
We explore the potential of LLMs to act as zero-shot human models for HRI. We contribute an empirical study and case studies on a simulated table-clearing task and a new robot utensil-passing experiment.
Refining 6-DoF Grasps with Context-Specific Classifiers
We propose a sampling-based grasp synthesis framework, GraspFlow, to generate context-specific grasps. We test GraspFlow in a real-world table-top grasping task.
Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
We develop a family of stable continuous-time neural state space-models.
Translating Natural Language to Planning Goals with Large-Language Models
We contribute an empirical study into the effectiveness of LLMs, specifically GPT-3.5 variants, for the task of natural language goal translation to PDDL.
Safety-Constrained Policy Transfer with Successor Features
Transfer source policies to a target reinforcement learning task with safety constraints using Successor Features.
Observed Adversaries in Deep Reinforcement Learning
We examine the problem of observed adversaries for deep policies, where observations of other agents can hamper robot performance.
Fairness meets CMDPs
This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on CMDPs.
The Dynamics of Q-learning in Population Games
We develop an accurate physics-inspired model for describing how a population of Q-learning agents adapt as they interact.
MIRROR for Assistive Human-Robot Communication
Inspired by Social Projection Theory, we use the robot's self model to efficiently model humans.
Deep Explicit Duration Switching Models for Time Series
We propose a deep switching state space model that can capture both state-dependent and time-dependent switching patterns in time series data.
Embedding Symbolic Temporal Knowledge into Deep Sequential Models
We embed symbolic knowledge expressed as linear temporal logic (LTL) and use these embeddings to guide the training of deep sequential models.
Learning Robust Latent Representation for Reinforcement Learning with Multi-Modal Observations
We construct a shared latent space from different sensory modalities via contrastive learning.
Enhancing Deep Learning with Symbolic Knowledge
Bridging the gap between symbolic and connectionist paradigms via Graph Neural Network embeddings
A characteristic function approach to deep implicit generative modeling
We present a new method for training GANs via characteristic functions
Juiced and Ready to Predict Private Information in Deep Cooperative Reinforcement Learning
Training robots that can interactively assist humans with private information
Embedding Symbolic Knowledge into Deep Networks
Leveraging prior symbolic knowledge to improve the performance of deep models.
Generation meets recommendation: proposing novel items for groups of users
We present an approach to generate new items for groups of users based on their interaction history.
Efficiently Exploring Reward Functions in Inverse RL
Using Bayesian Optimization to address the ill-posed nature of Inverse Reinforcement Learning
2 Posts
navigation
Arena 3.0 - Advancing Social Navigation in Collaborative and Highly Dynamic Environments
We introduce the third iteration of the Arena platform - Arena 3.0, a platform to develop, train, and benchmark social navigation approaches!
Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigation
We introduce a novel framework centered around the Probable Object Location (POLo) score, which allows the agent to make data-driven decisions for efficient object search.
1 Posts
neurips
Efficiently Exploring Reward Functions in Inverse RL
Using Bayesian Optimization to address the ill-posed nature of Inverse Reinforcement Learning
19 Posts
news
Harold delivers ICRA'24 Keynote
Harold gave a well-received Keynote at ICRA’24 on our group’s work on generative modeling and robotics! He spoke about two of our recent works on new kinds o...
4 Papers at R:SS'24.
All four of our submissions (3 papers, 1 demo) were accepted to R:SS 2024! A fantastic accomplishment by our CLeAR members Kaiqi, Samson, Tasbolat, Linh, Kel...
2 Papers at Neurips'23.
Both our submitted papers were accepted to Neurips this year! Come join us in New Orleans! Congrats to Alvin and Shuyue. Find out more about the papers below...
3 Papers at IROS'23.
CLeAR had three papers accepted to IROS this year! Come join us in Detroit! Congrats to Tasbolat, Bowen, and Kaiqi. Find out more about the papers below: ...
Introducing Dr. Sreejith Balakrishnan!
Congratulations Sreejith on successfully defending his thesis! Sreejith’s work was on value alignment in human-centric AI/robots. Check out his Neurips pap...
Early Career Spotlight at R:SS 2023!
Harold has been awarded an Early Career Spotlight at the Robotics: Science and Systems 2023. The Early Career Spotlight “* acknowledges the outstanding acc...
Introducing Dr. Xie Yaqi.
Xie Yaqi successfully defended her thesis and is now Dr. Xie. Congratulations Yaqi! You can find out more about Yaqi’s work on embedding symbolic knowledg...
IEEE T-AFFC Best Paper Award!
Our joint work with Desmond Ong, Jamil Zaki and Noah Goodman on Applying Probabilistic Programming to Affective Computing is one of 5 Best Papers (out of 82 ...
Kaiqi wins the 2022 Research Achievement Award.
Kaiqi Chen is awarded the Research Achievement Award for his RSS 2022 paper on Differentiable Social Projection for Human Robot Communication.
Harold wins NUS ATEA 2022.
Harold has been awarded the University Annual Teaching Excellence Award (2022). Harold also won the Faculty Teaching Excellence Award and is now on Faculty H...
CLeAR graduates its first PhD Student.
Abdul Fatir Ansari successfully defended his thesis and is now Dr. Ansari. Congratulations Fatir!
Sreejith wins the 2020 Research Achievement Award.
Sreejith Balakrishnan is awarded the Research Achievement Award for his NeurIPS 2020 paper on efficiently exploring reward functions in Inverse Reinforcement...
Tasbolat and team win IROS 2021 Best Paper Award.
We have won the Best Paper Award at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021 in October. IROS is a premier flagshi...
Harold wins Annual University and Faculty Teaching Awards!
Harold was awarded the university level and faculty teaching awards for the year 20/21!
Harold wins faculty teaching award for the 3rd time!
Harold was awarded the faculty teaching award for the year 19/20!
Yaqi wins the 2020 Research Achievement Award.
Yaqi Xie is awarded the Research Achievement Award for his NeurIPS 2019 paper on Embedding symbolic knowledge into deep networks.
Harold wins Faculty Teaching Award!
Harold was awarded the faculty teaching award for the year 18/19!
Sreejith makes it to the 2019 Honor List of Student Tutors
Sreejith Balakrishnan is awarded the 2019 Honor List of Student Tutors for his contribution as a tutor to the module “CS4246: AI Planning and Decision Making”.
Harold wins Faculty Teaching Award 17/18!
Harold was awarded the faculty teaching award for the year 17/18! A first time win.
7 Posts
physical
Don’t Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
We develop interpolant policies that leverage informative source distributions for imitation learning.
GRaCE: Balancing Multiple Criteria to Achieve Stable, Collision-Free, and Functional Grasps
We propose an optimization-based grasp synthesis framework, GRaCE, to generate context-specific grasps in complex scenarios. We test GRaCE in a simulator and a real-world grasping tasks.
Extended Tactile Perception
We show that robots can extend their perception through grasped tools/objects via dynamic tactile sensing.
Event-Driven Visual-Tactile Sensing and Learning
Accurate, Fast, and Low-powered Multi-Sensory Perception via Neuromorphic Sensing and Learning
TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition
We propose a Spiking Graph Neural Network to take into account taxel geometry.
Fast texture classification using tactile neural coding and spiking neural network
We show that robots achieve fast classification of textures through Neural Encoding and Spiking Neural Network.
Towards Effective Tactile Identification of Textures using a Hybrid Touch Approach
We show that iCub robot classifies the surface textures with both sliding and touch movements under loose constraints with high accuracy.
7 Posts
touch
Octopi: Object Property Reasoning with Large Tactile-Language Models
We introduce PhysiCLeaR, an annotated dataset of everyday objects and tactile readings collected from a Gelsight Mini sensor, as well as Octopi, a system that leverages both tactile representation learning and large vision-language models to perform physical reasoning and inference, given tactile videos of multiple objects.
Tasbolat and team win IROS 2021 Best Paper Award.
We have won the Best Paper Award at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021 in October. IROS is a premier flagshi...
Extended Tactile Perception
We show that robots can extend their perception through grasped tools/objects via dynamic tactile sensing.
Event-Driven Visual-Tactile Sensing and Learning
Accurate, Fast, and Low-powered Multi-Sensory Perception via Neuromorphic Sensing and Learning
TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition
We propose a Spiking Graph Neural Network to take into account taxel geometry.
Fast texture classification using tactile neural coding and spiking neural network
We show that robots achieve fast classification of textures through Neural Encoding and Spiking Neural Network.
Towards Effective Tactile Identification of Textures using a Hybrid Touch Approach
We show that iCub robot classifies the surface textures with both sliding and touch movements under loose constraints with high accuracy.
8 Posts
trust
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
We explore the potential of LLMs to act as zero-shot human models for HRI. We contribute an empirical study and case studies on a simulated table-clearing task and a new robot utensil-passing experiment.
MIRROR for Assistive Human-Robot Communication
Inspired by Social Projection Theory, we use the robot's self model to efficiently model humans.
Trust in Robots: Challenges and Opportunities
We examine how recent advances in psychometrics, trustworthy systems, deep learning etc. can help address challenges that arise with respect to trust in real-world human robot interactions.
Getting to Know One Another Calibrating Intent, Capabilities and Trust for Human-Robot Collaboration
We address the problem of intention and capability calibration in human-robot collaboration with a decision-theoretic approach.
Modeling the Interplay of Trust and Attention in HRI an Autonomous Vehicle Study
We study and model how trust and attention affect how humans interact with autonomous vehicles.
Robot Capability and Intention in Trust-based Decisions across Tasks
We present results from a human-subject study designed to explore two facets of human mental models of robots - inferred capability and intention - and their relationship to overall trust and eventual decisions.
social
Arena 3.0 - Advancing Social Navigation in Collaborative and Highly Dynamic Environments
We introduce the third iteration of the Arena platform - Arena 3.0, a platform to develop, train, and benchmark social navigation approaches!
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
We explore the potential of LLMs to act as zero-shot human models for HRI. We contribute an empirical study and case studies on a simulated table-clearing task and a new robot utensil-passing experiment.
Translating Natural Language to Planning Goals with Large-Language Models
We contribute an empirical study into the effectiveness of LLMs, specifically GPT-3.5 variants, for the task of natural language goal translation to PDDL.
Fairness meets CMDPs
This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on CMDPs.
The Dynamics of Q-learning in Population Games
We develop an accurate physics-inspired model for describing how a population of Q-learning agents adapt as they interact.
MIRROR for Assistive Human-Robot Communication
Inspired by Social Projection Theory, we use the robot's self model to efficiently model humans.
Getting to Know One Another Calibrating Intent, Capabilities and Trust for Human-Robot Collaboration
We address the problem of intention and capability calibration in human-robot collaboration with a decision-theoretic approach.
Modeling the Interplay of Trust and Attention in HRI an Autonomous Vehicle Study
We study and model how trust and attention affect how humans interact with autonomous vehicles.
Multi-Task Trust in Human-Robot Interaction
Human Trust in Robots across Task Contexts
Applying Probabilistic Programming to Affective Computing
We propose a probabilistic programming approach to affective computing.
Robot Capability and Intention in Trust-based Decisions across Tasks
We present results from a human-subject study designed to explore two facets of human mental models of robots - inferred capability and intention - and their relationship to overall trust and eventual decisions.
Planning with Trust for Human-Robot Collaboration
Robot decision-making with Human Trust.