Publications | Abhinav Kumar

2025

ICRA

Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulation

Abhinav Kumar*, Thomas Power*, Fan Yang, and 4 more authors

2025

Abs arXiv HTML PDF

Planning contact-rich interactions for multi-finger manipulation is challenging due to the high-dimensionality and hybrid nature of dynamics. Recent advances in data-driven methods have shown promise, but are sensitive to the quality of training data. Combining learning with classical methods like trajectory optimization and search adds additional structure to the problem and domain knowledge in the form of constraints, which can lead to outperforming the data on which models are trained. We present Diffusion-Informed Probabilistic Contact Search (DIPS), which uses an A* search to plan a sequence of contact modes informed by a diffusion model. We train the diffusion model on a dataset of demonstrations consisting of contact modes and trajectories generated by a trajectory optimizer given those modes. In addition, we use a particle filter-inspired method to reason about variability in diffusion sampling arising from model error, estimating likelihoods of trajectories using a learned discriminator. We show that our method outperforms ablations that do not reason about variability and can plan contact sequences that outperform those found in training data across multiple tasks. We evaluate on simulated tabletop card sliding and screwdriver turning tasks, as well as the screwdriver task in hardware to show that our combined learning and planning approach transfers to the real world.

2024

Constraining Gaussian Process Implicit Surfaces for Robot Manipulation via Dataset Refinement

Abhinav Kumar, Peter Mitrano, and Dmitry Berenson

IEEE Robotics and Automation Letters, 2024

Abs arXiv HTML PDF

Model-based control faces fundamental challenges in partially-observable environments due to unmodeled obstacles. We propose an online learning and optimization method to identify and avoid unobserved obstacles online. Our method, Constraint Obeying Gaussian Implicit Surfaces (COGIS), infers contact data using a combination of visual input and state tracking, informed by predictions from a nominal dynamics model. We then fit a Gaussian process implicit surface (GPIS) to these data and refine the dataset through a novel method of enforcing constraints on the estimated surface. This allows us to design a Model Predictive Control (MPC) method that leverages the obstacle estimate to complete multiple manipulation tasks. By modeling the environment instead of attempting to directly adapt the dynamics, our method succeeds at both lowdimensional peg-in-hole tasks and high-dimensional deformable object manipulation tasks. Our method succeeds in 10/10 trials vs 1/10 for a baseline on a real-world cable manipulation task under partial observability of the environment.

2023

IROS
Online Implicit Surfaces for Obstacle Modeling and Avoidance

Abhinav Kumar, and Dmitry Berenson

In IROS 2023 Workshop on Leveraging Models for Contact-Rich Manipulation, 2023

Abs Bib HTML PDF

Adapting model-based control to novel environ- ments is challenging as dynamics models learned offline may not generalize to the obstacle configuration of the novel environment. We propose a method to identify and avoid obstacles online whose geometry is not known a priori without updating the offline, nominal, dynamics. Our method relies on a Gaussian pro- cess implicit surface (GPIS) to construct data-efficient obstacle representations using visual and inferrred contact data derived from observed states and dynamics predictions. This allows us to design a model predictive controller (MPC) using the uncertainty estimates provided by the GPIS to successfully navigate around obstacles to complete multiple manipulation tasks. By modeling the environment instead of directly adapting the dynamics, our method is able to solve both low-dimensional peg-in-hole tasks and high-dimensional rope and cable manipulation tasks. This enables our method to succeed in 30/30 trials vs 15/30 for a baseline on a simulated rope manipulation task while requiring 63% fewer control steps to succeed.
@inproceedings{kumar2023online, title = {Online Implicit Surfaces for Obstacle Modeling and Avoidance}, author = {Kumar, Abhinav and Berenson, Dmitry}, booktitle = {IROS 2023 Workshop on Leveraging Models for Contact-Rich Manipulation}, year = {2023}, url = {https://openreview.net/forum?id=3mXn5Xjx9S}, }

2021

EMNLP
How much coffee was consumed during EMNLP 2019? fermi problems: A new reasoning challenge for AI

Ashwin Kalyan, Abhinav Kumar, Arjun Chandrasekaran, and 2 more authors

Empirical Methods in Natural Language Processing, 2021

Abs arXiv Bib PDF

Many real-world problems require the combined application of multiple reasoning abilities—employing suitable abstractions, commonsense knowledge, and creative synthesis of problem-solving strategies. To help advance AI systems towards such capabilities, we propose a new reasoning challenge, namely Fermi Problems (FPs), which are questions whose answers can only be approximately estimated because their precise computation is either impractical or impossible. For example, “How much would the sea level rise if all ice in the world melted?” FPs are commonly used in quizzes and interviews to bring out and evaluate the creative reasoning abilities of humans. To do the same for AI systems, we present two datasets: 1) A collection of 1k real-world FPs sourced from quizzes and olympiads; and 2) a bank of 10k synthetic FPs of intermediate complexity to serve as a sandbox for the harder real-world challenge. In addition to question-answer pairs, the datasets contain detailed solutions in the form of an executable program and supporting facts, helping in supervision and evaluation of intermediate steps. We demonstrate that even extensively fine-tuned large-scale language models perform poorly on these datasets, on average making estimates that are off by two orders of magnitude. Our contribution is thus the crystallization of several unsolved AI problems into a single, new challenge that we hope will spur further advances in building systems that can reason.
@article{kalyan2021much, title = {How much coffee was consumed during EMNLP 2019? fermi problems: A new reasoning challenge for AI}, author = {Kalyan, Ashwin and Kumar, Abhinav and Chandrasekaran, Arjun and Sabharwal, Ashish and Clark, Peter}, journal = {Empirical Methods in Natural Language Processing}, year = {2021}, }

2020

AGU

Extracting Disaster Variables with Deep Learning Methods to Improve Earth Science Data Set Retrieval

A. Kumar, A. Albayrak, W. L. Teng, and 1 more author

In AGU Fall Meeting Abstracts, Dec 2020

Bib

@inproceedings{2020AGUFMIN0140001K,
  author = {{Kumar}, A. and {Albayrak}, A. and {Teng}, W.~L. and {Pham}, L.},
  title = {{Extracting Disaster Variables with Deep Learning Methods to Improve Earth Science Data Set Retrieval}},
  keywords = {1920 Emerging informatics technologies, INFORMATICS, 1938 Knowledge representation and knowledge bases, INFORMATICS, 1954 Natural language processing, INFORMATICS, 1958 Ontologies, INFORMATICS},
  booktitle = {AGU Fall Meeting Abstracts},
  year = {2020},
  volume = {2020},
  month = dec,
  eid = {IN014-0001},
  pages = {IN014-0001},
  adsurl = {https://ui.adsabs.harvard.edu/abs/2020AGUFMIN0140001K},
  adsnote = {Provided by the SAO/NASA Astrophysics Data System},
}

AGU

Towards a Domain-Informed Search Engine for NASA Earth Science Data

William L Teng, Arif Albayrak, Laura Yu Zheng, and 5 more authors

In AGU Fall Meeting 2020, Dec 2020

Bib

@inproceedings{teng2020towards,
  title = {Towards a Domain-Informed Search Engine for NASA Earth Science Data},
  author = {Teng, William L and Albayrak, Arif and Zheng, Laura Yu and Kumar, Abhinav and Wu, Lauryn and Pham, Long and Khayat, Mohammad G and Hegde, Mahabal},
  booktitle = {AGU Fall Meeting 2020},
  year = {2020},
  organization = {AGU},
}