loading page

SimLiquid: A Simulation-Based Liquid Perception Pipeline for Robot Liquid Manipulation
  • +2
  • Yan Huang,
  • Jiawei Zhang,
  • Ran Yu,
  • Shoujie Li,
  • Wenbo Ding
Yan Huang
Tsinghua Shenzhen International Graduate School
Author Profile
Jiawei Zhang
Tsinghua Shenzhen International Graduate School
Author Profile
Ran Yu
Tsinghua Shenzhen International Graduate School
Author Profile
Shoujie Li
Tsinghua Shenzhen International Graduate School
Author Profile
Wenbo Ding
Tsinghua Shenzhen International Graduate School

Corresponding Author:ding.wenbo@sz.tsinghua.edu.cn

Author Profile

Abstract

Transparent liquid volume estimation is crucial for robot manipulation tasks, such as pouring. However, estimating the volume of transparent liquids is a challenging problem. Most existing methods primarily focus on data collection in the real world, and the sensors are fixed to the robot body for liquid volume estimation. These approaches limit both the timeliness of the research process and the flexibility of perception. In this paper, we present SimLiquid20k, a high-fidelity synthetic dataset for liquid volume estimation, and propose a YOLO-based multi-modal network trained on fully synthetic data for estimating the volume of transparent liquids. Extensive experiments demonstrate that our method can effectively transfer from simulation to the real world. In scenarios involving changes in background, viewpoint, and container variations, our approach achieves an average error of 5% in real-world volume estimation. In addition, our work conducts two application experiments integrate with ChatGPT, showcasing the potential of our method in service robotics. The accompanying video and supplementary materials are available at https://simliquid.github.io/.
01 Dec 2024Submitted to Journal of Field Robotics
03 Dec 2024Submission Checks Completed
03 Dec 2024Assigned to Editor
03 Dec 2024Review(s) Completed, Editorial Evaluation Pending
19 Dec 2024Reviewer(s) Assigned