SafePicking

Overview Video (with audio)

Pipeline for object mapping and extraction

We present a pipeline that finds target objects and extract them from a cluttered environment such as an object pile. This pipeline consists of 1) object-level mapping with volumetric reconstruction and pose estimation of detected objects with an on-board RGB-D camera, and 2) learning-based motion planning with a model that recursively generates end-effector relative motions using estimated object poses and a raw depth observation in the form of a heightmap.

Network model for motion generation

We train a deep Q-network using a "safety metric reward" during object extraction. The network model receives fused observations of a depth-based heightmap and estimated object poses to achieve high levels of success with object pose and robustness to estimation errors with the raw depth observation. We feed end-effector relative transformations as evaluation actions, from which the best-scored action is selected as the next action.

Training in simulation

We train the model in physics simulation using the CAD models of known objects. The safety metric reward is computed as the sum of the translations of objects in a pile, which incorporates undesirable consequences such as dropping and sliding.

Baseline comparison in safe object extraction

Our proposed method, SafePicking, surpasses all of the baseline methods: Naive, joint interpolation to a reset pose; Heuristic, a straight motion to lift a grasped object up; RRT-Connect, motion planning using ground-truth object poses.

Model comparison using different observations as input

Our proposed model shows the best result when given both object poses and a raw depth observation as input. The pose information enables the model to generate better motions using more complete semantic information, and the raw depth information enables it to be robust to estimation errors (pose errors; mis-detection) as a less processed scene observation.

Real-world experiments

Full pipeline demonstration

Heuristic vs. Learned

Learned model ablation

Adaptation to pile changes

Paper

Bibtex

  @inproceedings{Wada:etal:ICRA2022a,
    title={{SafePicking}: Learning Safe Object Extraction via Object-Level Mapping},
    author={Kentaro Wada and Stephen James and Andrew J. Davison},
    booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
    year={2022},
  }