Thesis: Dexterity on Robotic Grasping, Manipulation and Assembly

My thesis website is online! Click here!

Research

Overview

• Grasp Planning for Customized Grippers/ Multi-Fingered Hands

• Robust Dexterous In-Hand Manipulation for Multi-Fingered Hands

• Skill Learning for Precision Industrial Assembly

Grasp Planning for Customized Grippers/ Multi-Fingered Hands

Efficient Grasp Planning and Execution with Multi-Fingered Hands by Surface Fitting

Accepted by RAL with IROS2019 option. [Paper Download]

Grasp planning for multi-fingered hands is challenging due to the high-dimensionality, collision and sensing/actuation uncertainties. We propose a method called multi-dimensional iterative surface fitting (MDISF) to plan grasps for multi-fingered hands. The trajectories to reach these desired grasps are further generated by the proposed grasp trajectory optimization (GTO) module. The MDISF algorithm searches for optimal contact regions and hand configurations by minimizing the collision and surface fitting error, and the GTO algorithm generates optimal finger trajectories to reach the highly ranked grasp configurations and avoid collision with the environment. The proposed grasp planning and execution framework considers the collision avoidance and the kinematics of the hand-robot system, and is able to plan grasps and trajectories of different categories efficiently with gradient-based methods using the captured point cloud. The found grasps and trajectories are robust to sensing noises and underlying uncertainties.

Optimization Model for Planning Precision Grasps with Multi-Fingered Hands

Accepted to IROS2019.

We propose an optimization model to search for precision grasps with the multi-fingered hands. The optimization considers several geometric related qualities and collisions, and is relaxed with the proposed optimization modeling method. The relaxed optimization is solved by the proposed iterative PPO-JPO. PPO stands for the palm pose optimization while JPO stands for the joint position optimization. The proposed optimization model is able to locate collision-free optimal precision grasps efficiently. The average computation time is 0.50 sec/grasp. The searching is robust to the incompleteness and noise of the point cloud.

A Learning Framework for Robust Bin Picking by Customized Grippers

[Paper Download]
View This Page for More Details.

Customized grippers have specifically designed fingers to increase the contact area with the workpieces and improve the grasp robustness. However, grasp planning for customized grippers is challenging due to the object variations, surface contacts and structural constraints of the grippers. In this paper, we propose a learning framework to plan robust grasps for customized grippers in real-time. The learning framework contains a low-level optimization-based planner to search for optimal grasps locally under object shape variations, and a high-level learning-based explorer to learn the grasp exploration based on previous grasp experience. The optimization-based planner uses an iterative surface fitting (ISF) to simultaneously search for optimal gripper transformation and finger displacement by minimizing the surface fitting error. The high-level learning-based explorer trains a regionbased convolutional neural network (R-CNN) to propose good optimization regions, which avoids ISF getting stuck in bad local optima and improves the collision avoidance performance. The proposed learning framework with RCNN-ISF is able to consider the structural constraints of the gripper, learn grasp exploration strategy from previous experience, and plan optimal grasps in clutter environment in real-time. The effectiveness of the algorithm is verified by experiments.

Grasp Planning for Customized Grippers by Iterative Surface Fitting

IEEE CASE2018 Best Application Paper Award Winner [Paper Download]
View This Page for More Details.

Customized grippers have broad applications in industrial assembly lines. Compared with general parallel grippers, the customized grippers have specifically designed fingers to increase the contact area with the workpieces and improve the grasp robustness. However, grasp planning for customized grippers is challenging due to the object variations, surface contacts and structural constraints of the grippers. In this paper, an iterative surface fitting (ISF) algorithm is proposed to plan grasps for customized grippers. ISF simultaneously searches for optimal gripper transformation and finger displacement by minimizing the surface fitting error. A guided sampling is introduced to avoid ISF getting stuck in local optima and improve the collision avoidance performance. The proposed algorithm is able to consider the structural constraints of the gripper, such as the range of jaw width, and plan optimal grasps in real-time. The effectiveness of the algorithm is verified by both simulations and experiments.

Real-Time Grasp Planning for Multi-Fingered Hands by Finger Splitting

Accepted by IROS 2018 [Paper Download]
View This Page for More Details.

Grasp planning for multi-fingered hands has heavier computation load than that for parallel grippers, due to the joint-contact coupling, surface nonlinearities and high dimensionality, thus is generally not affordable for real-time implementations. The finger splitting strategy proposed in this paper plans grasps for multi-fingered hands by transferring the knowledge from grasp databases of parallel grippers. In finger splitting, the multi-fingered hand is first initialized by mapping an optimal grasp in the database for parallel grippers, followed by a dual-stage iterative optimization including a contact point optimization (CPO) and a palm pose optimization (PPO), to gradually split fingers and adjust both the contact points and the pose of the palm. The dual-stage optimization is able to consider both the object grasp quality and hand manipulability, address the nonlinearities and coupling, and achieve efficient convergence within one second. Simulation results demonstrate the effectiveness of the proposed approach.

Robust Dexterous In-Hand Manipulation for Multi-Fingered Hands

Robust Dexterous Manipulation under Object Dynamics Uncertainties

IEEE AIM2017 Best Conference Paper Award Finalist [Paper Download]
View This Page for More Details.

Dexterous manipulation has broad applications in assembly lines, warehouses and agriculture. To perform broadscale, complicated manipulation tasks, it is desired that a multifingered robotic hand can robustly manipulate objects without knowing the full dynamics of objects (i.e. mass, moment of inertia) in advance. However, realizing robust manipulation is challenging due to the complex contact dynamics, the nonlinearities of the system, and the potential sliding during manipulation. In this paper, a dual-stage grasp controller is proposed to handle these challenges. In the first stage, feedback linearization is utilized to linearize the nonlinear uncertain system. Considering the structures of uncertainties, a robust controller is designed for such a linearized system to obtain the desired Cartesian force on the object. In the second stage, a manipulation controller using force optimization and torque control regulates the contact force and torque based on the Cartesian force from the first stage. The dual-stage grasp controller is able to realize robust manipulation without contact modeling, prevent the slippage, and withstand at least 40 mass uncertainty and 50 moment of inertia uncertainty. Moreover, it does not require velocity information of the object. Simulation results on Mujoco verify the efficacy of the proposed dual-stage grasp controller.

Real-Time Finger Gaits Planning for Dexterous Manipulation

Accepted by IFAC2017 [Paper Download]
View This Page for More Details.
Dexterous manipulation has broad potential applications in assembly lines, warehouses and agriculture, and so on. To perform large-scale, complicated manipulation tasks, a multi-fingered robotic hand sometimes has to sequentially adjust its grasping status, i.e. the finger gaits, to deal with the workspace limits and object stability. However, realizing finger gaits planning in dexterous manipulation is challenging due to the involved hybrid dynamics, complicated grasp quality metrics, and uncertainties during the finger gaiting. In this paper, a dual-stage optimization based controller is proposed to handle these challenges. First, a velocity-level finger gaits planner is introduced by combining object grasp quality with hand kinematic limitations. The proposed finger gaits planner is computationally efficient and can be solved in real-time. Second, a manipulation controller using force optimization is presented. To deal with mass uncertainties and external disturbances, a modified impedance control is integrated into the manipulation controller. The dual-stage controller does not require the shape of the object, nor does it rely on expensive 3D/6D tactile sensors. Simulation results verify the efficacy of the proposed dual-stage controller.

Real-Time Robust Finger Gaits Planning under Object Shape and Dynamics Uncertainties

Accepted by IROS2017 [Paper Download]
View This Page for More Details.
Dexterous manipulation has broad applications in assembly lines, warehouses and agriculture. To perform large-scale manipulation tasks for various objects, a multi-fingered robotic hand sometimes has to sequentially adjust its grasping gestures, i.e. the finger gaits, to address the workspace limits and guarantee the object stability. However, realizing finger gaits planning in dexterous manipulation is challenging due to the complicated grasp quality metrics, uncertainties on object shapes and dynamics (mass and moment of inertia), and unexpected slippage under uncertain contact dynamics. In this paper, a dual-stage optimization based planner is proposed to handle these challenges. In the first stage, a velocity-level finger gaits planner is introduced by combining object grasp quality with hand manipulability. The proposed finger gaits planner is computationally efficient and realizes finger gaiting without 3D model of the object. In the second stage, a robust manipulation controller using robust control and force optimization is proposed to address object dynamics uncertainties and external disturbances. The dual-stage planner is able to guarantee stability under unexpected slippage caused by uncertain contact dynamics. Moreover, it does not require velocity measurement or expensive 3D/6D tactile sensors. The proposed dual-stage optimization based planner is verified by simulations on Mujoco.

Skill Learning for Precision Industrial Assembly

A Learning Framework for High Precision Industrial Assembly

Accepted by ICRA2019 [Paper Download]

Automatic assembly has broad applications in industries. Traditional assembly tasks utilize predefined trajectories or tuned force control parameters, which make the automatic assembly time-consuming, difficult to generalize, and not robust to uncertainties. In this paper, we propose a learning framework for high precision industrial assembly. The framework combines both the supervised learning and the reinforcement learning. The supervised learning utilizes trajectory optimization to provide the initial guidance to the policy, while the reinforcement learning utilizes actor-critic algorithm to establish the evaluation system when the supervisor is not accurate. The proposed learning framework is more efficient compared with the reinforcement learning and achieves better stability performance than the supervised learning. The effectiveness of the method is verified by both the simulation and experiment.