We present a novel vision-based grasp point detection algorithm that can reliably detect the corners of a piece of cloth, using only geometric cues that are robust to variation in texture. Furthermore, we demonstrate the effectiveness of our algorithm in the context of folding a towel using a general-purpose two-armed mobile robotic platform without the use of specialized end-effectors or tools. The robot begins by picking up a randomly dropped towel from a table, goes through a sequence of vision-based re-grasps and manipulations—partially in the air, partially on the table—and finally stacks the folded towel in a target location. The reliability and robustness of our algorithm enables for the first time a robot with general purpose manipulators to reliably and fully-autonomously fold previously unseen towels, demonstrating success on all 50 out of 50 single-towel trials as well as on a pile of 5 towels.

/pdf/cloth-grasp-point-detection-based-on-multiple-view-geometric-1karf8krbt.pdf

Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding

We consider the problem of autonomous robotic laundry folding, and propose a solution to the perception and manipulation challenges inherent to the task. At the core of our approach is a quasi-static cloth model which allows us to neglect the complex dynamics of cloth under significant parts of the state space, allowing us to reason instead in terms of simple geometry. We present an algorithm which, given a 2D cloth polygon and a desired sequence of folds, outputs a motion plan for executing the corresponding manipulations, deemed g-folds, on a minimal number of robot grippers. We define parametrized fold sequences for four clothing categories: towels, pants, short-sleeved shirts, and long-sleeved shirts, each represented as polygons. We then devise a model-based optimization approach for visually inferring the class and pose of a spread-out or folded clothing article from a single image, such that the resulting polygon provides a parse suitable for these folding primitives. We test the manipulation and perception tasks individually, and combine them to implement an autonomous folding system on the Willow Garage PR2. This enables the PR2 to identify a clothing article spread out on a table, execute the computed folding sequence, and visually track its progress over successive folds.

A geometric approach to robotic laundry folding

An introduction to computer vision illumination and fixturing sensors image acquisition and representation fundamentals of digital image processing image analysis the segmentation problem 2-D shape description and recognition 3-D object representations robot programming and robot vision bin picking trends and aspirations.

Machine vision: automated visual inspection and robot vision

We consider the problem of autonomously bringing an article of clothing into a desired configuration using a general-purpose two-armed robot. We propose a hidden Markov model (HMM) for estimating the identity of the article and tracking the article's configuration throughout a specific sequence of manipulations and observations. At the end of this sequence, the article's configuration is known, though not necessarily desired. The estimated identity and configuration of the article are then used to plan a second sequence of manipulations that brings the article into the desired configuration. We propose a relaxation of a strain-limiting finite element model for cloth simulation that can be solved via convex optimization; this serves as the basis of the transition and observation models of the HMM. The observation model uses simple perceptual cues consisting of the height of the article when held by a single gripper and the silhouette of the article when held by two grippers. The model accurately estimates the identity and configuration of clothing articles, enabling our procedure to autonomously bring a variety of articles into desired configurations that are useful for other tasks, such as folding.

/pdf/bringing-clothing-into-desired-configurations-with-limited-1eech1c6qk.pdf

Bringing clothing into desired configurations with limited perception

Unfolding of Massive Laundry and Classification Types by Dual Manipulator

A robotic system for handling textile and non rigid flat materials

The authors present a robotic system incorporating vision and force/torque sensing for handling flat textile materials and describe the results of experiments to measure the accuracy and reliability of the system for a variety of representative handling tasks for textile materials.

Robot handling of flat textile materials

The paper describes a robotic-arm gripper with soft-tipped fingers to perform human-like grasps of flexible material pieces out of an unordered bundle. The primary purpose is to achieve the stability and robustness of a human-like grasp with a simple and cost-effective gripper. The work has been stimulated by the handling problem present in the fur-tanning industry.

Picking up flexible pieces out of a bundle

This paper deals with the concept, design, construction and experimental results of a gripper dedicated to the grasping of nonrigid material pieces out of a bundle. Based on a number of functional requirements a gripper is proposed with two passively rotating fingers and its performance is assessed experimentally in the case of grasping furs.

A gripper for grasping non-rigid material pieces out of a bundle

A theoretical and experimental investigation on the stability proper ties of the hybrid control scheme was performed using Lyapunov's theory for both the original scheme, which uses the Jacobian in verse for mapping Cartesian errors to joint errors, and a scheme using the Jacobian pseudoinverse. Both schemes result in position and force controllers that are statically coupled in the task space. Stability analysis shows that the pseudoinverse scheme is asymp totically stable, whereas the inverse scheme may become unstable depending on the manipulator attitude and the environmental stiff ness. In the manipulator workspace, where kinematic instabilities have been reported to exist even away from kinematic singularities, the Jacobian inverse affects negatively the Lyapunov function's posi tive definiteness and the negative sign of its derivative; this effect may become dominant when the environmental stiffness is zero or very low. Experimental results for a 2- and 3-degrees-of-freedom planar manipulator using...

N. Fahantidis

Papers

A robotic system for handling textile and non rigid flat materials

Robot handling of flat textile materials

Picking up flexible pieces out of a bundle

A gripper for grasping non-rigid material pieces out of a bundle

Nonlinear Stability of Hybrid Control