Robots begin to cope with manipulating arbitrary objects.

Original author: Nikolaus Correll
  • Transfer

As a new generation of grips with improved three-dimensional perception and tactile sensations, it learns to manipulate objects from a large spectrum.



The capture, created by Robotic Materials Inc., founded by the author of the article, performs the task of manipulating the industrial assembly competition at the World Summit on Robotics in Tokyo.

Although robots have been able to make breakfast since 1961General-purpose manipulation (PWS) in the real world may be a problem more complicated than automatic car driving. However, it is quite difficult to describe exactly why this is so. If you look closely at the video from 1961, it is clear that the two-finger parallel grip is suitable for performing quite a large number of tasks, and only the lack of sensations and common sense built into the robot prevents it from performing similar tasks in the real world. A recent article in the journal Science reminded us that even such a rich in contact task as the assembly of furniture, is within the capabilities of existing industrial robots. The real problem is the huge amount of possible movements and manipulations, and the movements that are required to make a butter sandwich, do not necessarily coincide with the movements needed to assemble a chair.

From an industrial point of view, PWS may not be a problem worth solving. After all, we can create a machine for anything - cooking espresso, washing dishes, assembling wheat, mass production of sneakers. That is how most of the robotics is used in modern industry. Even robots that are promoted as “collaboration machines” basically become parts of a more complex machine on the assembly line (and it simply doesn’t need a barrier to work safely). Attempts to develop a PAS, which is interesting from a scientific point of view, are measured in relation to such use cases. Because of this, the advantages of a generalized solution of the problem are not so obvious, and they risk being stuck in the “alley of inefficiency” when investors and industry lose interest in them. However, the processes of production and delivery include a huge number of different stages of manipulation. Even if the value of each stage tends to zero, their total value is economically significant.

How do we know that the solution to the problem of manipulation will be generalized enough to reveal this value? The community of developers of robotics has offered several options for competitions, in which it is necessary either to solve various problems or to manipulate various objects. This, for example, such competitions as RoboCup @ Home , IROS and Amazon Picking Challenge . Although these competitions are agitating for generalized solutions, it is still difficult to come up with tasks that specialized solutions could not cope better with. For example, the winning team of the IROS competition from Korea used the robot Baxterand a system of self-adhesive foam cubes for manipulating objects such as plates and spoons. Similarly, most of the tasks from the Amazon Picking Challenge can be accomplished with just one vacuum pump. In fact, we need a single solution for manipulation, which copes well with all the tasks listed.

Another view was suggested at the industrial competition at the World Summit on Robotics in Tokyo, where they offered a prize of $ 130,000 to the team, which would be able to provide a generalized solution for several industrial problems of container loading and assembly of items that can be switched between in one day. The teams first needed to get items of very different sizes from the baskets (from M3 nuts to electric motors and flexible drive belts), place them in a container, and then assemble complex structures from them. For such a competition, a manipulation solution is needed that can not only capture and manipulate objects, but can also be easily reprogrammed during the day of the competition. In case of successful creation of such robots, they could be used as assistants in assembling furniture,


Vacuum pumps, grippers and soft robots


What are the options for achieving PWS? Industrial automation is dominated by three competing paradigms: pumps, mechanical grippers and hands, and more recently, soft robots. The pumps are in the foreground, as the cup suckers are deformed and take the shape of an object, even when its location is not exactly known. After that, you can suck the air, which makes the bowl hard and creates an annular restriction on the movement of the object. This is an attractive option, since a single suction cup can grab a large number of different objects. However, a suction cup does not solve all the problems - for example, when the object is too heavy, too porous, when further manipulations require precise movements of the object or application of certain effects to it.


Objects with holes are difficult to grab with a suction cup only.

Exact application of forces can be used when using mechanical grippers, which are most often implemented in the form of parallel grippers or two four-link mechanisms. Three-finger solutions are used much less frequently, and they show themselves well when it comes to gripping cylindrical objects from above. The problem of hard grips is that the capture rate when in contact with an object must be zero to avoid transmitting unnecessary pulses. In the case of elastic contacts, the impulse is preserved, with the result that small objects bounce off at high speed. The rebound can be reduced by applying a deformable grip for greater plasticity of the contact, increasing the perception accuracy so that the grip can close in time, or by limiting the possible movements of the object.

In extreme cases, these measures lead to the use of completely soft grips, the deformability of which does not allow the object to bounce, and softness reduces the necessary accuracy of perception. The success of the capture is a large contact area to maximize friction and reduce the rotational degrees of freedom of the object. When gripping a square rod with a two-finger grip, we need to position it so that the fingers are parallel to the two planes of the rod. The soft grip will not need to determine the orientation of the rod and plan the grip, since it will simply grip the object. But the deformability of the grips, reducing the requirements for perception and planning, complicates the controlled application of effort. The position of the object in the soft hand is unknown, and its deformability does not allow to make efforts in a controlled way.

Good practical results can be achieved by combining simple position control and limiting the maximum torque of the motors. As well as deformable soft robotic arms, the grip with the resistance control can adjust to the object, filling in the inaccuracies of perception.

Thus, the ideal capture should become, by necessity, hard or soft, allowing you to capture objects with minimal perception and planning, eliminating the uncertainty of the location of the object and allowing hard manipulations. At the same time, the gripping surface must maintain continuous contact with the object. This can be achieved by combining the above techniques. For example, a soft grip can become tough with the help of granular jamming, or the suction mechanism can be supplemented with a gripping grip to provide additional restrictions. Mechanical grip can be supplemented with a suction cup or electrostatic pads for detachable adhesion. The human hand copes amazingly with the combination of these properties: a combination of hard bones and soft tissues allows you to vary the stiffness, it can cover objects, keeping the possibility of precise control. These opportunities are realized with soft finger pads, rubbing of the skin and the ability to stick - like a small piece of paper sticks to a finger.

Torque Control Captures


Some of the easily achievable capabilities allow us to combine the advantages of soft and conventional robots, creating commercially attractive solutions for PWS. One of them is resistance control as applied to traditional two-finger grabs. By controlling resistance, we control the force of resistance to external movements imposed by the environment. Good practical results can be achieved by combining a simple control layout with the limitation of the maximum torque of the motors. By limiting torque, a hard grip can become arbitrarily deformable (within the accuracy of the torque sensors). Like its fully deformable equivalent, gripping with resistance control can adjust to the object, compensating for inaccurate perception. At the same time, such a scheme can be tough for precise manipulation. Resistance control, together with the perception of finger position, is a form of tactile sensation. Capture will be able to determine the presence of objects in the environment, tracking position and torque. Movement will be obtained soft, filling in the inaccuracy of perception.


Above: grab a strawberry with resistance control. Reducing the maximum allowable torque allows the fingers to stop when in contact with an obstacle and not crush the berry.

Below: Capture a solid object with resistance control. Torque control allows fingers to move until contact appears. Knowing the position of each finger, you can enter the control position at the level of the entire hand.


Capture with torque control can serve as a platform for the realization of the recent results of research of soft manipulators: complementing the fingertips and palm with a suction cup, we combine the advantages of precise position control and effort with the reliability of suction cups. The torque sensor in the finger joints can be supplemented with tactile pressure sensors that are strategically located on the grip. Tactile sensors of the palm and the tips can help to distinguish whether an obstacle prevents the movement of the fingers, or the hand touches the desired object. Tactile sensors also directly add visual sensors, determining the moment of contact and improving the assessment of the orientation of the object and the place with which his hand captured.

Modern achievements of three-dimensional perception make the goal of the MES more close than ever. Three-dimensional sensors like Intel RealSense can sense the presence of objects 11 cm from the camera with an accuracy that allows capturing even small items such as M3 nuts , and complex solutions are already available on the market - for example, Robotic Materials Inc., based on research from my laboratory, just released a beta version of her hand. The joint work of accurate three-dimensional perception, control of resistance for gentle interaction with the environment and various methods of tactile sensation, which make it possible to assess the success of the capture, make it possible to implement reliable manipulation of objects in an inaccurate environment.

For example, recently we have demonstrated the mobile filling of containers, in which the robot is required to extract objects of three different types (nuts M3, part of the mechanism and a rubber strap) from baskets, the location of which on the table is known only approximately. Despite the 10 cm error introduced by autonomous transportation moving through various survey points of the warehouse, the robot was able to detect individual baskets and objects using three-dimensional perception embedded in the arm. The torque limit is used to accurately interact with the contents of the baskets and minimizes the impact of possible collisions. A tactile sensation using torque measurement is used to assess the success of a grip.


Despite the impressive success of three-dimensional perception, resistance control and tactile perception in the PWS, these technologies contradict the prevailing industrial paradigm of specialized solutions for manipulators. Any form of perception takes time and imposes limitations on the speed of capture necessary to limit the energy of unexpected shocks. Therefore, small and medium-sized enterprises operating with a large assortment and a small number of products will be at the forefront of the PWS, as well as large players who want to differentiate their products, reducing the production cycle and increasing the possibilities of customization. At the same time, mobile robots are increasingly found in warehouses, hotels and hospitals. In such situations, certain manipulations like loading,

Also popular now: