Robots rent apartments through Airbnbs to learn to better grab items

Perhaps you like to look on Airbnb for dwellings with large rooms, or very bright apartments, or bathrooms with twin sinks. But if you are a robot, all you need is a little variety. Rug here, parquet there. Because you are a pioneer, not just a tourist.

At least this is the case for special robots developed by a team from Carnegie Mellon University. To train machines to manipulate objects in the real world, they needed access to many different houses — first they had their own houses, then friends’s houses, and then the available houses ran out. Therefore, having trained their robots to a certain limit, they went with them to the test on the unknown places of Airbnb.



And, yes, until you asked - the owners of the houses knew what the premises were for. And, yes, robots - mostly looking like robotic vacuum cleaners with a hand - proved to be great guests. (They stayed for about a day and a half, and along with them were several scientists who conducted the tests). “I remember how the owners were very interested in this work, and it was interesting for them to see how the robots would behave in various places of the houses,” says robot Lerrel Pinto. "They said that we can safely use robots in other parts of the house." And the researchers did so. "Some of them were very curious, they watched how the robot works, how it moves, asked if he can pick up trash from the floor."

They did not know how. What they could do was demonstrate the acquired ability to handle new objects brought by the researchers, from staplers to stuffed toys and spray bottles. Researchers placed them on different surfaces, carpets and parquet floors, which gave the robot the opportunity to practice working with different backgrounds.

In fact, cars can be taught to grab objects in two ways: in a simulation, or in the real world. Simulations are good for their speed; You can make a digital model of the robot drive hundreds of collisions in the time it takes for a real machine to move its elbow and wrist a little. Unfortunately, in the digital world it is impossible to thoroughly model the real: physical experiments remain the only way to verify that trainings are really able to cope with real physics. You can also use simulation training - when a person controls a robot, while the robot learns to do it in the meantime, but it takes a lot of effort.

The final physical test consists in bringing the robot outside the sterile environment, specially prepared for laboratory tests, into a chaotic and chaotic world of humans. “We need to take our robots home,” says Abhinav Gupta, a robot technician who helped develop the new system. "We need to collect a lot of data about the manipulations in a real environment in which the floors may differ - it could be a carpet, tile or board."

When researchers trained robots at home, they already had some prior knowledge. For example, that for capture it is necessary to see an object with the help of computer vision, reach it and take it. The question is what place. “The robot was picking a random spot and trying to squeeze the fingers, checking if the grip was a success,” says Gupta. “In fact, was it possible to lift the object from the floor or not.” The robot can determine the success of the capture thanks to the built-in force sensors, as well as, seeing the object in hand.

“At first, everything happens by chance, but after several thousand repetitions, he begins to learn where he managed to do it and where not,” adds Gupta. Thus, the robot can learn to work with real objects, and then use this data, trying to capture everything that comes to it in the house. Unlike the laboratory, everything happens under different lighting conditions and on different floors, so robots collect a richer set of data that more accurately represents the real jobs of robots in the future - for example, if they clean up the apartments of older people. Therefore, once in a rented apartment - in unfamiliar surroundings - he can adapt, not panic. As a result, the robot was able to grab objects unknown to him in 62% of cases, when the model, trained in the laboratory, managed only 18.5% of cases.

This does not mean that laboratory training is outdated; Complex robots capable of performing tasks with a tolerance of a few millimeters are crucial for research on robotic captures - and this area remains a problem for robots. But such robots are too big and expensive - up to tens of thousands of dollars - so that you can experiment with them at home. Researchers have collected more mobile home robot for only $ 3000.

Not without compromise, for example, the use of motors with a tolerance of centimeters, and not in millimeters. This is not very good - imagine that you are mistaken for a centimeter, trying to grab a can of soda. “But we tried to model chance,” says Gupta. “We are not only trying to learn to grab, but also to learn what mistakes the controllers may have.” When they managed to model it, they were able to adjust the robot's slightly capricious movements.

“The work shows how it is possible to take into account accidents in such an uncontrolled environment and work with inexpensive equipment, and at the same time bring the process of data collection outside the laboratory. This can help get an extensive set of highly scalable, varied and generic data, ”says Xavier Puy, who works in robot training in simulations at MIT.

This is great for robots and for owners of rented apartments. Robots in fact never dare to leave a mess in the apartment.