How to teach a robot to cook?

    Researchers at the University of Maryland and the Australian ICT Research Hub have announced a method that will allow robots to learn how to cook. At the same time, robots will use a fairly common method today - they will watch videos on Youtube.

    image

    Researchers used recently developed deep learning methods and neural networks - programs that simulate the functioning of the human brain using artificially created neurons. Scientists “fed” the system with training videos published on Youtube, and the system, in turn, identified the objects used by the cooks, and tried to understand what actions are needed for cooking.

    So scientists have developed a system that in the future will allow the robot to perform the most basic tasks in the kitchen, as well as recognize them directly while watching videos.

    Although, it would seem, some robots already know how to cook. For example, Cooki from a Serenity startup performs actions in a given sequence, boils water, adds pasta to it, fries meat - in general, does everything to replace you in the kitchen.

    image

    image

    image

    Motherboard has interviewed Cornelia Fermuller, a member of the development team working on teaching robot cooking in videos.

    How do you make a robot learn to cook?

    There is a lot of research on the subject of learning how to work with a robot; this is a well-studied field. But the work is aimed at copying movements - the robot sees the action and repeats it. In real life, everything is much more complicated, because you need to adapt to different situations. A can or a spatula can be of different sizes, can be in different places on the kitchen table, the table itself can be littered with something - all this complicates the work.

    The robot must be able to perceive the world. For example, a “cook” needs to open a can of peanut butter, which always stands in one place. And the man rearranged her. It is necessary to prepare the car for such situations.

    Is the deep learning technique and use of YouTube videos sufficient to overcome these problems?

    The robot watches ordinary videos, not scenes shot in the laboratory. There are many ways to shoot, videos differ from each other in angle, scenes, lighting and many other characteristics. The review may obscure the hand; the product may obscure other objects. Here we use the deep learning technique . This software allows you to recognize objects, including when they are in the hand of a person.

    Our robots are not cooking yet. But we are already capable of something: we can pour water and prevent something, but we have not yet done the robots that are able to work autonomously and cook food by learning this on their own.



    What technical hurdles do you face?

    First of all, you need to solve all the above problems. We must teach the robot to perform actions in different conditions, places and situations. On the other hand, anything can happen during cooking. Everything must be taken into account.

    The robot must have intelligence, the ability to reason, recognize unforeseen events and respond to them. That is, to be autonomous. We continue research, work on artificial intelligence, but have not yet created it. Progress is underway, but it takes more time.

    Returning to training - how did you think of using videos to train your robot?

    People manipulate in various ways. It is not enough to watch the video and force the robot to perform the same action. I and you can open a can of peanut butter in completely different ways.

    We solved this problem this way: we need to think not about movement, but about the result. The robot breaks the action into small pieces.

    To open a can of peanut butter, you need to perform the following steps: move your hand towards the can, hold the lid, turn the lid, remove the hand with the lid. Understanding these actions, this “grammar”, as understanding the language - you can also decompose a sentence into parts.

    When the robots start watching videos and cooking, how are we?

    Hard to tell. This will happen, especially if the situation is calm, children will not run around. But more time is needed. One must deal with all kinds of perceptions of things. For example, when the robot goes to the refrigerator, he must understand that the salad may not be at home - you have to go to the store.

    Also popular now: