An overview of deep machine learning algorithms for robots
The review will be useful for those who are engaged in physical robots and for whom arduino has become insufficient, as well as for people who would like to implement any of the functions of perception of the surrounding world in their robots or devices.
Misty Robotics from Misty Robotics presented at CES
Depending on the purpose of the robot, it needs to solve various problems. It happens that a controller and, for example, the simplest sensors are enough to solve them. However, sometimes it is necessary that the robot could perform tasks not according to rigidly fixed coordinates, but depending on the state of the surrounding space or the work area. Then it becomes necessary to use more sophisticated sensors, such as cameras or lidars, and use special algorithms to process incoming information.
To see and understand what is seen is the first function that can come to mind.
Allows you to find in the field of vision objects of a given type.
Algorithm Example
Allows you to track the movement of an object or objects in the field of vision.
Algorithm Example
Allows pixel-by-pixel determination of the contents of the vision area.
Algorithm Example
Allows you to identify obstacles in the path and distance to them using computer vision.
An example of the algorithm
Sample algorithm for a conventional camera
Sample algorithm for a stereo camera
If your robot's working conditions allow using Depth cameras with active IR backlighting, such as Intel Realsense, then you can use the proprietary SDK .
Most physical robots, be it a manipulator, a mobile robot, or anything else, need to somehow move in space. And sometimes in the process of these movements, especially if the conditions or the environment of functioning can change significantly, the robot needs to change the trajectory and speed of movement.
Allows you to determine the coordinates of the robot's own location in space, including inside buildings.
Algorithm Example
Allows the mobile robot to make decisions about the necessary maneuvers to optimize the trajectory of movement in a dynamic environment. The algorithm uses learning with reinforcement .
Algorithm Example
Allows the robot manipulator to make decisions about the necessary actions needed to capture objects of various shapes. The algorithm also uses reinforced learning.
Algorithm Example
The algorithms described in this article are computationally voracious enough and most often require a GPU. Therefore, depending on the requirements for the operating conditions of the robot, the developer needs to choose a design variant and optimize the code.
Possible options:
The choice made will dictate its requirements both to the equipment and to the optimization of the algorithms.
One algorithm is often not enough to solve a problem. Therefore, when using algorithms in robots, developers will have to face an integration task, i.e. It is necessary to ensure interaction with other algorithms and software.
Working in a company developing software for robots, one often encounters various robots and algorithms, and also finds new approaches to solving complex problems. He drew attention to the fact that interest in functions related to safety and the possibility of autonomous operation of mechanisms, especially in close proximity to people, has recently increased.
The presented review contains a far from exhaustive list of tasks and algorithms. The examples are given by one algorithm chosen by the author. Also note that any of the presented tasks can be solved in many other ways and / or using other algorithms. If you have any more examples of tasks with algorithms on hand, leave them in the comments.
PS: For dessert - a beautiful video from the “heaven” with a wheeled robot and a decision-making system on board
Depending on the purpose of the robot, it needs to solve various problems. It happens that a controller and, for example, the simplest sensors are enough to solve them. However, sometimes it is necessary that the robot could perform tasks not according to rigidly fixed coordinates, but depending on the state of the surrounding space or the work area. Then it becomes necessary to use more sophisticated sensors, such as cameras or lidars, and use special algorithms to process incoming information.
Vision and understanding
To see and understand what is seen is the first function that can come to mind.
Object Detection
Allows you to find in the field of vision objects of a given type.
Algorithm Example
Video
Object Tracking
Allows you to track the movement of an object or objects in the field of vision.
Algorithm Example
Video
Segmentation
Allows pixel-by-pixel determination of the contents of the vision area.
Algorithm Example
Video
Depth estimate
Allows you to identify obstacles in the path and distance to them using computer vision.
An example of the algorithm
Sample algorithm for a conventional camera
Sample algorithm for a stereo camera
If your robot's working conditions allow using Depth cameras with active IR backlighting, such as Intel Realsense, then you can use the proprietary SDK .
Animation
Moving and decision making
Most physical robots, be it a manipulator, a mobile robot, or anything else, need to somehow move in space. And sometimes in the process of these movements, especially if the conditions or the environment of functioning can change significantly, the robot needs to change the trajectory and speed of movement.
Orientation in space
Allows you to determine the coordinates of the robot's own location in space, including inside buildings.
Algorithm Example
Video
Decision making when moving
Allows the mobile robot to make decisions about the necessary maneuvers to optimize the trajectory of movement in a dynamic environment. The algorithm uses learning with reinforcement .
Algorithm Example
Animation
Capturing and manipulating objects
Allows the robot manipulator to make decisions about the necessary actions needed to capture objects of various shapes. The algorithm also uses reinforced learning.
Algorithm Example
Video
Features of implementation for robots
Performance
The algorithms described in this article are computationally voracious enough and most often require a GPU. Therefore, depending on the requirements for the operating conditions of the robot, the developer needs to choose a design variant and optimize the code.
Possible options:
- computing on a dedicated PC
- cloud computing
- calculations on board
The choice made will dictate its requirements both to the equipment and to the optimization of the algorithms.
Interaction
One algorithm is often not enough to solve a problem. Therefore, when using algorithms in robots, developers will have to face an integration task, i.e. It is necessary to ensure interaction with other algorithms and software.
Instead of output
Working in a company developing software for robots, one often encounters various robots and algorithms, and also finds new approaches to solving complex problems. He drew attention to the fact that interest in functions related to safety and the possibility of autonomous operation of mechanisms, especially in close proximity to people, has recently increased.
The presented review contains a far from exhaustive list of tasks and algorithms. The examples are given by one algorithm chosen by the author. Also note that any of the presented tasks can be solved in many other ways and / or using other algorithms. If you have any more examples of tasks with algorithms on hand, leave them in the comments.
PS: For dessert - a beautiful video from the “heaven” with a wheeled robot and a decision-making system on board
Video