July 8, 2019 at 15:47Running with prostheses: Nekstgen simulation of human movement using muscles, bones and a neural network
- Transfer
Employees at Seoul University have published a study on simulating the movement of bipedal characters based on joint work and muscle contractions, using a neural network with Deep Reinforcement Learning. Under cut translation of the overview.
My name is Jehee Lee. I am a professor at Seoul National University and a computer graphics researcher with over 25 years of experience. I am exploring new ways of understanding, representing and simulating the movements of man and animal.
The simulation of the movements of bipedal characters based on physics has been a well-known problem in the field of robotics and computer graphics since the mid-80s. In the 90s, most biped controllers were based on a simplified dynamic model (for example, on an inverted pendulum), which allowed using a balance strategy, which can be derived in a closed-form equation. Since 2007, controllers have appeared that use full-body dynamics to achieve rapid progress in this area. It is noteworthy that the theory of optimal control and stochastic optimization methods, such as CMS-ES, were the main tools for maintaining the balance of simulated biped.
Gradually, researchers built more detailed models of the human body. In 1990, the inverted pendulum model had less than five degrees of freedom. In 2007, the dynamic model was a 2D figure driven by engines at the joints with dozens of degrees of freedom. In 2009-2010, full 3D models appeared with 100 degrees of freedom.
In 2012-2014, controllers for biomechanical models appeared, driven by muscles. The controller sends a signal to each individual muscle at each moment in time to stimulate them. Muscle contraction pulls the attached bones and sets them in motion. In our work, we used 326 muscles to move the model, including all the main muscles of our body, with the exception of some small ones.
The number of degrees of freedom of a dynamic system has been increasing rapidly since 2007. Previous approaches to the design of controllers suffered from the “curse of dimensionality” - when the required computing resources (time and memory) increase exponentially as the number of degrees of freedom increases.
We used Deep Reinforcement Learning to solve the problems associated with the complexity of the model of the musculoskeletal system and the scalability of the control of bipedal creatures. Networks can efficiently represent and store multidimensional management policies (a function that maps states to actions) and explore invisible states and actions.
The main improvement is how we deal with muscle activation of the whole body. We have created a hierarchical network, which in the upper layers learns to simulate the movement of joints at a low frame rate (30 Hz), and in the lower layers it learns to stimulate muscles at high frequencies (1500 Hz).
The dynamics of muscle contraction requires greater accuracy than with a skeleton simulation. Our hierarchical structure eliminates differences in requirements.
It's nice to see how our algorithm works on a wide range of human movements. We still do not know how wide it really is and are trying to understand the boundaries. So far we have not reached them due to the limit of computing resources.
The new approach gives improved results every time we invest more resources (mainly processor cores). The good point is that Deep Reinforcement Learning requires computational cost only at the training stage. Once the multidimensional management policy has been explored, simulation and management are fast. The simulation of the musculoskeletal system will soon work in interactive applications in real time. For example, in games.
We use the Hill muscle model, which is the de facto standard in biomechanics. Our algorithm is very flexible, so any dynamic model of muscle contraction can be included in it. Using a high-precision muscle model allows you to generate human movements in various conditions, including pathology, prostheses and so on.
The rectus femoris muscle. 3D surface mesh (left). Approximation with waypoints (center). The approximate LBS coordinates of the waypoints when bending the knee (right).
We share the same fundamental idea with Deepmind's movement research, which is based on a stick-and-motor model. Surprisingly, the standard DRL algorithm works well with the stick-and-motor model, but it doesn’t work very well with muscle-driven biomechanical models.
At the last NeurlPS 2018 exhibition, there was an AI challenge for prosthetics . In the competitive model there are only 20+ muscles, but even for the winner, the result does not look the best.
This example shows the difficulty of learning muscle-driven models. Our hierarchical model makes a breakthrough and allows us to apply DRL to a biomechanical model of a person with a large number of muscles.
Project in PDF .
Project on Github .
Topic explored: Jehee Lee, Seunghwan Lee, Kyoungmin Lee and Moonseok Park.
My name is Jehee Lee. I am a professor at Seoul National University and a computer graphics researcher with over 25 years of experience. I am exploring new ways of understanding, representing and simulating the movements of man and animal.
The simulation of the movements of bipedal characters based on physics has been a well-known problem in the field of robotics and computer graphics since the mid-80s. In the 90s, most biped controllers were based on a simplified dynamic model (for example, on an inverted pendulum), which allowed using a balance strategy, which can be derived in a closed-form equation. Since 2007, controllers have appeared that use full-body dynamics to achieve rapid progress in this area. It is noteworthy that the theory of optimal control and stochastic optimization methods, such as CMS-ES, were the main tools for maintaining the balance of simulated biped.
Gradually, researchers built more detailed models of the human body. In 1990, the inverted pendulum model had less than five degrees of freedom. In 2007, the dynamic model was a 2D figure driven by engines at the joints with dozens of degrees of freedom. In 2009-2010, full 3D models appeared with 100 degrees of freedom.
In 2012-2014, controllers for biomechanical models appeared, driven by muscles. The controller sends a signal to each individual muscle at each moment in time to stimulate them. Muscle contraction pulls the attached bones and sets them in motion. In our work, we used 326 muscles to move the model, including all the main muscles of our body, with the exception of some small ones.
Difficulty in controlling the movement of a bipedal character
The number of degrees of freedom of a dynamic system has been increasing rapidly since 2007. Previous approaches to the design of controllers suffered from the “curse of dimensionality” - when the required computing resources (time and memory) increase exponentially as the number of degrees of freedom increases.
We used Deep Reinforcement Learning to solve the problems associated with the complexity of the model of the musculoskeletal system and the scalability of the control of bipedal creatures. Networks can efficiently represent and store multidimensional management policies (a function that maps states to actions) and explore invisible states and actions.
New approach
The main improvement is how we deal with muscle activation of the whole body. We have created a hierarchical network, which in the upper layers learns to simulate the movement of joints at a low frame rate (30 Hz), and in the lower layers it learns to stimulate muscles at high frequencies (1500 Hz).
The dynamics of muscle contraction requires greater accuracy than with a skeleton simulation. Our hierarchical structure eliminates differences in requirements.
What have we achieved
It's nice to see how our algorithm works on a wide range of human movements. We still do not know how wide it really is and are trying to understand the boundaries. So far we have not reached them due to the limit of computing resources.
The new approach gives improved results every time we invest more resources (mainly processor cores). The good point is that Deep Reinforcement Learning requires computational cost only at the training stage. Once the multidimensional management policy has been explored, simulation and management are fast. The simulation of the musculoskeletal system will soon work in interactive applications in real time. For example, in games.
We use the Hill muscle model, which is the de facto standard in biomechanics. Our algorithm is very flexible, so any dynamic model of muscle contraction can be included in it. Using a high-precision muscle model allows you to generate human movements in various conditions, including pathology, prostheses and so on.
The rectus femoris muscle. 3D surface mesh (left). Approximation with waypoints (center). The approximate LBS coordinates of the waypoints when bending the knee (right).
Using Deep Reinforcement Learning (DRL)
We share the same fundamental idea with Deepmind's movement research, which is based on a stick-and-motor model. Surprisingly, the standard DRL algorithm works well with the stick-and-motor model, but it doesn’t work very well with muscle-driven biomechanical models.
At the last NeurlPS 2018 exhibition, there was an AI challenge for prosthetics . In the competitive model there are only 20+ muscles, but even for the winner, the result does not look the best.
This example shows the difficulty of learning muscle-driven models. Our hierarchical model makes a breakthrough and allows us to apply DRL to a biomechanical model of a person with a large number of muscles.
Project in PDF .
Project on Github .
Topic explored: Jehee Lee, Seunghwan Lee, Kyoungmin Lee and Moonseok Park.
