Multi-user VR: how to implement?
Hi, Habr! After we have reviewed some of the ways to build multi-user virtual / 3D spaces in the previous article, let us return to them in the context of learning. How, for example, to qualitatively train the same team consisting of completely different people? Details under the cut!
Simulations and serious games are one of the most effective types of training - it is a recognized fact. Here is an example from the medical industry.
However, with simulations, things are not so obvious. Many modern companies include several generations in the headquarters of employees, and each of them learns in a new way. A good example is airlines — the age of employees can vary from 20 to 60 years, while continuing retraining is one of the requirements of the industry; It is worth noting that it is often required to analyze and train not one person, but a whole team, for example, testing the quality of collaboration.
With all this, as we have said - to enter the VR / 3D training in modern infrastructure is quite a difficult task. Here, the multi-user spaces described in the last part and a certain approach to their design can help.
Any simulation is fundamentally different from passing a test suite. The main difference is the degree of immersion in the situation. This difference can often be critical. A person can perfectly know the theory, having learned the answers to the questions of tests - and get lost in a real situation, which will be different from what he imagined. This is partly why, in fact, there is the concept of blended learning. There are things about which we cannot say that the person learned them until he really tried them.
At the same time, the range of such possible trainings is very large, and not all of them require some, for example, purely muscular / reflex skills; The very skill of applying theory also often requires being inside a situation, and not analyzing it from the outside. The simplest example is actions in emergency situations or other stressful situations. (Startle effect)
In this example, a stressful situation is the sum of concrete situations, for each of them there are clear instructions. But it is very difficult to assess whether a particular situation from this amount is known to man.
We provide a mechanism that allows you to simply run any set of situations in a multi-user virtual environment; there remains, however, the question - how do we evaluate the actions of users in such a simulation.
We assume that the collection of training data has two main objectives:
An integral part of blended learning is adaptive learning. The system analyzes the level of knowledge and selects the student theoretical material or individual trajectories. There are several options for analyzing information, on the basis of which the system makes a recommendation.
When a person solves a problem and makes many mistakes, the system selects for it a supporting option that closes the gap in the person’s knowledge.
The approach to adaptive learning, based on memory, a person constantly repeats the learned material.
The teacher, or an expert, creates a graph of topics, and it is used to create an individual learning path.
We consider two main approaches - using a general scheme of transitions within the course and with the topic graph.
The first approach is quite simple to implement - we simply tie in to the virtual experience, as if it were a regular lesson, that with the successful passage some further topics will open up.
The second is more interesting, since we can bind some topic from the course to a separate part of the virtual experience, and evaluate the knowledge of a person on this topic, having received data on a student’s knowledge of several topics on the basis of a single “VR track”.
But there is another option, and, in our opinion, it is most interesting.
Imagine a rather complex simulation, which includes several simultaneously working scenarios, and is multi-user. The goal is to evaluate the effectiveness / knowledge of not only one person, but the entire team; nor do we know exactly whether we built the training course itself. Of course, we have certain sets of competencies, exercises, and so on, but we ourselves cannot accurately assess the effectiveness of these materials.
We can mark out some outcomes of individual scenarios, and try to evaluate them together. But there is another option - to record every action of a student in the virtual space using the xAPI protocol - developed, in part, specifically for working with simulations and serious games.
At the same time, it is interesting that the record of the average “track” of the passage will be sufficient to apply the ML / Data Mining methods to it. We get (especially on the result of several “tracks”) a student profile in which we can look for the most varied correlations with our training course.
You can imagine a lot of options for working with this kind of educational statistics, it is quite difficult to list them entirely, they will depend directly on the structure of the training course and the requirements for it. The simplest, for example, is a variable complex scenario, where there is not one correct way to perform actions, but there is a certain criterion of efficiency / teamwork; in this case, you can record several “ideal” tracks of the passage and analyze the discrepancies with the student's track. Another option is the assumption that the course is generally formed imperfectly, and moving to classes in VR, students miss a certain theoretical point. It will be easy to select, simply noting all the errors in a certain step of the complex scenario.
Speaking of more “infrastructure” things, working with such data requires:
At the moment, we are continuing to develop similar options for working with statistics from VR applications, although, of course, in many ways these are still experimental options.
Company Jedium - Microsoft Partner company working in the field of virtual, augmented reality and artificial intelligence. Jedium has developed a framework to simplify the development of complex projects on Unity, part of which is publicly available on GitHub . Jedium plans to replenish the repository with new modules of the framework, as well as integration solutions with Microsoft Azure.
Vitaly Chaschin - Software developer with more than 10 years of experience in the design and implementation of three-dimensional client-server applications - from concept to full implementation and integration of applications and virtual reality solutions. Systems Architect Jedium LLC, MSc in IT.
Alexey Sarafanov
marketing manager at Jedium LLC.
Sergey Kudryavtsev
CEO and founder of Jedium LLC.
Simulations and serious games are one of the most effective types of training - it is a recognized fact. Here is an example from the medical industry.
However, with simulations, things are not so obvious. Many modern companies include several generations in the headquarters of employees, and each of them learns in a new way. A good example is airlines — the age of employees can vary from 20 to 60 years, while continuing retraining is one of the requirements of the industry; It is worth noting that it is often required to analyze and train not one person, but a whole team, for example, testing the quality of collaboration.
With all this, as we have said - to enter the VR / 3D training in modern infrastructure is quite a difficult task. Here, the multi-user spaces described in the last part and a certain approach to their design can help.
Any simulation is fundamentally different from passing a test suite. The main difference is the degree of immersion in the situation. This difference can often be critical. A person can perfectly know the theory, having learned the answers to the questions of tests - and get lost in a real situation, which will be different from what he imagined. This is partly why, in fact, there is the concept of blended learning. There are things about which we cannot say that the person learned them until he really tried them.
At the same time, the range of such possible trainings is very large, and not all of them require some, for example, purely muscular / reflex skills; The very skill of applying theory also often requires being inside a situation, and not analyzing it from the outside. The simplest example is actions in emergency situations or other stressful situations. (Startle effect)
In this example, a stressful situation is the sum of concrete situations, for each of them there are clear instructions. But it is very difficult to assess whether a particular situation from this amount is known to man.
We provide a mechanism that allows you to simply run any set of situations in a multi-user virtual environment; there remains, however, the question - how do we evaluate the actions of users in such a simulation.
We assume that the collection of training data has two main objectives:
- Assess student competencies;
- Help him learn in the most effective way. In other words - to build an individual learning path.
An integral part of blended learning is adaptive learning. The system analyzes the level of knowledge and selects the student theoretical material or individual trajectories. There are several options for analyzing information, on the basis of which the system makes a recommendation.
- Based on transition rules
When a person solves a problem and makes many mistakes, the system selects for it a supporting option that closes the gap in the person’s knowledge.
- Cognitive approaches
The approach to adaptive learning, based on memory, a person constantly repeats the learned material.
- Based on the topic graph
The teacher, or an expert, creates a graph of topics, and it is used to create an individual learning path.
We consider two main approaches - using a general scheme of transitions within the course and with the topic graph.
The first approach is quite simple to implement - we simply tie in to the virtual experience, as if it were a regular lesson, that with the successful passage some further topics will open up.
The second is more interesting, since we can bind some topic from the course to a separate part of the virtual experience, and evaluate the knowledge of a person on this topic, having received data on a student’s knowledge of several topics on the basis of a single “VR track”.
But there is another option, and, in our opinion, it is most interesting.
Imagine a rather complex simulation, which includes several simultaneously working scenarios, and is multi-user. The goal is to evaluate the effectiveness / knowledge of not only one person, but the entire team; nor do we know exactly whether we built the training course itself. Of course, we have certain sets of competencies, exercises, and so on, but we ourselves cannot accurately assess the effectiveness of these materials.
We can mark out some outcomes of individual scenarios, and try to evaluate them together. But there is another option - to record every action of a student in the virtual space using the xAPI protocol - developed, in part, specifically for working with simulations and serious games.
At the same time, it is interesting that the record of the average “track” of the passage will be sufficient to apply the ML / Data Mining methods to it. We get (especially on the result of several “tracks”) a student profile in which we can look for the most varied correlations with our training course.
You can imagine a lot of options for working with this kind of educational statistics, it is quite difficult to list them entirely, they will depend directly on the structure of the training course and the requirements for it. The simplest, for example, is a variable complex scenario, where there is not one correct way to perform actions, but there is a certain criterion of efficiency / teamwork; in this case, you can record several “ideal” tracks of the passage and analyze the discrepancies with the student's track. Another option is the assumption that the course is generally formed imperfectly, and moving to classes in VR, students miss a certain theoretical point. It will be easy to select, simply noting all the errors in a certain step of the complex scenario.
Speaking of more “infrastructure” things, working with such data requires:
- Availability LRS. The xAPI standard rigidly separates a database of similar statistics from LMS / analysis / processing. But since we are talking about fairly large amounts of data, we consider the concept of a distributed repository of training records, with various options for their verification. For example, a variant using blockchain is possible;
- Enough rich visualization environment of similar statistics and work with it. There are few traditional tools for building progress reports; other tools are needed; for example, we are considering Elasticsearch + Kibana, another option is PowerBI.
At the moment, we are continuing to develop similar options for working with statistics from VR applications, although, of course, in many ways these are still experimental options.
The authors
Company Jedium - Microsoft Partner company working in the field of virtual, augmented reality and artificial intelligence. Jedium has developed a framework to simplify the development of complex projects on Unity, part of which is publicly available on GitHub . Jedium plans to replenish the repository with new modules of the framework, as well as integration solutions with Microsoft Azure.
Vitaly Chaschin - Software developer with more than 10 years of experience in the design and implementation of three-dimensional client-server applications - from concept to full implementation and integration of applications and virtual reality solutions. Systems Architect Jedium LLC, MSc in IT. Alexey Sarafanovmarketing manager at Jedium LLC.
Sergey KudryavtsevCEO and founder of Jedium LLC.