Full immersion in virtual reality: present and future

What is total immersion? This is when the difference between the virtual and real worlds is not felt. That is, you don’t feel which of the worlds you are in.

In this article, we will talk about what technologies are currently for immersion in virtual reality at present, about the pros and cons of different types of tactile feedback and about the future of complete immersion.

The material was prepared on the basis of a lecture by Denis Dybsky, which was held at the VR-Today conference as part of our educational program "Game Project Management" at the Higher School of Economics . Video and abstract under the cut.

Full immersion components

• The first and most important point is the visual picture . Everyone is used to the fact that immersion in virtual reality occurs with the help of helmets of virtual reality. As a rule, HTC Vive, Oculus Rift, Gear VR, PS VR and other helmets that are currently on the market.
• The second important point is the sound . Without sound, it is impossible to plunge into virtual reality at the moment, because the picture must be fully combined with sound. In order for the user, while in virtual reality, to be able to position himself in space and know where he is.
• The next, even more important point is the tactile connection or haptic . In Western terminology, it is called haptic feedback - “tactile feedback”.
• Simulation of taste .
• Odor simulation .
• The position of man in space .

These are 6 components that allow a person to fully immerse themselves in VR. Consider in more detail tactile feedback (haptic) . This is a technology that allows you to receive tactile information through touch. This is a rather complicated technology, at the moment there are several variations of its implementation on the market that allow you to transfer the sense of touch, sensation and so on in different ways.



Types of tactile feedback:

• The first of these is power feedback . For example, in car simulators using the steering wheel, the tactile feedback from it is felt in collisions, and so on. This is power feedback. It allows you to feel pressure on the hands (usually) or on the body.
  
  
• The next and most common, currently most studied type of tactile feedback is vibrotactile feedback . The most striking example is the vibration of a smartphone. She lets us know when a message arrives or a call arrives.
  
  
• The next type is quite complex and not widely distributed at the moment on the market - this is ultrasound . It allows using the generation of high frequency sounds to feel the shape and texture of the object. At the moment there are a couple of solutions on the market that allow you to use this technology.
  
  
• Thermal feedback is another type of feedback. It allows in virtual reality to feel the cold, heat, the transition from heat to cold and vice versa.
  
  
• Probably the most accurate way to transmit sensations is electrical stimulation . For example, belts in order to get in shape, which are positioned as belts for burning excess subcutaneous fat, use electrical stimulation. These are small electrical impulses that work, as a rule, at different frequencies, amplitudes and amperage. In VR, a chemical reaction is quite difficult to imitate, but electrical stimulation is very easy to reproduce. You can configure an individual electrical signal for each sensation and feel touch, hit the ball in any part of the body or even rain.

Existing problems

1. Now for immersion in virtual reality, only 2 of 5 feelings are stimulated - this is vision and hearing.
   
   
2. An even bigger problem than the previous one is the presence of wires in the PC and console helmets. A large amount of data passes through a virtual reality helmet, and this requires wires. In order to feel completely free in virtual reality, you need to remove them.
   
   
3. The problem of interaction with the virtual world. As a rule, in order to fully interact with it, we need controllers. Now the role of controllers from different manufacturers is played by ordinary controllers Vive, Oculus Touch, etc. But in order to fully interact with objects, to be able to touch them, turn, take, feel its texture, weight, you need gloves of virtual reality. What should be included in them: at least it is a motion capture system in order to be able to track the position of the hand in space, how the fingers move, whether they are compressed, how the hand rotates relative to the whole body. Fine motor skills should be monitored. When you touch a virtual object, there should be feedback. For example, a texture can be made using electrical stimulation. If this is some kind of large object, for example,
   
   
4. Due to the fact that at this moment there are solutions on the market that mainly use vibration or power feedback, this reduces the quality of interaction in a virtual environment , because they do not allow you to accurately convey all the sensations.
   
   
5. Another point: in virtual reality, when fully immersed, you need to simulate walking . How can I do that? The first way is teleportation. For example, it is implemented in HTC Vive., there, using the controller, you can teleport to different places in a virtual environment. The second is directly physical walking around the room. But in order to fully immerse yourself, depending on the volume of the virtual world into which you are immersed, you need a small or large room. But for full movement in the largest open worlds, it is impossible to use only a small room. Since you need to go in all directions, it is not very convenient. The third gadget that more or less solves this problem is Tread Meal (now there are several offers on the market), which allows you to move directly in it, it supports the human body and does not let the person get tired.

Based on the foregoing, we can conclude that at the moment there are no systems of complete immersion in virtual reality. These systems so far can only be called partial immersion systems. It may seem to someone that it is enough to use only a helmet to immerse yourself in virtual reality, but no, this is not true.

Examples of systems that go to completely immerse a person in VR.

CAVE
This is a project that was launched in 1992. It usually consists of huge displays throughout the room. These are speakers built into the wall, directional sound, a motion capture system and stereoscopic displays.

The VOID
VR Amusement Park, which opened several years ago in Utah, USA. Physical locations on which you can walk in it are fully consistent with virtual.

AlloSphere
10-meter sphere with many stereoscopic displays, built-in motion capture system, sound and so on.

Teslasuit
Suit for full immersion in VR. It looks like an ordinary suit, has several systems in it.

• The sensory transmission system, that is, the Haptic Feedback System. It allows you to masquerade accurately convey the sensations of VR. You can feel how someone is touching you, even if he is 1000 km away from you.
• Motion capture system. Allows the user to track his position in space and move around it. At the moment this is inertial tracking, but now a hybrid mocap is being developed, in which optical tracking will be used.
• Climate control. Allows you to feel the cold, heat, decrease or increase in temperature.
• The suit is completely wireless.
• In order to allow third-party developers to use the costume, their own SDK was developed.
• Glove with a haptik.
• 5G and cloud for processing (in plans). Today, all devices require powerful hardware in order to fully launch content and there are no lags. The suit will allow all processing to be transferred to the cloud. This will save users from iron, remove all excess peripherals. Since peripherals greatly reduce mobility in VR, the cloud must be quite powerful, distributed and all sorts of nuances. Therefore, its creation will take at least 3-5 years.

Now look a little into the future

Systems are being developed that will allow you to connect a computer directly to the human brain. They have been developed for a long time. Well, the most talked about project is Neuralink from Ilon Mask. This is a very complex project, judging by the current level of technology, this will happen no earlier than in 15-25 years, and possibly more.

A suit is a closer option in time. A suit is a system that allows for the full training of a person’s vital indicators. The developers have plans to develop smart clothes in which various sensors will be built-in (temperature sensors that measure the level of oxygen in the blood, humidity and other sensors that allow you to take vital signs of the body).

In parallel with the suit, there are many developments in gloves. They allow you to interact with objects. Gloves are also crammed with sensors that work on electrical stimulation. In the glove itself there are all the same control systems that are in the suit.

Another important point is the simulation of weight in virtual reality. Everyone wants to not only feel the object, touch it, feel its shape, but also to feel how much it weighs. This is real with the help of electrical stimulation. Now there are studies that allow us to say that with the help of electrical stimulation you can very accurately simulate the weight of an object. Such a thing will also be timely or a little later.


In the near future we will post on our Habrahabr blog notes and notes from all the other lectures from the VR-Today virtual reality conference. Subscribe to receive notification of their release.

In the fall, training begins on our educational program “Game Project Management”, we are inviting you to open doors on June 28 !