Digital actors

In 2001, the full-length animated film Final Fantasy: Spirits Within was released.(Final Fantasy: Perfume). About a third of the huge budget of $ 145 million was spent on the purchase of servers, workstations, and software. And the filmmakers paid special attention to the realistic modeling of live characters. At that time, it was a real breakthrough in the field of creating electronic actors. Of course, it was still impossible to confuse them with living people, the difference was already small. And many actors from flesh and blood had bad premonitions, voiced by Tom Hanks shortly after the film’s premiere: “It really worries me, very much. This threat is coming and we, the actors, must rally in front of her face to do something. ” 13 years have passed since then, technology has gone far ahead. How did this affect electronic actors? Was it really worth worrying Forrest Gump?

Sinister valley

Three years later, ironically, Tom Hanks himself became the "source" for the character in the full-length animated film Polar Express (Polar Express):



Some kind of creepy feeling from the faces of all the characters, right? This is the so-called "sinister valley" , a psychological effect in which the audience causes a feeling of discomfort (anxiety, fear, uncomfortability) contemplation of characters that look almost like a living person. This unpleasant underlying sensation occurs when the character is deprived of all the features of caricature and simplifications that clearly tell our consciousness that this is a fake, an imitation. And this has become, oddly enough, one of the main problems on the way to creating perfect digital actors.

In the mid-2000s, it seemed that we could never achieve such a realistic artificial people, to finally get rid of the effect of the ominous valley. However, in 2008, the wonderful film The Curious Case of Benjamin Button ( The Curious Case of Benjamin Button) was released. Through the efforts of animators, Brad Pitt throughout the film received the appearance of both an old man and a teenager, while remaining recognizable by old Brad.



And now, six years later, in most scenes it seems like he's just a very well-made actor. But this is not so:



An example of this film showed that in episodes it is already possible to get rid of the effect of the ominous valley, since the cost and labor intensity do not yet allow a full-fledged double. For example, in some scenes of the movie Iron Man 3 , a digital copy of actor Guy Pearce was involved because he managed to grow a beard for filming in another film.

Someone even has a very high-quality simulation of monkeys in Dawn of the planet of apes (Planet of the Apes: Revolution) causes not only surprise, but also some inexplicable wariness. Although this can be explained by the unusual humanization of monkey faces.



The inconvenience of the “sinister valley” effect for the cinema industry is that instead of surprise and joy, the viewer may experience negative emotions. And this is a direct way to the commercial failure of the film, and for what then time, effort and huge money were spent creating special effects? And in modern blockbusters, this is the main expense item of all kinds of resources, not just financial ones.

How many resources are required today for realistic human modeling? Of course, this very much depends on a number of conditions, but on average in the hospital each frameIt requires about 30 hours of operation of a large number of very powerful servers. And in five minutes of the film, 7200 frames at a frequency of 24 frames / sec. At one time, the real breakthrough was the use of motion capture technology, without which it is not possible to achieve full realism of 3D model movements.

Digital Ira

About a year and a half ago, Nvidia published a demo of Digital Ira's face rendering technology.



Here, a very bald man of about thirty with a small one, speaking a little nose, is very realistic. If you look from afar or briefly, then from a living person can not be distinguished. You can even confuse it if you don’t see the eye, despite the fact that serious computing power is used for realistic modeling of their movements.

Surprisingly, until now, the eyes of digital actors have not “yielded” to the animators, they cannot be made completely indistinguishable. Here you will involuntarily reflect on the meaning of the phrase “Eyes are the mirror of the soul” ... Each feature of this digital model requires serious calculations: folds and irregularities of the skin, blood vessels, movements of facial muscles. A man can express pleasure, bewilderment, irritation, surprise, boredom, pleasure and anxiety. Moreover, almost at the same speed as a living person. This is a very important factor, because our brain instantly notes too low or too high a speed of change of facial expressions, as a result of which we have a feeling of falsehood.

How this technology works: first, the head of the human model is scanned, the movements of the facial muscles, jaw, and eyes are digitized. In addition to mechanics, the reflection of light sources, a map of shadows with a particular facial expression, during a conversation is calculated.

So far, two versions of Digital Ira have been demonstrated: from Nvidia in March 2013 at a conference on GPU technologies, and from Activision in July 2013 at Siggraph. Nvidia launched it on one of its video cards, with 4.9 trillion operations per second. The Activision version worked on the PlayStation 4 with a 1.84 teraflop GPU.

Movie

As you can see, we are already close to the main goal that the special effects industry has set for itself - the creation of interactive photorealistic digital actors. And here a whole series of questions and doubts arise.
Imagine that the developers finally managed to create digital people indistinguishable from the real ones (and this will happen sooner or later). This will completely reverse our idea of ​​acting and the role of actors in general. Games will turn into a kind of interactive films. Believing television news in general will be pointless, given the possibility of fake any video. Although even now they cannot be trusted, to be honest.
Despite the technical feasibility of creating an almost completely realistic digital actor, the cost and laboriousness are such that it is cheaper to hire a high-paying Hollywood star. The main reason is the need to use very large computing power, which is constantly getting cheaper. And when the cost of creating an ideal, perfect “digital” becomes comparable or lower than hiring a good actor, then we will face rapid changes.

There will be no need to divide artificial hypocrites into dramatic, comedic, etc. They can play any role, any character. Almost all films with digital actors use three-dimensional models of living people, as in the same “Curious Case of Benjamin Button”. However, the main goal of specialists in special effects is to create arbitrary characters indistinguishable from people. Arbitrary age, appearance, gender, height, fullness, etc. Although digital copies will also be in demand: imagine how many wonderful, great actors could be immortalized in the ever-young “digital artists”, using them for many decades. While we are all aging, the same actors with age can no longer play many roles.

Games

It is quite possible that the rise of “film games” awaits us with a large number of plot forks in which the story will develop in one of several scenarios, depending on the choice made by the “game viewer”. This principle is already used in games, one example is two games from the Walking Dead series .





Here the player is shown a certain story, which he watches as a spectator. But occasionally situations arise that require the player to interact and choose what to do to the character. Depending on this, the plot may develop in a different direction, but within certain boundaries.

This approach can be developed further by creating a small world with a large number of plots. Players will be able to choose characters whose storyline they want to “play”, with the ability to switch to another character if his story seems interesting.

Logical step

Today, in movies and games, characters are usually created often using motion capture technology. True, games are still far from the degree of reliability before cinema. On the other hand, it is a matter of budget and computing power: in the cinema, the viewer is shown a ready-made rendered picture, and in games all the work falls on the long-suffering computers and user consoles.



But still you cannot call an actor a digital copy of a living person. After all, the actor himself forms his image, behavior, emotions, gestures. And after we can reproduce a person’s appearance at an acceptable cost, the next step will be to create “acting” technology for digital characters. So that they do not repeat the operator’s algorithm of actions laid down by the operator, do not “lose” the pre-scanned facial expressions. In fact, a digital actor should be, to some extent, artificial intelligence, which will choose for itself how to play this or that role or scene, how to express emotions. So that he acts on the same principle as we do when we choose how we behave in a particular situation, how to show our feelings and thoughts.

Of course, dreaming is not harmful. The creation of such a technology will require solving the most difficult tasks that today stand in the way of developing artificial intelligence. And even more, because this intellect must be able not only to “speak”, but also to “express itself” non-verbally.

The problem is complicated by the fact that we ourselves need to understand how acting talent works, what psychological (or physiological?) Features make some people actors? We also do not know the mechanism by which the manifestation of some kind of emotion can affect our facial expression, posture, gestures. Scientists still argue that it is primarily from the point of view of deep psychophysiological mechanisms that we either feel happy because we smiled, or we smile because we felt happiness.

So there will be more than one decade, while it will be possible to talk about the possibility of creating full-fledged digital actors. Until then, we will be “content” with digital counterparts of varying degrees of persuasiveness. By the way, there is an opinion that already now we need to begin to fix in detail the appearance, facial expressions and behavior of all kinds of politicians, scientists, influential people - all those whose digital copies could be used in the future to create films and games. For the sake of historical authenticity, so to speak. In fact, this is no different from continuous photographing and video shooting, using these materials for many tasks. Although there is such an opinion that digital actors are evil, and the real art of acting is available only to living people.