Microsoft's new technology allows 3D copies of a real person to speak any language
It seems that in the near future it will not be a big problem if two people speak different languages. Of course, knowledge of an additional language is a big plus, but it happens that you need to discuss an urgent issue, for work, for example, and the interlocutor does not speak your language.
About a week ago, a representative of the corporation, Julia White, demonstrated the new technology at the conference . It allows not only to form a rather realistic hologram (in virtual reality), but also gives this hologram knowledge of a certain language, and the voice - tonality, volume, timbre and other parameters are taken from the original hologram. Thus, the interlocutor sees in front of him a virtual copy of another person, and this copy speaks the desired language.
The technology was made possible by mixing two different solutions - mixed reality and neural text-to-speech. It seems that technology will provide an opportunity to remove the communication barriers that still exist. The Internet has enabled people to communicate in real time, and now there is the opportunity to speak the same language.
The task was solved by the corporation gradually. The first stage is the creation of a realistic white hologram in full growth. In order to achieve this. She visited a Microsoft specialized lab where her presentation was recorded in English. The recording was voluminous in order to create a three-dimensional model of a person from the recording elements.
As a result, this was done - after the completion of the stage, any holder of Microsoft HoloLens video points could watch her performance. Well, after that, work began on copying White's voice and translating her speech into Japanese using text-to-speech technology based on neural networks. The result was excellent - the voice parameters were transmitted almost perfectly. Of course, as much as possible, given that the final speech was in Japanese, the sound of which is very different from any other languages.
Naturally, this is just a demonstration, which took quite a while to cook. But, like any technology, over time, it becomes more efficient and easy to use. Microsoft Corporation plans to further improve and complement its project.
At first, its application will be targeted - for example, with the spread of 3D glasses, performances by famous artists or political leaders will become more popular. They can be seen next to them, and they will speak in their native language for the viewer.
You can also imagine lectures organized in this way. Moreover, it can be safely assumed that turning a person into a hologram that speaks the same language as the viewer will be a matter of several hours, not days. The main thing is the equipment for recording performances in 3D and a neural network, which is able to “translate” the speaker’s speech.