Kinect taught to recognize and translate sign language

Despite the fact that the latest gaming gadget from Microsoft, the Xbox One, was somewhat skeptical (for several reasons, some of which have already been eliminated by the company), the same Kinect remains one of the most interesting technological achievements of recent years. And not in the sense of adapting to the game, but in the sense of the possibility of working with a person as if “directly”. The other day, Kinect was "taught" to recognize sign language, and translate it into phrases that we all understand on the screen.

The new development is a joint project of Microsoft Research Asia and the Institute of Computer Technology of the Chinese Academy of Sciences, Beijing. Of course, software plays a big role in the project, Kinect itself, despite all its power, is just a piece of hardware without software. So, Kinect in this case recognizes the visual information supplied by a person, and the corresponding software perceives this information, analyzes it and translates it into a language that we all understand. The project has two components - in fact, "translation", and work in two directions, when the system translates into sign language and ordinary text printed by a person with normal hearing and language skills.

In the latter case, a 3D avatar is used, which communicates in sign language. By the way, all this is not part of the Xbox gaming world, it is a serious scientific study (with the corresponding achievements), the results of which were published .

Most likely, this development will find its place in the system of communication technologies from Microsoft. As mentioned above, not only for the game.



Via Microsoft