Sony: we will teach you to look at the world through the eyes of others

Jun Rekimoto, exploring the possibilities of augmented reality at Sony Computer Science Labs, uses cameras, drones, and sensors to share what other people see and feel.

Yun Rekimoto predicts a future in which sports fans will be able to see the game through the eyes of the players themselves, to feel their feelings on the playing field, without getting off their home sofa.

The jacking in concept, popularized in William Gibson’s canonical novel Neuromancer (1984), has been roaming science fiction for three decades - Rekimoto has been trying to make this idea a little more real.

Rekimoto, deputy director of research at Sony Computer Science Laboratories in Tokyo, outlined his research in this area as part of the first scientific symposium held by a laboratory in the United States. The event was held last week at the New York Museum of Modern Art, where scientists discussed their work in the fields of music, art and ... prosthetics. And this is only a small part of the scientific areas discussed during the symposium.

Jun Rekimoto is currently interested in augmented reality and decided to borrow ideas from Gibson’s novel and apply them to modern technologies: “I wanted to expand this concept, which implies the ability to connect to other people or drones with the creation of a full presence effect,” the researcher said during interview.

Augmented reality technologies have been around for years. But even the uninitiated can notice the rapid development of technologies that realize virtual reality - remember Oculus Rift or Project Morpheus, focused on gaming virtuality, developed by Sony. Even Samsung, working with Oculus, plans to sell a virtual helmet that works in conjunction with the Galaxy Note 4 smartphone. These devices can completely change the way we watch movies, play games and communicate with each other.

“When you combine visual experience (in this way), it can lead to an entirely new way of communicating,” said Brian Blau, a consumer technology analyst with research firm Gartner. “To see something through the eyes of another person - I think this is a promising idea.”

Rekimoto is a patient person. Some of the ideas he is working on are unlikely to hit the market in the coming years. In one project, he and his colleagues are developing a gadget called LiveSphere, which is worn on the user's head, includes six cameras and can capture a panoramic image of everything that happens around its owner. According to Rekimoto, a LiveSphere user will be able to use outside help, for example, to learn new culinary skills or perform medical procedures. Athletes will be able to wear a similar device so that the viewer can see in a new way what is happening during the competition.

“The resulting image is impressive,” is how Rekimoto describes testing a gadget with a gymnast spinning on a bar.

Rekimoto also said that researchers working for Sony are working on implementing tactile sensations in LiveSphere using a “tactile device” consisting of micro-drives that are worn on the user's fingers.

Another project of the Rekimoto laboratory is something called the Flying Head (Russian “flying head”). Literally, a drone repeating the movements of the user's head. This technology can, for example, be used to assess athletes' athletic skills during training. Another option - connecting to a personal drone can simplify control of the drone in areas that are dangerous for the direct presence of people.

“I think that an even more important or promising application (of such technology) can be to expand the user's perception through the perception of other people,” Rekimoto shares his ideas. “This could lead to the emergence of an entire industry where the abilities of one person are transferred to another.”

The original article about Yuna Rekimoto's projects is  here .