Ray of light in the dark kingdom: how technology returns lost vision

    The main value of a post-industrial society is technology. Forever renewed and improved, running in step with progress and nurtured on the basis of fierce competition, they change everything.



    Life, work, entertainment and improving the world are no longer possible without new technologies. People believe that with their help they can do whatever they want. And they do.

    For a modern person, technology is an integral part of life - its working tool, means of communication, movement, education, comfort. They are “crutches” of our mind and physically limited body. But there are people for whom technology is even more important and serve as crutches, not only in a figurative sense.

    One of the global goals of progress is to remove all existing constraints. Features of health or illness should not become an obstacle for a person, affect the possibilities of his self-realization and quality of life.

    Following this rule, many IT companies that are at the forefront of scientific and technological progress create technologies and solutions that allow people with disabilities to overcome their limitations. Today we’ll talk about this interesting facet of progress.

    One of the most complex and at the same time promising directions in the field of innovation remains the development of rehabilitation tools for people who have visual impairments or who do not see at all. The originality and manufacturability of projects in this area are amazing.

    Artificial vision


    What is a prosthesis - everyone knows. People with disabilities have been using artificial body parts for thousands of years. Modern prostheses have bionic compatibility, and sometimes functionality, which seeks to perform the work of a lost organ.

    Today, almost the same success was achieved by scientists who create visual prostheses. And this is not about cosmetic replacement of the eyes with glass, but about implants that allow completely blind people to see the world around them again.

    In 2012, the bionic eye Argus II was created. The implantable part is an electronic retinal implant, and the external system consists of a camera mounted in glasses in combination with a small processor. Such a prosthesis actually gives vision to people living in total darkness. With some reservations, of course. Patient blindness must be caused by certain diseases, and the artificial vision of the prosthesis will be very limited.



    Antonina Zakharchenko is the first woman in Russia to have Argus II installed. In December 2018, she attended the Technology Hope forum, dedicated to the rehabilitation industry and supporting people with disabilities. Antonina arrived with her husband, but moved on her own, and when talking, she turned her head as if she saw her interlocutors perfectly.

    She lived for many years in absolute darkness, not seeing a ray of light. But in 2017, she underwent surgery to introduce a retinal prosthesis. Then Antonina installed 60 photosensitive electrodes, set up the cameras and the processor of the external device. And as soon as the woman put on her glasses, she regained her sight.



    Of course, this cannot be called full vision: Antonina sees light and dark silhouettes, obstacles, outlines of objects. But most importantly, she sees the light. After years of darkness. “Grandchildren no longer want to play hide and seek,” Antonina complains jokingly.

    Anyone can look at the world as Antonin Zakharchenko sees it right now. The Sensor-Tech laboratory, which actively promotes the implantation of the prosthesis, has developed a VR-simulator that demonstrates the surrounding picture of the world as shown by Argus II. The application is laid out in open access, it can be downloaded for free at one of the links: App Store , Google Play .

    In addition to bionic vision, the VR-simulator of the Sensor-Tech Laboratory shows the majority of known diseases and visual impairments. The development is designed to help ophthalmologists and other specialists understand patients, and ordinary people - their visually impaired friends and relatives.



    The number of people who use bionic vision, today around the world has already exceeded one hundred. Of these, only two are Russian citizens. The installation of a visual prosthesis is a very complicated, high-tech and, as a result, very expensive business. For both Russian patients, the operation was fully paid for by a well-known charity fund in Russia associated with large business.

    But over time, progress will inevitably make solutions for bionic retinal implantation easier, and technologies more accessible and effective. Probably, after two or three decades, each person will be able to use artificial vision, there will be no blind people on Earth and blindness will sink into history.

    You hear? No, I see it that way


    But so far, neither technology nor the economy is able to provide all people who have lost their sight with bionic implants. And progress is looking for roundabout ways. There are a lot of them and each deserves special attention.



    One of the most interesting is sensory replacement technology. This path is based on the mechanism of neuroplasticity of the human brain. In fact, a person is invited to perceive the surrounding reality with “ears”.

    However, natural sounds in order to “see” the world are not enough. Therefore, the Russian project vOICe vision developed the concept of sound vision.

    The project engineers created a device that captures the surrounding space with the help of a camera and converts the picture into sounds using a certain technology. Sound is perceived as an analogue of a black and white image. And the bone conduction headphones that the device is equipped with do not block the ear canal and do not interfere with the perception of the environment.

    As conceived by the developers, vOICe vision replaces one way of perceiving another, and without limitation for the second. A person hears and “sees” at the same time, using for this one organ of perception. Hearing.


    Sounds of various durations and heights are used to turn a picture into a signal. A brighter image fragment is accompanied by a strong signal, and on the side on which the object is located. If an object moves in the field of view, then the signal also moves. Two separate objects are decrypted by two signals, three by three and so on. Increasing the tonality of "neighboring" sounds means an obstacle, while the higher the tone, the higher the obstacle in front of a person.

    Due to the peculiarities of neuroplasticity, the brain over time changes to the conditions offered by vOICe vision technology. Areas of the cortex, which are usually responsible for hearing, learn to convert sound signals into visual images. As a result, after several weeks of training, a completely blind person begins to “see” at the level of the visually impaired and even distinguish between images, and eventually household items, signs with large print and the depth of space.



    With proper diligence and constant use of the device, a person will subjectively see again. That is, his brain will literally begin to turn audio signals into an appropriate image, embossed and voluminous, similar to what our eyes transmit to us.

    And man will see clearly, remaining blind.

    You can learn more about vOICe vision or contact the developers on the project's official website .

    Artificial intelligence sees for you


    The idea of ​​“looking at the world with your ears” is also popular abroad, albeit in a somewhat simplified format. Imagine a technology that gives a “voice” to every object or object that surrounds a person - a desktop, a door, a road sign, a wall and a parked car.

    The device, which looks like vOICe vision glasses, also scans the space, but instead of “black and white” sounds it gives the user the exact name of the objects and the distance to them. This is possible thanks to artificial intelligence technologies that quickly and accurately process the image from the camera.

    It was based on artificial intelligence that the developers of the Department of Neurobiology from the Translational Research Institute named after Tianqiao and Chrissy Chen (China) created CARA or Cognitive Augmented Reality Assistant[intellectual assistant in augmented reality].

    The system combines augmented reality and digital vision technologies. The program uses the so-called spatial sounds, which make objects sound differently, depending on their location in the room. For example, if an object is located to the left of the user, then his voice will sound as if it is coming from the left, and the closer the object is, the higher its “voice” will sound.

    To avoid potential cacophony, when all objects speak at the same time, scientists programmed CARA into several different modes. In the first mode, the surrounding objects speak only if the user looks directly at them. As the owner of the device turns its head, the next object that comes into view calls itself. Thus, a blind person can “look around” to explore the surrounding space.

    In the second mode, all objects take turns calling themselves from left to right, regardless of the user's gaze. In the third mode - focal - the user selects a specific object and only "communicates" with it, using it as a guideline for navigation.



    The Dubai version of smart glasses for the blind Amal 1 has a slightly different functionality. Unlike the Chinese device, Amal 1 reads newspapers and books printed in a standard font, and also understands the image on a computer monitor. Such a device actually allows a blind person to work with text in print or electronic form.

    In addition, the developer equipped his product with a GPS system that helps the blind to determine their location and build routes. For emergency situations, Amal 1 has a SOS function, which, when activated, instantly transmits a message to relatives or special services, attaching a card and photos from the user's location to it.

    By the way, something similar is being done in Russia. The Sensor-Tech laboratory has created an intelligent assistant for the blind, which combines the main functions of foreign analogues: it scans the space around, guesses objects, and also recognizes signs and road signs. In the domestic market, he is better known as "Smart Assistant" Robin " .



    "Robin" is similar to a portable video camera and, in addition to what its foreign counterparts do, can recognize faces. And the signal about the surrounding space is transmitted not only by sound, but also by vibration, like a sonar or rangefinder, reporting the distance to obstacles or objects.

    Another interesting solution was proposed by the Israeli company ICI Vision. Although the device is powerless to help totally blind people, it can make life easier for the visually impaired.

    Israeli Orama glasses combine artificial intelligence, IT tracking software and computer vision technology. The device includes a 3D camera and a miniature laser scanner that projects a three-dimensional image onto healthy parts of the retina, allowing a person to see even those areas of the visual field that have disappeared due to illness.



    The procedure for mapping healthy and damaged cells of the retina allows you to adjust the glasses to the individual characteristics of the user. This is the uniqueness of the Israeli development.

    It is important to note here that with certain types of blindness, part of the visual cells may persist, and a person may not see the space around, but see black and white spots. It is precisely these people who will make ICI Vision glasses again sighted.

    Orama is now intended for indoor use. Glasses allow the wearer to read books, watch TV and navigate in space. Thanks to this, today the device can help those who have difficulties in the workplace.



    Virtual reality without limits


    Over the past year, VR technologies have become more accessible. About 340 million blind people live around the world, and IT giants like Microsoft cannot but take this into account. Several technological solutions of this corporation are entirely focused on those who have lost their eyesight.

    So, last year, Microsoft engineers introduced a virtual reality system adapted for totally blind people. Interaction with game objects takes place in it with the help of a virtual white cane and a joystick synchronized with it, equipped with a vibromotor and sensors that track its position relative to the virtual reality helmet.


    Such a cane allows you to feel the fact of a collision with objects in a VR environment, and also to distinguish objects and surfaces of various types through vibration feedback and sound.

    During the virtual simulation, the user walks around the game room and moves around with a short stick-joystick, as well as real life. But in virtual reality, the cane is much longer. When she collides with an obstacle, the brake stops her movement, creating the feeling of a real collision.

    At the same time, the motor on the cane vibrates, and the speakers in the helmet make a sound that matches the situation in the virtual world. For different surfaces, the system simulates a different tactile response.

    Recent Microsoft developments are designed to incorporate visually impaired people into existing virtual reality. A month ago, the corporation engineers created a set of software functions for VR-helmets that simplify the perception of the virtual world.

    This set includes 14 services that adapt the graphics for people with visual impairments . It includes increasing contrast, brightness, enlarging the image, highlighting the contours of objects, or repainting them in different colors, as well as a sound annotation of objects and actions.

    Nine functions are implemented as a simple plug-in available with any HTC Vive helmet application. The remaining five must be implemented in the application using the package for the popular game engine Unity.

    Microsoft developers hope that such functionality will help people close to loss of vision to get full pleasure from immersion in virtual reality.



    Biological singularity


    In today's world, technology is no longer just a working tool. Now this is also a way for a person to study the surrounding reality. In any case, for those who are deprived of one of the five ways of perceiving the world bestowed by nature.

    Science has not yet learned to completely replace the senses at the same qualitative level. But she is relentlessly moving towards this. Every year, assistive technologies offer more and more alternatives and ways of rehabilitation, significantly improving the quality of life of those whose health is limited due to illness.



    Sooner or later, the moment will come when technological solutions for people with disabilities will exceed the capabilities of an ordinary person. This hypothetical moment of the future will change everything. What today is called “assistive technology” and is being developed for a relatively small category of people, suddenly everyone needs it. After all, everyone wants to have the best.

    When this happens, the person in his classic image will not. This biological type will be replaced by cyborgs, which have more advanced organs of perception and physical, and possibly intellectual capabilities. They will get a new world, the foundation of which is now being laid by the rehabilitation industry.

    And in this world there will be no restrictions caused by disability.

    Also popular now: