November 27, 2013 at 15:06ScienceHub # 08: Neuro Interfaces
One of PostNauka’s favorite experts, Alexander Kaplan, not only told us everything about neurointerfaces, the ability to control his brain and interact with external devices, but also showed his office and also conducted an experiment with a member of our film crew. About this in more detail below.
What Kaplan is engaged in can be called the development of a new paradigm in psychophysiology. Until recently, scientists studied the brain in terms of its electrical activity, since neurons or the brain work on electricity. Nerve cells communicate with each other, transmit information to each other using current pulses or what is called spikes. And so the idea came up to take an electric signal and make a command out of it to control external devices. This was made possible thanks to electroencephalography, which was previously used to diagnose diseases. This method consists in the fact that the electromagnetic field from the electrical life of cells breaks through the skull to the surface of the head onto electrodes that are pressed against the skin surface.
But the conversion of this electrical signal into an instrument for controlling external technology is not a toy at all. The information about the outside world that we store and synthesize in ourselves, we manifest outward with the help of muscles, this is our only way, but can everything be transmitted through thoughts?
Alexander Kaplan: “This is an organic channel through which you can express the inner world, which is created at many entrances. If there was any direct channel for delivering information from the brain, we could at least compare something. Or to develop a philosophical neurophysiological problem. We say now that such a channel can appear if electric signals are used as an engine. But this is a more general aspect, and there is also a pragmatic aspect. If we receive a control signal directly from the brain, then in this way a person can directly say something without using muscles. Exactly this situation occurs in a number of diseases. In particular, with a stroke. After a stroke, people may be immobilized, the motor system may be knocked out. In this case, a person can be conscious. Are destroyed areas of the brain
It turns out that we see two aspects of the new approach. If we take an electric signal and make a command system out of it, then we make a neural interface. On the one hand, the brain, and on the other hand, some software technical devices that allow the electrical signal of the brain to appear in the information, in the motor or in the managerial sense. That is, we gain new access to the brain. And the brain gets a new exit into the world. The development of this channel is the topic for one of the most modern sections of neurophysiology and psychophysiology. You could even say psychology. The topic we are talking about is now defined by a broad term. An interface is the brain of a computer, neural interfaces. ”
The idea of using an electric brain signal as a channel for communication arose long ago, but researchers did not immediately understand the potential of such an initiative. The only problem is that this channel is slow. To manage something, you need to perform actions immediately after the emergence of thought. It was not possible to make such a fast channel at that time. A request for such technologies has been formed for a long time - most of all they were required in the clinical field, when the patient has a brain, but his motor skills are impaired. Another catch is that such devices must be portable so that a person does not depend on a huge structure. Therefore, a technical request for a microchip, which registers the electrical activity of the brain, for a microprocessor that processes and decrypts, and finally, for wireless communication, appeared here. I.e,
AK: “The key new question in this area was to understand whether a person can arbitrarily change his electrical activity. It changes naturally due to the action of the internal mechanisms of the brain. But if I myself or someone can change my electrical activity, it means that in fact a communication channel is already emerging. And now it’s necessary to decipher not the signs of diseases, but the signs of volitional action: “I want to change the electrical activity of the brain in a certain direction.” A person needs to learn how to make several movements in this electroencephalogram. It will be a few letters or several teams. And then we will agree. For example, changing “A” would mean turning on the TV. Change “B” - turn on the refrigerator and so on. Or you can type letters. But there are more letters. There can be many characters in the alphabet or on the control panel. Is it possible to achieve this so that many gradations of changes in one's electrical activity can be obtained? This is the problem that scientists are busy today.
I must say that no one succeeds in changing their electrical activity so easily. Something needs to be imagined so that it has a good response in the change in electrical activity. For example, I represent an orange. Unfortunately, it is not possible to identify a good clear change in the electrical activity associated with my thinking about the orange. What to do? It turned out that only bodily images give a good response. For example, I want to squeeze my right hand or left hand. I imagine that I am clutching my right hand. In fact, this does not occur as a physical phenomenon. But in my view, there is already a squeezing of the right hand. It can be decrypted.
It turns out that I really can arbitrarily let know in the encephalogram what I want to do. So we’ll agree in advance that I’m squeezing my right hand - this means some kind of specific team. But these are just a few teams.
We are now talking about a key achievement after opening the encephalogram and after deciphering the diagnostic signs of the disease. Now there are signs of volitional effort in the encephalogram itself. There are 4-6 of them. But how can you type letters? Scientists have gone the other way. This is the development of this principle - how to get signs of strong-willed intentions?
Scientists did not demand from a person more imagery or more images that he can imagine. They took a different path so that the images represented by a person would respond in the electrical activity of the brain. They presented these images outside. Let's say there will be a matrix, in each cell of which some symbols, letters will be drawn. Each cell starts flashing at different times. In the continuous recording of the electrical activity of the brain, one can catch the reactions to the wink of each symbol.
The find of the researchers is that if I conceived a symbol in myself, the reaction to this symbol will be slightly different. Relatively speaking. It will be strengthened. And there is nothing unnatural in this. I want this symbol. So there must be some other reaction. And the reactions to all other characters will be approximately the same.
Our task is to catch an unusual reaction very quickly. If we catch it, then we understand what kind of symbol we are talking about. If it is a letter, print this letter. So, letter by letter, a person shifts his attention from one character to another. And now another symbol gives a big reaction a little different. Thus, we get the intentional responses of the brain to the volitional effort of a person. Thus, it turns out that in one way or another the volitional intention, volitional impulse of a person to this or that external action can be obtained behind the scenes, simply by registering the electrical activity of the brain. This, in fact, is the interface-brain-computer technology. "
With the help of decryption and additional devices, you can understand the intentions of a person, but for this you need to agree in advance with him about what each change in the electrical activity of the brain means. Although a person’s thoughts cannot be read in this way, because we didn’t agree in advance what these or those changes would mean.
AK: “There is a double transformation. First, we agree with the person about what meaning this or that imagination will have (to squeeze the right hand or left hand or attention to one or another image that is given from the outside - a letter, symbol, picture). That is, first we agree on what action will be performed outside. Say, if the picture is a museum, then let different museums of the world open on the screen. But then you need to pay attention to a particular museum, the following picture will open.
The key here is that we must correctly guess the willful intentions of a person previously agreed upon. Here is the decryption question. Of course, it is not completely reliable as decryption of any code. There is always some kind of error. Scientists are fighting to make this error less.
If we talk about our laboratory, then if we print letters by volitional efforts, then we give three to five percent of the error. That is, a person just sat down at the apparatus, for example. He put the electrodes. They put a computer in front of him. It will print with an error of no more than 5%. If it is practiced, some people at us one hundred percent faultlessly print for a long time. This can be achieved. The problem is not inaccuracy. The problem is speed. How quickly can a person’s intentions be deciphered? This delay determines the speed of interaction with the outside world.
If we type letters, then this is only 15 letters per minute. This is the highest speed that we achieve in our laboratory. This is much less than if a person who could not type on the keyboard typed letters with two fingers. That would be 90 letters per minute. And here is such a low speed. "
Actually, such an experiment, Alexander and his assistant conducted with my colleague Olya - they put a special hat on her head - a kind of horror helmet, and connected 8 electrons. Flickering triangles that rotated in a circle were displayed on the screen. After that, as Olga went through a short training course and the machine synchronized her processes with her brain, then, choosing the color and number that designates it, she thought about him, and he appeared on the screen.
In all technologies, interface-brain-computer, psychophysiology is a key science. This science has methods for analyzing the activity of the nervous system, and neurophysiology is responsible for the mechanisms of the nervous system. She explains how the brain works, how nerve cells are arranged, how they generate electrical signals.
When scientists try to catch a person’s reactions to some external stimuli or to representations of internal images, this is already the prerogative of psychology. The brain itself is neurophysiology, and the content of the brain is already psychology, images, our ideas about the outside world, our volitional efforts. But we need to cross this, because we must catch these psychic phenomena in the responses of the neuropsychological substrate, that is, in the electrical vibrations of the brain. That is, at the intersection of physiology and psychology, neurophysiology is born.
Physiologists and psychologists, respectively, work in this area, while experts of about three disciplines adjoin them.
1. Bioengineering. These are people who can design devices, for example, recording biopotential. Of course, we can say that this is a regular electronic engineer. But still, there is great specificity in registering biological potentials, and especially brain potentials, although they are very small. When a signal is caught it needs to be decrypted, and this is a big difficulty.
2. Mathematics. The decoding of the signal is already the prerogative of mathematicians. Although the mathematician himself cannot decrypt, he needs to be given hints. This is done by neurophysiologists.
3. Programming. If there is an engineering part: a good signal, understanding how to decrypt, you need to program this. That is, you need to make reliable good software that will scroll through all these calculations on some kind of computing tool. So, we need programmers and not simple ones. They need to understand things like speed of action. “It’s not necessary to program anyhow, but to select configurations (such libraries) that will do it quickly. Then they need to be done on the microcontroller. And not on a regular large computer. It must be an interdisciplinary team. We work with such a team. ”
In the future of neural interface technology, there are two branches that may overlap. The first direction is medical. New technologies are needed for people who are physically unable to work and cannot maintain normal communication, that is, they completely fell out of society. It is important in this area to find specialists who will focus on the installation and maintenance of these devices, adaptation to each specific category of patients.
The second direction is neurocybernetics.
AK: “Now I have the neurocybernetics of real devices, when the brain will communicate fairly tightly with computer technology. And this is real neurocybernetics, because feedbacks should appear. For example, we want to control a manipulator. We have a mockup manipulator with six motors. We, by the power of thought or our volitional effort, must somehow control these six motors so that he stretches out his arrow, takes something with his claws and moves it to another place.
Here, some kind of internal own intelligence must be contained in the form of a formula for the sequence of turning on the motors. And I just have to give a mental command "forward". And motors turn on sequentially. We must interact with this manipulator. He must understand what I want from him, and organize this action inside his motors. This is the real interaction between the brain and the external device. These devices must be cybernetic. That is, do not just turn on the TV. There is no cybernetics. And this is our future - this is when we get truly controllable devices that work in conjunction with our brains. And this is absolutely not the idea that profess in science fiction films such as "Robocops", when a person is simply immersed in a technical tool and can not live separately. Not, these are devices such as a mobile phone. I can pull it out of my pocket and throw it away. I can live without him. But some restrictions are imposed. However, they can even be an advantage. So it is here. ”
The last area that concerns neurointerfaces is the philosophical-humanistic problem. Actually, this is a brain life problem. Increasingly, we are faced with a situation where the brain does not die of its own free will, but because the organs of the body that ensure its life fail.
AK: “Of course, the problem arises of the continuation of the life of an individual. Because the main thing that is contained in our personality is the maintenance of the brain by the body. The body contributes. But memory, our memories, are the brain. The problem of preserving personality is the problem of preserving the communication skills of the brain with external devices.
After all, if we control the manipulator, being in a healthy body, the brain also controls it. If the body is unhealthy (for example, complete paralysis of the body), a person can control the manipulator. Now it all works in the United States: in several laboratories such manipulators are made that are controlled by a completely immobilized person who cannot even speak. In this case, the manipulator can take cookies, treat you. And this is the will of man. He cannot say anything, nothing moves with him, but he treated you. And he can just as well type letters. Therefore, what to do in such a situation? The body is with him, because the organs that are in this body are healthy. Not only the motor system is healthy. And if a person has a broken heart or liver that cannot be replaced? Can one then go on to make a complete prosthesis of the body? That is a prosthesis, which provides the brain with life, but it will no longer be biological? The biological is the brain. How does he interact with the outside world? Neurointerface. But this is a problem that requires an ethical philosophical and humanistic understanding. ”
All of the above, as well as an experiment in video format here.
What Kaplan is engaged in can be called the development of a new paradigm in psychophysiology. Until recently, scientists studied the brain in terms of its electrical activity, since neurons or the brain work on electricity. Nerve cells communicate with each other, transmit information to each other using current pulses or what is called spikes. And so the idea came up to take an electric signal and make a command out of it to control external devices. This was made possible thanks to electroencephalography, which was previously used to diagnose diseases. This method consists in the fact that the electromagnetic field from the electrical life of cells breaks through the skull to the surface of the head onto electrodes that are pressed against the skin surface.
But the conversion of this electrical signal into an instrument for controlling external technology is not a toy at all. The information about the outside world that we store and synthesize in ourselves, we manifest outward with the help of muscles, this is our only way, but can everything be transmitted through thoughts?
Alexander Kaplan: “This is an organic channel through which you can express the inner world, which is created at many entrances. If there was any direct channel for delivering information from the brain, we could at least compare something. Or to develop a philosophical neurophysiological problem. We say now that such a channel can appear if electric signals are used as an engine. But this is a more general aspect, and there is also a pragmatic aspect. If we receive a control signal directly from the brain, then in this way a person can directly say something without using muscles. Exactly this situation occurs in a number of diseases. In particular, with a stroke. After a stroke, people may be immobilized, the motor system may be knocked out. In this case, a person can be conscious. Are destroyed areas of the brain
It turns out that we see two aspects of the new approach. If we take an electric signal and make a command system out of it, then we make a neural interface. On the one hand, the brain, and on the other hand, some software technical devices that allow the electrical signal of the brain to appear in the information, in the motor or in the managerial sense. That is, we gain new access to the brain. And the brain gets a new exit into the world. The development of this channel is the topic for one of the most modern sections of neurophysiology and psychophysiology. You could even say psychology. The topic we are talking about is now defined by a broad term. An interface is the brain of a computer, neural interfaces. ”
The idea of using an electric brain signal as a channel for communication arose long ago, but researchers did not immediately understand the potential of such an initiative. The only problem is that this channel is slow. To manage something, you need to perform actions immediately after the emergence of thought. It was not possible to make such a fast channel at that time. A request for such technologies has been formed for a long time - most of all they were required in the clinical field, when the patient has a brain, but his motor skills are impaired. Another catch is that such devices must be portable so that a person does not depend on a huge structure. Therefore, a technical request for a microchip, which registers the electrical activity of the brain, for a microprocessor that processes and decrypts, and finally, for wireless communication, appeared here. I.e,
AK: “The key new question in this area was to understand whether a person can arbitrarily change his electrical activity. It changes naturally due to the action of the internal mechanisms of the brain. But if I myself or someone can change my electrical activity, it means that in fact a communication channel is already emerging. And now it’s necessary to decipher not the signs of diseases, but the signs of volitional action: “I want to change the electrical activity of the brain in a certain direction.” A person needs to learn how to make several movements in this electroencephalogram. It will be a few letters or several teams. And then we will agree. For example, changing “A” would mean turning on the TV. Change “B” - turn on the refrigerator and so on. Or you can type letters. But there are more letters. There can be many characters in the alphabet or on the control panel. Is it possible to achieve this so that many gradations of changes in one's electrical activity can be obtained? This is the problem that scientists are busy today.
I must say that no one succeeds in changing their electrical activity so easily. Something needs to be imagined so that it has a good response in the change in electrical activity. For example, I represent an orange. Unfortunately, it is not possible to identify a good clear change in the electrical activity associated with my thinking about the orange. What to do? It turned out that only bodily images give a good response. For example, I want to squeeze my right hand or left hand. I imagine that I am clutching my right hand. In fact, this does not occur as a physical phenomenon. But in my view, there is already a squeezing of the right hand. It can be decrypted.
It turns out that I really can arbitrarily let know in the encephalogram what I want to do. So we’ll agree in advance that I’m squeezing my right hand - this means some kind of specific team. But these are just a few teams.
We are now talking about a key achievement after opening the encephalogram and after deciphering the diagnostic signs of the disease. Now there are signs of volitional effort in the encephalogram itself. There are 4-6 of them. But how can you type letters? Scientists have gone the other way. This is the development of this principle - how to get signs of strong-willed intentions?
Scientists did not demand from a person more imagery or more images that he can imagine. They took a different path so that the images represented by a person would respond in the electrical activity of the brain. They presented these images outside. Let's say there will be a matrix, in each cell of which some symbols, letters will be drawn. Each cell starts flashing at different times. In the continuous recording of the electrical activity of the brain, one can catch the reactions to the wink of each symbol.
The find of the researchers is that if I conceived a symbol in myself, the reaction to this symbol will be slightly different. Relatively speaking. It will be strengthened. And there is nothing unnatural in this. I want this symbol. So there must be some other reaction. And the reactions to all other characters will be approximately the same.
Our task is to catch an unusual reaction very quickly. If we catch it, then we understand what kind of symbol we are talking about. If it is a letter, print this letter. So, letter by letter, a person shifts his attention from one character to another. And now another symbol gives a big reaction a little different. Thus, we get the intentional responses of the brain to the volitional effort of a person. Thus, it turns out that in one way or another the volitional intention, volitional impulse of a person to this or that external action can be obtained behind the scenes, simply by registering the electrical activity of the brain. This, in fact, is the interface-brain-computer technology. "
Is it possible to read a person’s thoughts?
With the help of decryption and additional devices, you can understand the intentions of a person, but for this you need to agree in advance with him about what each change in the electrical activity of the brain means. Although a person’s thoughts cannot be read in this way, because we didn’t agree in advance what these or those changes would mean.
AK: “There is a double transformation. First, we agree with the person about what meaning this or that imagination will have (to squeeze the right hand or left hand or attention to one or another image that is given from the outside - a letter, symbol, picture). That is, first we agree on what action will be performed outside. Say, if the picture is a museum, then let different museums of the world open on the screen. But then you need to pay attention to a particular museum, the following picture will open.
The key here is that we must correctly guess the willful intentions of a person previously agreed upon. Here is the decryption question. Of course, it is not completely reliable as decryption of any code. There is always some kind of error. Scientists are fighting to make this error less.
If we talk about our laboratory, then if we print letters by volitional efforts, then we give three to five percent of the error. That is, a person just sat down at the apparatus, for example. He put the electrodes. They put a computer in front of him. It will print with an error of no more than 5%. If it is practiced, some people at us one hundred percent faultlessly print for a long time. This can be achieved. The problem is not inaccuracy. The problem is speed. How quickly can a person’s intentions be deciphered? This delay determines the speed of interaction with the outside world.
If we type letters, then this is only 15 letters per minute. This is the highest speed that we achieve in our laboratory. This is much less than if a person who could not type on the keyboard typed letters with two fingers. That would be 90 letters per minute. And here is such a low speed. "
Actually, such an experiment, Alexander and his assistant conducted with my colleague Olya - they put a special hat on her head - a kind of horror helmet, and connected 8 electrons. Flickering triangles that rotated in a circle were displayed on the screen. After that, as Olga went through a short training course and the machine synchronized her processes with her brain, then, choosing the color and number that designates it, she thought about him, and he appeared on the screen.
What and who
In all technologies, interface-brain-computer, psychophysiology is a key science. This science has methods for analyzing the activity of the nervous system, and neurophysiology is responsible for the mechanisms of the nervous system. She explains how the brain works, how nerve cells are arranged, how they generate electrical signals.
When scientists try to catch a person’s reactions to some external stimuli or to representations of internal images, this is already the prerogative of psychology. The brain itself is neurophysiology, and the content of the brain is already psychology, images, our ideas about the outside world, our volitional efforts. But we need to cross this, because we must catch these psychic phenomena in the responses of the neuropsychological substrate, that is, in the electrical vibrations of the brain. That is, at the intersection of physiology and psychology, neurophysiology is born.
Physiologists and psychologists, respectively, work in this area, while experts of about three disciplines adjoin them.
1. Bioengineering. These are people who can design devices, for example, recording biopotential. Of course, we can say that this is a regular electronic engineer. But still, there is great specificity in registering biological potentials, and especially brain potentials, although they are very small. When a signal is caught it needs to be decrypted, and this is a big difficulty.
2. Mathematics. The decoding of the signal is already the prerogative of mathematicians. Although the mathematician himself cannot decrypt, he needs to be given hints. This is done by neurophysiologists.
3. Programming. If there is an engineering part: a good signal, understanding how to decrypt, you need to program this. That is, you need to make reliable good software that will scroll through all these calculations on some kind of computing tool. So, we need programmers and not simple ones. They need to understand things like speed of action. “It’s not necessary to program anyhow, but to select configurations (such libraries) that will do it quickly. Then they need to be done on the microcontroller. And not on a regular large computer. It must be an interdisciplinary team. We work with such a team. ”
Future
In the future of neural interface technology, there are two branches that may overlap. The first direction is medical. New technologies are needed for people who are physically unable to work and cannot maintain normal communication, that is, they completely fell out of society. It is important in this area to find specialists who will focus on the installation and maintenance of these devices, adaptation to each specific category of patients.
The second direction is neurocybernetics.
AK: “Now I have the neurocybernetics of real devices, when the brain will communicate fairly tightly with computer technology. And this is real neurocybernetics, because feedbacks should appear. For example, we want to control a manipulator. We have a mockup manipulator with six motors. We, by the power of thought or our volitional effort, must somehow control these six motors so that he stretches out his arrow, takes something with his claws and moves it to another place.
Here, some kind of internal own intelligence must be contained in the form of a formula for the sequence of turning on the motors. And I just have to give a mental command "forward". And motors turn on sequentially. We must interact with this manipulator. He must understand what I want from him, and organize this action inside his motors. This is the real interaction between the brain and the external device. These devices must be cybernetic. That is, do not just turn on the TV. There is no cybernetics. And this is our future - this is when we get truly controllable devices that work in conjunction with our brains. And this is absolutely not the idea that profess in science fiction films such as "Robocops", when a person is simply immersed in a technical tool and can not live separately. Not, these are devices such as a mobile phone. I can pull it out of my pocket and throw it away. I can live without him. But some restrictions are imposed. However, they can even be an advantage. So it is here. ”
Ethics issue
The last area that concerns neurointerfaces is the philosophical-humanistic problem. Actually, this is a brain life problem. Increasingly, we are faced with a situation where the brain does not die of its own free will, but because the organs of the body that ensure its life fail.
AK: “Of course, the problem arises of the continuation of the life of an individual. Because the main thing that is contained in our personality is the maintenance of the brain by the body. The body contributes. But memory, our memories, are the brain. The problem of preserving personality is the problem of preserving the communication skills of the brain with external devices.
After all, if we control the manipulator, being in a healthy body, the brain also controls it. If the body is unhealthy (for example, complete paralysis of the body), a person can control the manipulator. Now it all works in the United States: in several laboratories such manipulators are made that are controlled by a completely immobilized person who cannot even speak. In this case, the manipulator can take cookies, treat you. And this is the will of man. He cannot say anything, nothing moves with him, but he treated you. And he can just as well type letters. Therefore, what to do in such a situation? The body is with him, because the organs that are in this body are healthy. Not only the motor system is healthy. And if a person has a broken heart or liver that cannot be replaced? Can one then go on to make a complete prosthesis of the body? That is a prosthesis, which provides the brain with life, but it will no longer be biological? The biological is the brain. How does he interact with the outside world? Neurointerface. But this is a problem that requires an ethical philosophical and humanistic understanding. ”
All of the above, as well as an experiment in video format here.