Neuro interfaces: from photo paper to neural dust

A person does all interaction with the outside world with the help of muscles (speech, fingers, gestures, etc.). The neurointerface allows you to rule interact with the world without muscle activity. The first step to the brain in the bank. And to breaking the brain.

In the movie “Chappy”, with the help of an EEG helmet, the robot copied consciousness (both ours and human), and DARPA meanwhile taught a paralyzed woman not only to eat chocolates, but also to control an inconspicuous fifth-generation fighter-bomber F-35. On the simulator.

We learned to send audio and video signals directly to the brain several decades ago. Now in secretMoscow State University’s laboratories teach people not only to type in “thought” (13-15 characters per minute), but also to set up a hidden “subconscious-computer” interface, and there are amateur devices from 10,000 rubles on sale ( NeuroSky , freely available in Hackspace ) and quite professional for 200,000 rubles ( BioRadio , I tested in St. Petersburg)



We will talk about how we got to such a life (about the history of EEG and neurointerfaces). (And also about the first attempts to use neural interfaces for information security).

While writing this post, I realized that Ghost in the shell (the entire series, including the manga) is the most visual and most hard-fix-predictionthe near future. There, hacking of eye prostheses (including spoofing of the visual flow), hacking of a cyber brain, and falsification of memories.

Over the past 10 years, it’s just an avalanche of discoveries - they look into dreams, talk with people in a coma, control the mouse brain using a laser, transmit a signal from the brain to the brain via the Internet (1 pixel), control the finger of another person with the help of thought through the Internet, create a channel computer-subconscious communications, use the human brain as a peripheral sensor for the perimeter security system, create an exoskeleton with brain control for the paralyzed, etc. And how did it all begin?

1875 : Richard Caton discovered the presence of electrical signals on the surface of the brain of animals

1924 :Hans Berger was able to fix with a galvanometer on paper in the form of a curve electrical signals from the surface of the head (and not directly from the brain itself, as before it), generated by the brain.

1929 : Hans Berger published his first work describing experiments with human EEG.

Alpha waves of brain activity, having a frequency of 8-12 Hz, were called Berger waves. He studied the electrical activity of the brain (primarily in terms of amplitude parameters) under various conditions: in a calm state, when solving problems, during anesthesia. In his book, Psyche (Jena, 1940) addressed the solution to the problem of extrasensory perception, considering the possibilities of the microwave model to explain this phenomenon and pointing to its insufficiency.

1932: German engineer Jan Friedrich Toennies designed the first EEG recorder to print with ink on plain paper. (Before that there was photo paper)

1940s : Wilder Penfieldused the information obtained during hundreds of operations on the brain to create functional maps of the cortex (surface) of the brain. He summarized the results of cartography of the main motor and sensory areas of the cortex and for the first time accurately mapped the cortical areas relating to speech. Using the method of electrical stimulation of individual parts of the brain, Penfield established the exact representation of various muscles and organs of the human body in the cerebral cortex. Schematically, he is depicted in the form of a “homunculus” (man), whose body parts are proportional to the zones of the brain in which they are represented. Therefore, fingers, lips and tongue with a large number of nerve endings are depicted larger than the trunk and legs.

1950s : Jose Manuel Rodriguez Delgado, professor of physiology at Yale University, invented the Stimoceiver - a scientist developed radiofrequency stimosivers the size of a fifty-cent coin that could be fully implanted and controlled via an FM radio channel


The most spectacular experiment of Delgado was carried out in 1963 on a ranch in the province of Cordoba , Spain. Having implanted stimosivers into the brain of several “fighting” bulls, he got the opportunity to control all their movements using a portable transmitter. An amazing photograph has been preserved, depicting how Delgado made the attacking bull stop dead in his tracks only a few feet away, including the stimulation of the caudate nucleus.

However, according to the scientist, his experiment with a female chimpanzee Paddy deserves much more attention. The animal was implanted with a stimosiver programmed to receive distinct signals called spindles that spontaneously occur in the amygdala. As soon as the device caught the spindle, it stimulated a region of central gray matter in the Paddy’s brain, causing an “aversive reaction”, that is, a painful or unpleasant sensation. After two hours of such exposure, the tonsil of the monkey gave out 50% fewer spindles, in six days the frequency of their occurrence fell by 99%. This is not to say that Paddy benefited: she became “calmer, less attentive and less motivated when testing her behavior,” Delgado wrote. However, he suggested that a similar technique of “ automated learning”»Could be used to suppress epileptic seizures, panic attacks or in the treatment of other diseases accompanied by the appearance of characteristic signals in the brain.


A macaque female (far left on the first photo) quickly realized that with the help of a lever it is possible to calm her cell neighbor - a cocky dominant male. The lever sent a signal to the stimosiver in his brain, and anger passed. The far right on the left photo is a pacified male. In another photograph, he again became aggressive. In the early 60s. Delgado has conducted many such studies, studying the effects of brain stimulation on social relationships.

1960s : Gray Walter(inventor of the turtle robot), which discovered the delta and theta rhythms of the brain, connected the electrodes to the brain and made the patient move his finger. He invented the toposcope.

1960: Elwood Henneman and colleagues found that to “activate” motor neurons of the smallest diameter, a weaker electrical signal is required than for neurons with a larger diameter. The more neurons are recruited by the brain to perform an action, the more effort a muscle can make.

1960s : Neil Miillerin the early 60s, following the traces of experiments, James Olds (1956) taught rats to apply electrical stimulation to the center of pleasure, changing the parameters of any autonomic function (peristalsis, heart rate) and even the parameters of the EEG that “connected” to the stimulator. rat, if brought to this, is able to control the blood pressure in the caudal artery. To do this, you just need to attach a pressure sensor to her tail and give food when the pressure exceeds a certain value. Hunger is a powerful incentive for the rat brain to learn how to manage such a vital indicator that it does not have to control in ordinary life.



1967 psychiatrist Edmond M. Dewan published an article in the journal Nature about an experiment whose members were asked to focus on particular Morse code symbols. While they were doing this, an electroencephalogram recorded the electrical activity of their brain. At moments of concentration on the symbols, the electroencephalogram changed. After a short training, the subjects were able to convey whole words to the equipment using Morse code. It was not clear which neurons and brain waves were involved during the experiment, but it worked - for the first time, a person was able to send his thought to a machine and be understood by it.


1968 : Joe Kamiya demonstrates that a person can regulate the rhythms of his brain - to strengthen and weaken the alpha rhythm. (Psychology Today, 1968, 1, 56-60.)

1968 Barry Sherman discovered the SMR rhythm at 12-15 Hz in the sensorimotor cortex. Rhythm is associated with calm in the body, readiness to perform motor actions, with an increased concentration of attention to the outside world, and improved memory.

The first experiments were carried out on cats. Cats were trained to increase the amplitude of brain waves in the range of 12-15 Hz. Later, the scientist received a NASA order to study the effect of rocket oxidizer toxin (hydrazine) on animals. In a group of 50 cats, ten trained SMR amplitudes. To his surprise, cats with increased SMR showed resistance to toxin. Cats with normal SMR levels died from exposure to the substance.

In the 70s, Sterman proposed training SMR to treat epilepsy. His employee Margarte Fairbanks suffered from bouts of epilepsy and couldn’t get a driver’s license. She decided to try out the training earlier conducted only on animals. Of course, not right away, but the symptoms of the disease began to recede. When the course was completed. She received a driver’s license.

Later, other scientists also obtained positive results. " At the same time, Eberhard Fetz forced the monkey to get food by changing the activity of a single neuron of the cerebral cortex. He linked the activity of the first neuron that came to the electrode with the supply of juice, and the monkey’s brain learned optionally incorporate this neuron

1972: Cochlear implant went on sale. A device that converts sound into an electrical signal that is fed into the auditory nerve, and the brain learns to recognize these electrical signals as sound. Now about 25,000 people use such implants.

A cochlear implant is a microphone with a transmitter that transmits sound signals at radio frequencies to the inside of the device installed in the cochlea of ​​the ear, that is, to the implant itself. The sound signal in this case turns into electrical impulses, which are transmitted to the auditory neurons and sent further to the cerebral cortex.

1973: Jacques Vidal first used the term “brain-computer interface” in Toward Direct Brain-Computer Communication [pdf] . The scientist also formulated two fundamental questions:

• Is it possible to use data on the electrical activity of the brain as a storage medium when a person communicates with a computer?
• Can they be used to control various mechanisms - from the prosthesis to the spaceship?

1978 : First implant providing a video channel directly into the visual cortex. (Video camera-computer-brain)
Consists of a b / w video camera (292 x 512-pixel CCD, 69 ° viewing angle), an ultrasonic rangefinder, an intracranial implant (68 platinum electrodes in the visual cortex) and a computer (5 kg).

William Dobelle implanted a visual prosthesis to a 62 year old patient. The implant produces a black and white image of the “phosphenes” of the visual cortex, similar to the images projected onto the bulbs of stadium displays.
Upon stimulation, each electrode produces 1-4 closely spaced phosphenes. Each phosphene in a cluster perceives a region with a diameter of a pencil located at arm's length. Dr. Dobell's working group determined that in this case, the patient’s phosphenic map covers an area measuring about 20 cm high and 7.5 cm wide.


Patient System, Device Diagram, and Electrode Diagram

Dr. Dobelle's system provides weak near-central tunnel vision. The picture that the patient sees is black and white with a defect in the scattered field (caused by gaps between the phosphenes); depth perception is absent.
Thanks to the ultrasonic range finder, it was possible to transmit additional information to the patient (modulation of the image brightness, blinking frequency and feature of individual phosphenes)

The frame refresh rate is 1 to 5 per second. The operation is already being carried out commercially and is valued at approximately $ 120K. This technology allows you to return vision only to people who lost it as a result of accidents - that is, those who already "knew how to see" before.

Read more in English “Artificial Vision for the Blind by Connecting a Television Camera to the Visual Cortex”

1988 : “virtual keyboard” by Farvel and Donchin. Thanks to this method, people can type in the text, mentally choosing the desired letter at the intersection of character strings and rows.

Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials.

1998:Researcher Phillip Kennedy implanted the first BCI (Human Computer Interface) into Johnny Ray, a patient who lost mobility after a stroke. As a result of implantation, Ray learned to move the cursor.

2004: 25-year-old Matthew Neagle Using an implanted device, he got the opportunity to control the cursor on the screen, read e-mail, play simple video games, and even draw something. He also learned to switch channels and TV volume and move his electromechanical hand

a) An electrode matrix on a one-cent coin and a plug inserted into the skull. b) An array of one hundred electrodes. c) Array location. d) First patient with interface (Leigh R. Hochberg, Mijail D. Serruya, Gerhard M. Friehs, Jon A. Mukand, Maryam Saleh, Abraham H. Caplan, Almut Branner, David Chen, Richard D. Penn, John P. Donoghue, 2006)


2008: The first commercial / consumer neural interface in the market.

Then it started (the number of articles on the brain-computer interface).


It is interesting that there are authentication methods based on ECG and thought passport based on EEG

Heart and brain as documents

ECG
Bionym is a Canadian company developing a unique biometric authentication system. The company's main project is Nymi, a heart rate sensor bracelet designed for three-factor authentication.


The Verge article in English (2013)

EEG

Thought Passport It
will be enough for users to think about some thing, and the system will automatically put them into a program or service.

Brain waves are unique to each person, so even if someone knew your “passthought”, their signal will still be different. After a series of tests, the participants in the experiment completed seven different mental tasks with the device. The simplest actions, such as focusing on breathing or some kind of thought, for ten seconds made it easy to carry out successful authentication in a computer system. The main thing here is to find a task that would not strain users so that they can carry out it several times a day. For example, count objects of a certain color or hum a song to yourself.

Article in Wired (2013) A
scientific article from Berkeley " I Think, Therefore I Am: Usability and Security of Authentication Using Brainwaves "

The idea was first proposed in 2007 in the article Person Authentication using Brainwaves (EEG) and Maximum A Posteriori Model Adaptation

WIRED in 2009 puzzled by this question: The Next Hacking Frontier: Your Brain?

There are also Russian developments: User identification based on electroencephalography using brain-computer interface technologies.

Secret Information Leak Channel - Your Brain
On the Feasibility of Side-Channel Attacks with Brain-Computer Interfaces

Publication
Article in WIRED Bruce Schneier's
response : “This is a new development in spyware.”

A little about “brain fingerprints” including to catch terrorists

How can you develop now?

NeuroSky $ 100 (available in Moscow Hackspace and for sale in Russian online stores)
OpenBCI $ 200 (ordered from America / Europe)
Bioradio 200-300 thousand rubles (available in St. Petersburg, sometimes comes to exhibitions in Moscow)

BioRadio
BioRadio - device which can be used in neuroscience / marketing / economics / ethics, usability laboratories, for controlling robots / cars / spiders / helicopters, for medical research and diagnostics. For the price of a 3d printer. The size of a cassette player.


This gizmo is a full-fledged psychophysiological laboratory, which allows you to control the robot arm (both signals from the EEG and the myogram), track heart rate, plethysmogram (change in lung volume), skin resistance, etc.

The main feature of Bioradio is that it fits in my palm

while the full set of MGUShny EEG that I experimented with is placed in the trunk of a car.

There is a BioRadio API

Near future

Already on the approach 2 technologies - neuropol and tattoo.

Neuro dust
A group from the University of California at Berkeley has proposed a way to reduce the size of implantable elements to a few micrometers and literally fill them with the choroid.

Particle arrangement “neural dusts"

The ultra-miniature electronic sensors developed by them consist of a microcircuit made using CMOS technology, a piezocrystal, electrodes and an insulating polymer shell. The principle of their operation resembles the practice of using radio frequency identification (RFID) chips that do not require a built-in power supply.

According to the authors, neuro dust particles circulate freely in the bloodstream. In practice, this is difficult to achieve because of the complex composition of the blood, the biological mechanisms of its purification, and the structure of the endothelium, but imagine for a moment that these problems have been resolved. Then the simultaneous number of microsensors in the vessels of the brain at any given time can amount to thousands.

Each of these “smart particles” will be able to measure the electrical activity of nearby neurons. During the first phase, the piezoelectric crystal converts the ultrasonic waves from the intermediate module into electrical signals and feeds the CMOS circuit. During the second, he acts the other way around - vibrates under the influence of the action potentials of the nearest group of neurons.


Piezoelectric effect in the particle "neural dust"

Compared with RFID, the proposed scheme has at least two important differences. In addition to electromagnetic waves (external level), ultrasound (internal level) is used, and the intermediate module provides amplification of the response signal of microsensors, its conversion and further transmission.
The latter is located under the dura mater and acts as a transceiver, avoiding the strong attenuation of ultrasound and overcoming the shielding effect of the bones of the skull.

As applied to the “brain - computer” interfaces, to a first approximation, this is equivalent to increasing the accuracy of determining the mental command tenfold.
More on Computerra or in the article

Flexible smart “tattoos”
For the first time, scientists from the University of California at San Diego introduced the invention to the world in 2011.
The components are located on an area of ​​1x2 centimeter. Transistors, antenna, coils, temperature sensors, photo detectors, EEG, ECG and EMG sensors. Connecting contacts made of silicon and gallium arsenide.

Article

This high-tech “sandwich” has a thickness of 40 microns, and a fastener made of polyester is used for fasteners - which is also used on translated tattoos for children.

Currently, the tattoo sensor is able to continuously collect data for 6 hours and is on the body for up to one day, while the main task of the researchers now is to increase these indicators. At the same time, it is worth noting that despite the ready-made solution, the release date of this device for general use has not yet been announced.

In the meantime, marketing and hackers have taken up the neural interfaces (at the 31C3 Congress in 2014 there was a report on work towards hacking BCI):


PS Probably, antiviruses will be treated much more carefully in the future