Neural Interfaces Today

    image


    Since the invention of the mouse, half a century has passed, and this is still one of the main ways of human-computer interaction. I went to the conference at the Institute of Cognitive Neurosciences of HSE to learn about the latest achievements in the field of BCI, which is beyond the horizon, and therefore it is so interesting.


    The conference report I reworked into an article for related narration. Some moments I simplify and omit, and some I add from my observations and reports from other events. After reading it, I hope you will have a general understanding of the approaches to BCI and the current state in this area. For original interpretations, it is better to refer to original articles, fortunately, almost everything is publicly available.


    Story


    The history of BCI began in 1973 with the publication of Toward direct brain-computer communication [1], where Jacques Vidal outlined ideas in communication between man and machine and described the laboratory for analyzing the EEG signal for such purposes. A decade later, Wolpaw focused the use of BCI on helping paralyzed people and described the concept of BCI [2]:


    image


    The main BCI implementations made it possible for people to enter text with isolation syndrome . These were difficult to use systems, because the user had to go through a long training session [3], in contrast, there appeared “spellers” based on recognition of the P300 , a component that occurs at the moment when a person makes a choice, which reduced the requirements for the user [4] .
     
    In the 90s, the topic is becoming more and more known, especially with the advent of machine learning techniques [5]. With the increase in BCI reliability, people are interested in expanding applications to new areas.


    Thorsten Zander proposed the following BCI classification [6]:


    • Active BCI - the user initiates the command unconditionally
    • Reactive BCI - the user initiates a command in response to the impact of the system
    • Passive BCI - the user does not give a command, but the system reads and analyzes its state

    Separately, it is worthwhile to consider the issue of brain stimulation, this topic, although not directly related to BCI, is a related technology that expands the possibilities of BCI to control.
     
    BCI can also be classified by the method of receiving the signal:


    • Invasive (implanted electrodes, ECoG and others)
    • Non-invasive ( EEG , NIRS and others)

    EEG is the most common way to get a signal, so, unless otherwise indicated, I mean it by default.
     


    Active BCI 


    Basketparadigm


    This is a symbol of the possibility of control by activating imaginary movements. The fact is that the motor cortex is compactly located in the center of the head, therefore, imaginary movements of different parts of the body are well classified and used to construct the BCI. The user, to work with such BCI, must be mentally imagined as he makes movements with different parts of the body.


    image


    To facilitate experiments, scientists develop their own frameworks, for example, BCILAB . With his help, an experiment was conducted to demonstrate to the skeptic the possibility of controlling with imaginary movements. The result was 80% - so-so result in conditions where a person has familiar alternatives, but deserves a high mark, especially for an unprepared respondent [7].
     
    The same approach was used to control the horizon for the flight simulator. The results are ambiguous, for 3 respondents it was possible to achieve a result of 94%, for another 4 64% and less than 60% for three more. The success lies in the fact that the first trinity controlled the aircraft in the same way as the steering wheel does. The rest of the pilots did not focus enough on the internal state and made muscular movements, which made a negative contribution to the management.
     


    Rehabilitation systems


    BCI, which recognize motor commands, are well studied and are already being used to rehabilitate patients after a stroke: to restore the broken bonds necessary to control paralyzed limbs. Pavel Bobrov demonstrated the results of clinical trials of a rehabilitation complex to restore the motor functions of the hands, which proved the effectiveness of use. Moreover, there is a significant difference for patients who began rehabilitation after a month and 6 months after a stroke, the earlier rehabilitation begins, the better the effect. [11]
     
    Chapter g.tec Gunter Edlinger spoke about the work of special gyms for rehabilitation, an interesting point is that electrostimulation of the extremities was added to the rehabilitation process, and if the electro-mechanical installation was used above, then there is current stimulation, which reduces the cost of the complex.



     
    If we add elements of the game and competition to the process, the involvement will be higher, which means the patient will go through rehabilitation better. At the Center for Bioelectric Interfaces, the HSE, under the leadership of Alexey Osadchy, developed prototypes to improve the rehabilitation process. The video demonstrates a prototype system for two people, where they control a vessel, performing imaginary motor commands, trying to incline the vessel in their direction:



     
    Single player game:



     
    Or, for example, the handwriting recognition algorithm for muscle activity using a compact array of electrodes allows you to reconstruct what is written: [12]
    image
     
    The pinnacle in their work is work on BCI in the ExoAtlet project , which allows people with disabilities to move independently or use it for rehabilitation.



     
    Invasive BCI is a more complex subject, and now experiments are being conducted on animals or on people who have electrodes installed for medical reasons. A series of studies was highlighted, which showed that it is possible to determine not only individual components (meaning all the same fictional movements), but also to divide movement, attention, and the direction of gaze between them. A recording of a similar report from the conference in Samara is available.
     


    Reactive BCI


    image


    The classic example of reactive BCI is the “speller” on the P300 effect, this is the “wave” that appears in response to the choice of the stimulus shown, and in the “speller”, the stimulus is a certain way coded symbols of the alphabet or command. The user must mentally interact with the stimuli that the system displays — count the number of flashes of the selected symbol.


    Not to mention the project Neurochat , which allows people with disabilities to communicate:



    Passive BCI


    The basic idea of ​​passive BCI is an assessment of a person’s condition, for example, an assessment of cognitive load (workload), it can be applied in training systems, a study was conducted to solve this problem.


    The qualifier was trained on the following tasks:


    • For high loads, the respondent subtracted 1-2 digits from 3-4 figures, excluding simple options with dozens.
    • For a light load, they asked the respondent to focus on a pleasant memory.

    The accuracy of the algorithm was 70%. The classifier was tested on other tasks (multiplication, playing the scramble), and obtained similar accuracy, thereby confirming the fact that it is possible to make the classifier independent of the person and the tasks. [13]
    This idea can be applied to control the surgeon during the operation [14]. The task of determining the load on the surgeon during the execution of different on the complexity of the manipulations on the simulator was solved. The system has learned to determine how the surgeon performs the operation with high accuracy.
     
    Another option is to measure the degree of relaxation. On the basis of the state of the visitor of the interactive installation in the “ Museum of Silence ” a vivid picture was created that reflected its internal state. [15]


    image
      
    Passive BCIs can also be used for control tasks, a rather original approach is to give a person not direct control over the cursor, but only the right to judge whether the cursor is moving along the correct path to the goal. The experiment was conducted on small matrices of 4x4 and 6x6 pixels. At first, the system was trained on an arbitrary point motion, and the person’s task was to determine whether the point was moving in the right direction, then they were tested in live mode and the result was close to the optimal path. [16] You can see a demonstration .


    Midas touch problem and E-BCI interfaces


    Controlling the cursor with the help of the direction of sight is a simple task, which is solved with the help of eye-tracker'ing (also known as video-culography). But in these interfaces there are problems, for example, involuntary eye movements and the problem of choice, by the way, it is very symbolic that it is called the problem of the touch of Midas, the Phrygian king, any touch of which turned the object into gold. The applications of passive BCI allow you to solve these problems.


    The approach where active BCI was used to make choices when controlling with the eye-tracker is known for a long time, but does not differ in speed. The study, where respondents rated different methods of choice on the NASA TLX scale , showed that the BCI variant is not faster in time than the long fixation option for selecting an object, but the BCI causes a lower degree of frustration [10].
     
    Further work of the team of Torsten Zander showed that it is possible to distinguish conscious fixation on the object from the unconscious with an accuracy of 90% [17]. For the experiment, the paradigm "Oddball" was used - the respondent looked through a series of figures containing a figure that he wants to choose in combination with distracting figures.


    Sergey Shishkin spoke about the improvement of the above approach [8]. A significant plus of their solutions is a reduction in the speed of selection to 300 ms-500 ms, which requires very fast classification, for this purpose EEGNet was used [9].


    Attention mechanisms are a separate topic that can broaden the scope of BCI and create systems for rehabilitating patients with ADHD, Mehdi Ordikhani talks about the basic idea in his  Tedtalk



     


    Stimulation


    The question of the ethics of conducting experiments is very acute for neuroscience, and animals take the brunt of research for research beyond the limits. What if we want to affect a particular area in the depths of the brain? Now it is possible only with the help of implanted electrodes. But, for example, in nature there are creatures that are sensitive to a magnetic field, the Galit Pelled team from the University of Michigan isolated this gene in fish, introduced it to rats, and learned to control their behavior by the action of a magnetic field [18]. Thus, it is possible to have a targeted effect on the necessary areas, for example, to stop epileptic seizures.
     
    And a whole group of studies of invasive interfaces from Mikhail Lebedevon rhesus monkeys: a brain-computer-brain interface was built, which allowed, by controlling virtual extremities, to receive tactile feedback. You can take a closer look at the excerpt from the lecture " Interface between the brain and computer "
     


    Kingdom deep learning


    Besides the fact that the “deep learning” algorithms allow us to finish the already high precision of “machine learning”, it can be noted that people are working on the “inverse problem”. Based on the fast EEG and MEG data, you can try to restore the actual activation of neurons in the brain, which is now shown, for example, by the fMRI method, but with very low temporal resolution. One can only rejoice at optimism and believe in the speedy success of this work.
     
    Another BCI problem based on EEG or MEG is that the activity results in different areas of the brain for the same components differ among users, we have to teach the neural network for each user and the task, which complicates the work with the system and makes it more expensive. However, changes with the “transfer of training” are possible here, when the neural network uses data from different users / in different tasks and is trained online, as a result, the calibration step can be skipped. [nineteen]
     


    Hardware


    Finally, we got to the glands!


    Here it is important to say about 2 points, on the one hand, the equipment for BCI is rather cumbersome, the person in it attracts attention, in one of the presentations miniature electrodes were demonstrated, such that the person does not stand out in them. [20]


    imageimage


    Unfortunately, there is no possibility to insert a larger photo, but you can view a photo via Google .


    Despite all the miniature, to install these electrodes is not convenient, it is necessary to glue each individual electrode. For acceleration use various devices:


    • EEG caps, which are marked with holes for the electrodes
    • Hoops and helmets of various designs, where basically the position of the electrodes is fixed, only OpenBCI is allocated Ultracortex, in which the electrodes can be rearranged depending on the tasks.

    image


    A relatively fresh idea is the arrays from CeeGrid electrodes for attachment in the ear, which are both invisible and easy to install, but a significant disadvantage is the limited application, although there are works that show that it is realistic to use this option for BCI ERP [21] .


    image


    And the second problem is the need for a conductive gel for a high-quality signal, it is shown here that the differences are acceptable, and the use of dry electrodes is justified [22], but it all depends on the amount of hair. Over this issue, for example, recently Florida Research Instruments began to sell an elongated dry electrode (in the picture below it is on the left), which differs from the original version with a larger roundness of pins and, as you understand, causes less negative sensations among users. Even more advanced options are when the pins themselves on the electrodes are provided with shock absorption, thanks to the material or with the help of springs (in the picture below they are in the center and on the right).


    image


    Conclusion


    The spread of BCI to the masses will not be quick and easy, very limited possibilities are now open for understanding brain states, but progress in this area cannot be ignored. The main thing is that there is a correct tendency to reduce the cost of devices / provision of devices by subscription and the emergence of projects that are aimed at enthusiasts.


    Personally, I am very pleased that among the promoted Emotive, MUSE, OpenBCI Russian projects are beginning to appear. At the recent Neuroforum, which was held in St. Petersburg, were demonstrated:



    The expansion of available devices makes the interface area attractive for study and experimentation. The threshold of entry is low, you can always find an adequate task, and you can improve algorithms all the way, acquiring new knowledge and skills. What I wish for you.


    Such I saw the area of ​​BCI, let's see what will be interesting next year.


    Sources

    1 Toward direct brain-computer communication
    2 Brain–computer interfaces for communication and control
    3 A spelling device for the paralysed
    4 Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials
    5 Classifying Single Trial EEG: Towards Brain Computer Interfacing
    6 Towards passive Brain–Computer interfaces: applying Brain–Computer interface technology to human-machine systems in general
    7 Team PhyPA: Brain-computer interfacing for everyday human-computer interaction
    8 The Expectation Based Eye-Brain-Computer Interface: An Attempt of Online Test
    EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface
    EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
    10 Combining Eye Gaze Input With a Brain–Computer Interface for Touchless Human–Computer Interaction
    11 Роботизированные устройства в реабилитации после инсульта
    12 A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings
    13 Team PhyPA: Brain-Computer Interfacing for Everyday HumanComputer Interaction
    14 Automated Task Load Detection with Electroencephalography
    15 Passive Brain-Computer Interfacing in the Museum of Stillness
    16 Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity
    17 A Passive Brain-Computer Interface for Supporting Gaze-Based Human-Machine Interaction
    18 Wireless control of cellular function by activation of a novel protein responsive to electromagnetic fields
    19 Robust and highly adaptable brain-computer interface with convolutional net architecture based on a generative model of neuromagnetic measurements
    20 Miniaturized electroencephalographic scalp electrode for optimal wearing comfort
    21 Event-Related Potentials Measured From In and Around the Ear Electrodes Integrated in a Live Hearing Device for Monitoring Sound Perception
    22 A dry EEG-system for scientific research and brain–computer interfaces


    UPD corrected description of EBCI interfaces, by error they were assigned to active BCI, which is not true


    Also popular now: