In the wake of intelligence

    In the wake of intelligence


    Since the first attempts to simulate the processes in the human brain, science has gone through many steps bringing us closer to AI. But the human brain, for the time being, is doing an unbearable and poorly tracked job of continuously processing the flow of sensory information. I will try to meaningfully tell the main milestones of the evolution of artificial neural networks (ANNs).

    A person always wanted to simply understand how he thinks. And applied for such a task the standard methods of science: analysis, observation, experiments. But such a study yielded only descriptive knowledge, which did not reach the rigor of the physical sciences. I had to move away from the direct question “What do we think?” And find out, try to find the answer to the question “What does a person think?”. It turned out that the whole thing is in the brain, which consists of many of the same type of nerve cells - neurons. In addition, there were many of these cells, but there were even more connections between them. Modeling the experimentally observed properties of a neuron gave some repetition of the results of human thinking. It remained to connect this into the picture of the functioning of the entire nervous system of a person already at the level of physiology and psychology, but it was here that the problems began.

    The first ANN models are direct distribution networks without feedback. Their output was completely determined by the current values ​​of the sensors and coefficients (weights) at each input of each neuron. In addition, a teaching method with a teacher was used, when the scales of the inputs of the neurons were adjusted so that the network produced an allowable deviation from the required results for the training set of input values. Such a method was biologically implausible.

    The first to solve the problem of learning a network without a teacher. The basic principle of such training is that if the sending and receiving neurons are simultaneously active (output = 1), then the connection weight of these neurons increases. The biologically acceptable here is the logic of training a neuron within the boundaries of the neuron itself and the repetition of the connection (synapse) property of biological neurons. The neuron has become an integral logical self-organizing unit within the ANN.

    The second one solved the problem of the influence of sequences on the result of the ANN operation, first simply adding the output from the ANN to the incoming signal (Hopfield network), and then to the recurrent ANNs by adding connections from internal neurons (Elman network) or from the ANN output (Jordan network) with a single delay. Recursive connections at the input are usually called the context for the input data. Recursive ANNs can already be used in control systems of moving objects, as well as used to memorize sequences.

    The next property of memory was its simultaneous plasticity and stability. The principle of stability-plasticity is that new images are remembered in such a way that previously remembered ones are not modified or forgotten. All previous types of ANNs were not able to repeat such a property. The answer of science to this problem was adaptive resonance theory (ART). Its meaning boils down to the fact that if the network decides that the incoming set does not look like any one already remembered, then it adds one more exit from the network for a new image. If the network recognized a known set in the input set within the selected boundary of similarity, then the additional image is retrained within the network. The mismatch between stability-plasticity and ART is that there is still a change in the recognized set. A special sequence of input sets with a deviation less than the similarity boundary can completely retrain the ANN. But the addition of new outputs to the network, that is, new neurons, brings us closer to the repetition of biological neural networks; throughout life, new neurons form in humans. The areas of the brain that are responsible for new or constantly trained human skills are sealed with new neurons.

    The next problem was image recognition, regardless of the position in the incoming set. Such a problem is solved by cognitrons and neocognitrons - multilevel hierarchical self-organizing networks. The main property of such networks is the limitation of the region of connections by the number of inputs from the previous layer of neurons, which leads us to a similar structure of the visual cortex of the brain. As well as the presence of negative inputs in neurons, previously negative connections were not used in the ANN, although the presence of such connections has been proven in biological neurons. Also interesting is the way to check the completion of ANN training. The network starts in the reverse direction, to supply only one signal to one of the outputs, the ANN produces a stored image, the recognition of which this output is responsible.

    The idea of ​​limiting the area of ​​connections for self-organizing neurons is that during training you can go crazy if you try to remember everything, therefore it is natural that you need to abstract away, highlighting the most common and important properties for all phenomena and objects surrounding us, creating the minimum possible number of outputs from the layer of neurons and then again use these abstractions for self-training of neurons of the next level.

    Also popular now: