Neural networks for the smallest

Hi, in this example, I want to show how you can implement the Hopfield network for pattern recognition.

I myself, like many in one day, decided to ask about software training, AI and neuro networks. Fortunately, there are many examples and examples in the network, but they all operate with an abundance of function formulas and if you are not knowledgeable in mathematics (like me), I will try to demonstrate a simple example of the Hopfield network using the Golang language (GO).

Mathematical network description - Hopfield network

Why the Hopfield network?

A fairly quick and more or less understandable example if you can use such terms as “simple” and “understandable” in the world of AI.

In this example, we will try to recognize images from a black and white image with a size of 20x20 pixels.

Let's try to understand the steps that we have to complete before we get our desired result:

1. Convert image to vector
2. Convert vectors to matrices
3. Sum the matrices and get one single matrix (W)
4. Zero a single matrix on the diagonal
5. Multiply the matrix W by the vector of the incoming image
6. Pass the received vector through the activation function for this network (F)
7. Substitute the new vector to point 5 and continue the operation until we get a stable state of the network (at the output we will receive the same vector).

Let's go to the code with a detailed description. All libraries that we need:

package main
import (
	"github.com/oelmekki/matrix""fmt""os""log""image""math""image/color""image/png"
	_ "image/jpeg"
)

Create an array of vectors of 3 elements (the number of samples), convert images into vectors and add samples to the array. The 'Y' vector is the image we want to recognize.

   
	vectorArrays := [3][]float64{}
	x1 := getVectorFromImage("Images/А.jpg")
	x2 := getVectorFromImage("Images/Б.jpg")
	x3 := getVectorFromImage("Images/О.jpg")
	y := getVectorFromImage("Images/Income.jpg")
	// Add images to the array
	vectorArrays[0] = x1
	vectorArrays[1] = x2
	vectorArrays[2] = x3

Create an array of matrices, convert all vectors into matrices and add them to the array of matrices. Create a preset of the matrix W and begin the summation of all matrices, put the result in W.

   
	matrixArray := [len(vectorArrays)]matrix.Matrix{}
	for i, vInArray := range vectorArrays {
		matrixArray[i] = vectorToMatrix(vInArray, vInArray)
	}
	W := matrix.Matrix{}
	for i, matrixInArray := range matrixArray {
		if i == 0 {
			W = matrixInArray
			continue
		}
		W, _ = W.Add(matrixInArray)
	}

Zero the matrix diagonally.

for i := 0; i < W.Rows(); i++ {
		W.SetAt(i, i, 0)
	}

Create the output vector blank, multiply the matrix by the Y vector, put the result in the vector S and substitute it back to the multiplication.

S := make([]float64, 400)
	for II := 0; II < 100; II++ {
		if II == 0 {
			S, _ = W.VectorMultiply(y)
			for i, element := range S {
				// Activation Func "sigmod"
				S[i] = sigmod(element)
			}
			continue
		} else {
			S, _ = W.VectorMultiply(S)
			for i, element := range S {
				// Activation Func "sigmod"
				S[i] = sigmod( element)
			}
		}
	}

More on the sigmod () function.

This is the activation function F and its principle of operation (in our example) is to convert the data in the outgoing vector and bring it either to 1 or to -1.

Since we work with the bipolar network, the data can only be 1 and -1.

The image should be brought from RGB to 1 and -1, where the sum of all points divided by 3 (conditional pixel brightness) should tend to black or white. Since R = 255, G = 255, B = 255 is white, and R = 0, G = 0, B = 0 is black. I chose a threshold of 150, therefore that greater than or equal to 150 will be white (1), all that is less black (-1), where the choice between black at -1 and white at 1 can be conditional, all we need is to decompose black and white by values. White can also be -1, and black 1, in this case it does not matter. It is also worth considering that we work with a symmetric matrix and the images should be equilateral.

In order to convert an image into a vector, you need to present the image as a matrix that we cut horizontally and add each cut layer (and we will have 20) to the end of the previous layer and get a vector of 400 (20x20) in length.

In the example, I do not check the output vector for stability, but simply go through the cycle 100 times, and at the end I check which sample looks like our result. The network either guesses or not while giving a so-called chimera or a loose interpretation of what she could see. This is the result I save in the image.

Since we use the synchronous mode of the Hopfield network, the result will be weak. Of course, you can use asynchronous, which will take more time and resources, but the result will be much better.

Work Example:

Input Image -
The answer is