Neural networks for curious programmers (with an example in c #)

Since the title was noted “for curious programmers,” I want to say that my curiosity also led me, as a developer of mobile games, to write such a post. I am absolutely sure that there will be programmers who once thought about artificial intelligence and this is a very good chance for them.

After reading a lot of articles about neural networks, I would like to mention some of them that really helped me to master the topic:

java example and useful links,
visual implementation using OOP.

Since there are a lot of theories on this topic, I would like to start implementation.

Implementation

using UnityEngine;
using System.Collections;
using System.Xml.Serialization;
public class Neuron {
	[XmlAttribute("weight")]
	public string data;
	[XmlIgnore]
	public int[,] weight; // веса нейронов
	[XmlIgnore]
	public int minimum = 50; // порог
	[XmlIgnore]
	public int row = 64,column = 64;
	/**
     * Конструктор нейрона, создает веси и устанавливает случайные значения
     */
	public Neuron()
	{
		weight = new int[row,column];
		randomizeWeights();
	}
	/**
     * ответы нейронов, жесткая пороговая
     * @param input - входной вектор
     * @return ответ 0 или 1
     */
	public int transferHard(int[,] input)
	{
		int power = 0;
		for(int r = 0; r < row;r++)
		{
			for(int c = 0; c < column;c++)
			{
				power += weight[r,c]*input[r,c];
			}
		}
		//Debug.Log("Power: " + power);
		return power >= minimum ? 1 : 0;
	}
	/**
     * ответы нейронов с вероятностями
     * @param input - входной вектор
     * @return n вероятность
     */
	public int transfer(int[,] input)
	{
		int power = 0;
		for(int r = 0; r < row;r++)
			for(int c = 0; c < column;c++)
				power += weight[r,c]*input[r,c];
		//Debug.Log("Power: " + power);
		return power;
	}
	/**
     * устанавливает начальные произвольные значения весам 
     */
	void randomizeWeights()
	{
		for(int r = 0; r < row;r++)
			for(int c = 0; c < column;c++)
				weight[r,c] = Random.Range(0,10);
	}
	/**
     * изменяет веса нейронов
     * @param input - входной вектор
     * @param d - разница между выходом нейрона и нужным выходом
     */
	public void changeWeights(int[,] input,int d)
	{
		for(int r = 0; r < row;r++)
			for(int c = 0; c < column;c++)
				weight[r,c] += d*input[r,c];
	}
	public void prepareForSerialization()
	{
		data = "";
		for(int r = 0; r < row;r++)
		{
			for(int c = 0; c < column;c++)
			{
				data += weight[r,c] + " ";
			}
			data += "\n";
		}
	}
	public void onDeserialize()
	{
		weight = new int[row,column];
		string[] rows = data.Split(new char[]{'\n'});
		for(int r = 0; r < row;r++)
		{
			string[] columns = rows[r].Split(new char[]{' '});
			for(int c = 0; c < column;c++)
			{
				weight[r,c] = int.Parse(columns[c]);
			}
		}
	}
}


The class of neurons that contains weight is a binary array of weights, minimum is the threshold of the function. The transferHard function returns a response to an input vector. Since the response of the function is hard, I use it for training. In my opinion, it trains neurons more efficiently. I will be very grateful if there are reviews on this subject. The transfer function returns a response to the input vector but with probability, the sum may be closer to zero or negative if the neuron is trained for another symbol.

using UnityEngine;
using System.Collections;
using System.Xml.Serialization;
using System.Xml;
using System.IO;
public class NeuralNetwork {
	[XmlArray("Neurons")]
	public Neuron[] neurons;
	/**
     * Конструктор сети создает нейроны
     */
	public NeuralNetwork()
	{
		neurons = new Neuron[10];
		for(int i = 0;i output[maxIndex])
				maxIndex = i;
		return maxIndex;
	}
	/**
     * функция обучения
     * @param input - входной вектор
     * @param correctAnswer - правильный ответ
     */
	public void study(int[,] input,int correctAnswer)
	{
		int[] correctOutput = new int[neurons.Length];
		correctOutput[correctAnswer] = 1;
		int[] output = handleHard(input);
		while(!compareArrays(correctOutput,output))
		{
			for(int i = 0; i < neurons.Length;i++)
			{
				int dif = correctOutput[i]-output[i];
				neurons[i].changeWeights(input,dif);
			}
			output = handleHard(input);
		}
	}
	/**
     * сравнение двух вектор
     * @param true - если массивы одинаковые, false - если нет
     */
	bool compareArrays(int[] a,int[] b)
	{
		if(a.Length != b.Length)
			return false;
		for(int i = 0;i 0)
		{
			byte[] tempData = new byte[fStream.Length];
			fStream.Read(tempData, 0, tempData.Length);
			xml = System.Text.Encoding.ASCII.GetString(tempData);
		}
		fStream.Close();
		if(string.IsNullOrEmpty(xml))
			return new NeuralNetwork();
		NeuralNetwork data;
		XmlSerializer serializer = new XmlSerializer(typeof(NeuralNetwork));
		using(TextReader reader = new StringReader(xml))
		{
			data = serializer.Deserialize(reader) as NeuralNetwork;
		}
		data.onDeserialize();
		return data;
	}
}


The NeuralNetwork class contains an array of neurons, each of them is designed for a specific symbol. An array of ten elements is created in the constructor, because the example is made for the recognition of numbers (0-9). If you want to use the network to recognize letters, then resize the array accordingly. The handleHard function is called to train neurons; it returns an array of zeros and ones. The getAnswer function is the network response for the input vector, uses the handle function to get an array of responses, each element of the array contains a neuron response with probability. The getAnswer function selects the index of the element that contains the highest probability and returns the ego as an answer.

After studying a lot of literature, I learned that an activator can be implemented using a Sigmoidal transfer function, which amplifies weak signals and holds strong ones, but I can’t understand how this will affect the improvement of character recognition.

I would like to see an example of character recognition using the Radial Basis function, as it is said almost everywhere that this method improves recognition.

Conclusion

In conclusion, I would like to say that for a better understanding of the codes of neural networks I advise you to read some literature and try to solve problems of this type on your own, starting with a primitive single-layer perceptron.

I wanted to see various reviews on the topic and on the post.

Also popular now: