Feel neural networks or neural network constructor

I have long been interested in neural networks, but only from the perspective of the viewer - I followed the new opportunities that they provide compared to conventional programming. But he never climbed into theory or practice. And suddenly (after the sensational news about AlphaZero) I wanted to make my neural network. After watching a few lessons on this topic on YouTube, I got into a little theory and went on to practice. In the end, I did even better than my neural network. The result was a designer of neural networks and a visual aid for them (that is, you can watch what is happening inside the neural network). Here's what it looks like:



And now a little more. Using this constructor, you can create direct distribution networks (Feedforward neural network) of up to 8 hidden layers (plus an input layer and an output layer, totaling 10 layers (usually 4 layers are more than enough)) in each layer up to 30 neurons (the restriction is related to the fact that all this is simultaneously displayed on the screen, if there are requests in the comments I will release the version without restrictions and visualization). The activation function of all neurons is a sigmoid based on a logistic function. You can also train the resulting network using the back propagation method using gradient descent using given examples. And, most importantly, you can look at each neuron in each individual case (what value it passes on, its displacement (correction, bias) - neurons with a negative displacement are white, with a positive one - bright green), neuron connections, depending on their weight, are marked in red - positive, blue - negative, and also differ in thickness - the larger the modulus of weight, the thicker. And if you hover the mouse over a neuron, you can still see which signal comes to it, and what specific offset it has. This is useful to understand how a particular network works or to show students how direct distribution networks work. But most importantly, you can save your network to a file and share it with the world. how a particular network works or show students how direct distribution networks work. But most importantly, you can save your network to a file and share it with the world. how a particular network works or show students how direct distribution networks work. But most importantly, you can save your network to a file and share it with the world.

Next, there will be instructions for using the program, embedding the created networks in your projects, as well as the analysis of several networks included in the kit.

How to use the constructor

To get started, download the archive from here .

Unzip to the root of the drive D:\

Run NeuroNet.exe

You can try “Download” some network, look at it, click on “Training”, see its accuracy, poke arrows left, right (on the sides) to see different options for input (left column of neurons) and output (right) data, press “Stop” and try to enter your input data (any values ​​from 0 to 1 are allowed, take this into account when creating your networks and normalize the input and output data).

Now how to build your network. First of all, you need to set the network architecture (the number of neurons in each layer separated by a comma), click “Build” (or “Tear down” first, then build, if another network is already displayed on your screen), click “Training sample”, “Delete all »And enter your training examples according to the instructions on the screen. You can also specify small square images (maximum 5x5 pixels) to enter and exit, from which normalized pixel brightness values ​​will be determined (not taking into account their color), for which you need to click on “in” and “out”, respectively. Click "Add Example", repeat the procedure as many times as necessary. Click “Finish”, “Training” and how accuracy will be satisfactory (usually 98%), click “Stop”, a disk icon (save), give the network a name and be glad that you yourself created a neural network.

Integration of created networks into your projects

To use my neural networks in my own projects, I created a separate application doNet.exethat needs to be launched with the following parameters: “ D:\NeuroNet\doNet.exe <название сети> <входные данные через пробел>”, wait for the application to finish working, and then read the output from D:\NeuroNet\temp.txt

For example, the application 4-5.exe was created using the network “4-5” (about this and other networks below). This application details how to run doNet.exe correctly

Analysis of the networks included

Let's start with the classics - “XOR (Half Adder)”. Among others, in particular, this task - modulo 2 addition - was cited in 1969 as an example of the limitations of neural networks (namely, single-layer perceptrons). In general, there are two inputs (with values ​​of either 0 or 1 for each), our task is to answer 1 if the values ​​of the inputs are different, 0 if they are the same.



The following is “Quantity-units”. Three inputs (0 or 1 on each). It is required to calculate how many units have been submitted. Implemented as a classification task - four outputs for each answer option (0,1,2,3 units). On which output the maximum value is, the answer is correspondingly.



"Multiplication" - Two inputs (real from 0 to 1), the output of their product.



“4-5” - Normalized brightness values ​​of pixels of a 4x4 image are fed to the input, at the output we have normalized brightness values ​​of pixels of a 5x4 image.



The network thought about how to increase the quality of a large picture by 25%, but an interesting filter for a photo came out:



UPD:NeuroNet2.exe application was added to the archive (the same constructor, but without visualization (thanks to which it works 2 times faster) and restrictions on the number of neurons in the layer (up to 1024 instead of 30), and square pictures can also be applied to the input and output in the training set up to 32x32). A training schedule has also been added. Now they can use neural networks (and embed in their projects (even on the server)) those who do not know their theory! In semi-automatic mode (after learning to manually input values ​​and get the result on the screen), they can be used even without programming knowledge!



That's all, I'm waiting for comments.

PS If an error pops up, try to register comdlg32 files from the administrator using regsvr32, which are also in the archive.