Development of a “universal” library for creating a neural network (Home)

Background

I, a mere mortal, didn’t even think about AI as a universal assistant, calmly coded myself for AWPs on C (C / C ++), and everything would be fine, if not one BUT.
I once met a man on a forum devoted to the topic of coding for AWPs, for quite some time he explained a lot to me, helped me, + he himself did quite a lot of interesting things. But one day it was about AI, and then I was interested in his answer in the style of "Lives on my computer something that interests you, a good debugger, task manager + bot for ICQ, which has already passed the Turing test more than once" . Of course, I, as a completely adequate person, did not believe it, because He refused to give the source code, he promised to give the finished program only if I hack his computer and get access to the FS. Several years have passed since then, and more than once the topic was raised at the expense of his AI. Although I still could not get a prototype, and I am not 100% sure that he has one at all, but it doesn’t matter either. each of our conversations gave rise to the idea of ​​humming my own AI in me.

Start

For a start, of course, it was necessary to make a list of everything necessary for development, when I chose my favorite C / C ++ as a basis, a friend told me that "your intellects are usually not written in languages, and you need to choose a declared language for this." I did not immediately like this idea because I would have to re-learn a new language, completely different in principle, and I still decided to take a chance.
So the C / C ++ language was chosen.
Next, it was up to the small (as I thought then) choice of the library on which development would begin, and here the second pitfall was met. I reviewed many articles on ANNs, redid many libraries, but did not find the right one due to the fact that either the library was not flexible enough, or it was difficult to understand, or it did not satisfy in terms of speed. There was of course one more BUT, about him later.
So the task was set to write your library.
Go

I chose Rad Studio on Wednesday , feeling tons of hate. I’ll try to explain:
I need a quick “spanker” for verification, so as not to spend a lot of time on the interface. I
needed ease of transfer to Delphi because many friends chose it as the main language, and Rad Studio is the main environment for it.
I was not going to beat the code to VCL very much for its easier porting to gcc, MS VS.
I will try to use a minimal set of tools for easier cross-platform migration.

So, the main structure of each network is
  • Communications
  • Neurons
  • Layers

And for the task, I needed to somehow untie them from each other but at the same time make them work together. Immediately an idea came to mind - Classes, but after a little thought, after weighing everything + for and - on the contrary, I had to abandon them. The network should be designed for the number of neurons / layers / connections limited only by memory, + the speed of work should be quite high so that this network does not become a brake, + the flexibility of the network should allow simple OnLine operation with both layers and with each neuron in this layer, and with the connections of each neuron individually (This is exactly what was missing in any free library I found).
Linked lists were chosen for this, and it was decided to use pointers. Let's take a look at the frame of the structure, until we fill it ...

typedef struct _link{
}LINK;
typedef struct _neuron{
struct _neuron *prev,*next;
}NEURON;
typedef struct _n_layer{
struct _n_layer *prev,*next;
}NLAYER;

Now the idea is how to tie this all together?
After thinking a bit, I decided to do this: 1 communication unit connects 2 neurons-> each communication instance should be easily accessible and apply only to 2 specific neurons, but a neuron can have an unlimited number of connections (I tried to make a library without software restrictions). Neurons, in turn, must be in the layer for simpler operation, layers could not be used, but working with such clusters of neurons would be extremely difficult. But not only the layer should have a pointer to the neurons belonging to it, but each neuron should have a pointer to the layer in which it is located, but this will already be needed for easier and faster technical work with the network.

So the structure migrated to another more thoughtful state, as it seems to me.
typedef struct _link{
void *from,*to;		//указатель на LINKL
}LINK;
typedef struct _link_list{
LINK *link;
void *Neuron;		//указатель на сами_знаете_что
struct _link_list *next,*prev;
}LINKL;
typedef struct _neuron{
LINKL *in,*out;
void *Layer;		//указатель на сами_знаете_что
struct _neuron *prev,*next;
}NEURON;
typedef struct _n_layer{
NEURON *first,*end;
struct _n_layer *prev,*next;
}NLAYER;

At this point, I write handlers to remove / add an item to the list.
But again, this structure is not filled and it lacks one very important point ...
Let's stop and think, unless when you look somewhere, you can’t think?
Or when you hear something you cannot see? The answer is no.
That is why the idea came up to make threads!

This I have not seen even in paid libraries.
How should this work? Yes, it’s very simple, all you need to do is add 1 parameter to each branch of the main structure, this will be the state parameter — the status will indicate whether further work with this or that network element is possible.
And again, our structure is being modified.

#define STATE_NOT_READY	0
#define STATE_READY		1
typedef struct _link{
void *from,*to;
}LINK;
typedef struct _link_list{
LINK *link;
char state;
void *Neuron;
struct _link_list *next,*prev;
}LINKL;
typedef struct _neuron{
char state;
LINKL *in,*out;
void *Layer;
struct _neuron *prev,*next;
}NEURON;
typedef struct _n_layer{
NEURON *first,*end;
char state;
struct _n_layer *prev,*next;
}NLAYER;


Voila, the neural network framework is ready.

Now let's move on to more delicious buns, and in order not to turn out to be idle sound, I am putting you to study a ready-made library written by me with the help of the esteemed Ilya90 respect to him!

NewNeuroLib Tonich

Pass: XabraZabra

Conclusion

This article describes my approach to creating a “smart” neural network, now I just have to figure out how to fill out each branch, most likely I will choose the data type double, but 3 things are not clear to me and until I understand them, the prototype will not see the light .
  1. (SOLVED) By what condition is a new connection created and / or how is it chosen from which neuron to which it will lead.
  2. (SOLVED) By what condition is a new neuron created (from what and why is it created)
  3. (partially resolved) When should a neuron “shoot”? When all the bond weights are placed, or not necessary, and if so, what will the function look like if it works with a constant amount and quantity of bond weights?

If these points are resolved in the next issue, I will analyze the operation of this network and provide a complete example code.

If there is interest, I can lay out an example of use, but without the SaveLoad library (for it I have not received a Nobel Prize yet)


Welcome to the comments!

Researched Materials

Of course, many articles on Habré.
The FANN library on which he drew the most attention.
The site from which I read a large number of articles.
A very interesting article that made me think (:

Also popular now: