March 12, 2019 at 10:00Demystify convolutional neural networks
- Transfer
Translation of Demystifying Convolutional Neural Networks .
Convolutional neural networks.
In the past decade, we have seen amazing and unprecedented advances in computer vision. Today, computers can recognize objects in images and frames of video with an accuracy of 98%, already ahead of a person with its 97%. It was the functions of the human brain that inspired developers to create and improve recognition techniques.
Once neurologists conducted experiments on cats and found that the same parts of the image activate the same parts of the cat's brain. That is, when the cat looks at the circle, the alpha zone is activated in her brain, and when she looks at the square, the beta zone is activated. The researchers concluded that in the brain of animals there are areas of neurons that respond to specific characteristics of the image. In other words, animals perceive the environment through the multilayer neural architecture of the brain. And each scene, each image passes through a peculiar block of selection of signs, and only then it is transmitted to the deeper structures of the brain.
Inspired by this, mathematicians have developed a system in which groups of neurons are emulated that operate on different image properties and interact with each other to form a common picture.
The idea of a group of activated neurons that are supplied with specific input data was turned into a mathematical expression of a multidimensional matrix that plays the role of a determinant of a set of properties - it is called a filter or kernel. Each such filter looks for some peculiarity in the image. For example, there may be a filter to determine the boundaries. The properties found are then transferred to another set of filters that can determine higher-level properties of the image, for example, eyes, nose, etc.
Convolution of the image using Laplace filters to determine the boundaries.
From the point of view of mathematics, between the input image, presented in the form of a matrix of pixel intensity, and the filter, we perform a convolution operation, resulting in a so-called property map (feature map). This map will serve as input to the next filter layer.
Convolution is a process in which the network tries to mark up the input signal by comparing it with information previously known. If the input signal looks like previous images of cats, already known networks, then the reference signal “cat” will be minimized - mixed - with the input signal. The resulting signal is transmitted to the next layer. In this case, the input signal means a three-dimensional representation of the picture in the form of intensities of RGB pixels, and the reference signal “cat” is learned by the kernel to recognize cats.
Image convolution operation and filter. Source .
The convolution operation has an excellent property - translation invariant. This means that each convolution filter reflects a certain set of properties, for example, eyes, ears, etc., and the convolutional neural network algorithm learns to determine which set of properties corresponds to the reference of, say, a cat. The intensity of the output signal does not depend on the location of the properties, but on their presence. Therefore, the cat can be depicted in various poses, but the algorithm can still recognize it.
By following the principle of the biological brain, scientists were able to develop a mathematical apparatus for extracting properties. But after evaluating the total number of layers and properties that need to be analyzed to track complex geometric shapes, scientists realized that computers would not have enough memory to store all the data. Moreover, the amount of required computing resources grows exponentially with the increase in the number of properties. To solve this problem, a pooling technique was developed. Her idea is very simple: if a certain area contains pronounced properties, then we can refuse to search for other properties in this area.
Example of pooling the maximum value.
The pooling operation not only saves memory and processing power, but also helps to clear images from noise.
Okay, why would a neural network come in handy if it can only define sets of image properties? We need to somehow teach her how to categorize images. And the traditional approach to the formation of neural networks will help us in this. In particular, property maps obtained on previous layers can be collected into a layer that is fully associated with all the labels that we prepared for categorization. This last layer will assign the probabilities of matching each class. And based on these final probabilities, we can attribute the image to some category.
Fully bonded layer. Source .
Now it remains only to combine all the concepts studied by the network into a single framework - the convolutional neural network (Convolution Neural Network, CNN). CNN consists of a series of convolutional layers that can be combined with pooling layers to generate a property map that is passed to fully connected layers to determine the probabilities of matching any classes. Bringing back the errors we get, we can train this neural network right up to getting accurate results.
Now that we understand the functional perspectives of CNN, let's take a closer look at the aspects of using CNN.
Convolutional layer.
The convolutional layer is the main building block of CNN. Each such layer includes a set of independent filters, each of which is looking for its own set of properties in the incoming image.
Convolution operation. Source .
From the point of view of mathematics, we take a filter of a fixed size, superimpose it on the image and calculate the scalar product of the filter and a piece of the input image. The results of the work are placed in the final property map. Then we move the filter to the right and repeat the operation, also adding the result of the calculation to the property map. After convolution of the whole image with the help of a filter, we get a property map, which is a set of explicit signs and is fed as input to the next layer.
Stride is the amount of filter offset. In the above illustration, we shift the filter by a factor of 1. But sometimes you need to increase the size of the offset. For example, if neighboring pixels strongly correlate with each other (especially on the lower layers), then it makes sense to reduce the size of the output using the appropriate stride. But if the stride is made too large, then a lot of information will be lost, so be careful.
Stride is 2. Source .
Padding single layer. Source .
One of the side effects of striding is the consistent decrease in the property map as more and more new convolutions are performed. This may be undesirable since “reduction” means loss of information. To make it clearer, pay attention to the number of times the filter is applied to the cell in the middle and in the corner. It turns out that for no reason the information in the middle part is more important than at the edges. And to extract useful information from earlier layers, you can surround the matrix with layers of zeros.
Why do we need convolutional networks if we already have good deep learning neural networks? It is noteworthy that if we use deep learning networks to classify images, the number of parameters on each layer will be a thousand times larger than that of a convolutional neural network.
Sharing parameters in a convolutional neural network.
Convolutional neural networks.
In the past decade, we have seen amazing and unprecedented advances in computer vision. Today, computers can recognize objects in images and frames of video with an accuracy of 98%, already ahead of a person with its 97%. It was the functions of the human brain that inspired developers to create and improve recognition techniques.
Once neurologists conducted experiments on cats and found that the same parts of the image activate the same parts of the cat's brain. That is, when the cat looks at the circle, the alpha zone is activated in her brain, and when she looks at the square, the beta zone is activated. The researchers concluded that in the brain of animals there are areas of neurons that respond to specific characteristics of the image. In other words, animals perceive the environment through the multilayer neural architecture of the brain. And each scene, each image passes through a peculiar block of selection of signs, and only then it is transmitted to the deeper structures of the brain.
Inspired by this, mathematicians have developed a system in which groups of neurons are emulated that operate on different image properties and interact with each other to form a common picture.
Retrieving Properties
The idea of a group of activated neurons that are supplied with specific input data was turned into a mathematical expression of a multidimensional matrix that plays the role of a determinant of a set of properties - it is called a filter or kernel. Each such filter looks for some peculiarity in the image. For example, there may be a filter to determine the boundaries. The properties found are then transferred to another set of filters that can determine higher-level properties of the image, for example, eyes, nose, etc.
Convolution of the image using Laplace filters to determine the boundaries.
From the point of view of mathematics, between the input image, presented in the form of a matrix of pixel intensity, and the filter, we perform a convolution operation, resulting in a so-called property map (feature map). This map will serve as input to the next filter layer.
Why a convolution?
Convolution is a process in which the network tries to mark up the input signal by comparing it with information previously known. If the input signal looks like previous images of cats, already known networks, then the reference signal “cat” will be minimized - mixed - with the input signal. The resulting signal is transmitted to the next layer. In this case, the input signal means a three-dimensional representation of the picture in the form of intensities of RGB pixels, and the reference signal “cat” is learned by the kernel to recognize cats.
Image convolution operation and filter. Source .
The convolution operation has an excellent property - translation invariant. This means that each convolution filter reflects a certain set of properties, for example, eyes, ears, etc., and the convolutional neural network algorithm learns to determine which set of properties corresponds to the reference of, say, a cat. The intensity of the output signal does not depend on the location of the properties, but on their presence. Therefore, the cat can be depicted in various poses, but the algorithm can still recognize it.
Pooling
By following the principle of the biological brain, scientists were able to develop a mathematical apparatus for extracting properties. But after evaluating the total number of layers and properties that need to be analyzed to track complex geometric shapes, scientists realized that computers would not have enough memory to store all the data. Moreover, the amount of required computing resources grows exponentially with the increase in the number of properties. To solve this problem, a pooling technique was developed. Her idea is very simple: if a certain area contains pronounced properties, then we can refuse to search for other properties in this area.
Example of pooling the maximum value.
The pooling operation not only saves memory and processing power, but also helps to clear images from noise.
Fully bonded layer
Okay, why would a neural network come in handy if it can only define sets of image properties? We need to somehow teach her how to categorize images. And the traditional approach to the formation of neural networks will help us in this. In particular, property maps obtained on previous layers can be collected into a layer that is fully associated with all the labels that we prepared for categorization. This last layer will assign the probabilities of matching each class. And based on these final probabilities, we can attribute the image to some category.
Fully bonded layer. Source .
Final architecture
Now it remains only to combine all the concepts studied by the network into a single framework - the convolutional neural network (Convolution Neural Network, CNN). CNN consists of a series of convolutional layers that can be combined with pooling layers to generate a property map that is passed to fully connected layers to determine the probabilities of matching any classes. Bringing back the errors we get, we can train this neural network right up to getting accurate results.
Now that we understand the functional perspectives of CNN, let's take a closer look at the aspects of using CNN.
Convolutional neural networks
Convolutional layer.
The convolutional layer is the main building block of CNN. Each such layer includes a set of independent filters, each of which is looking for its own set of properties in the incoming image.
Convolution operation. Source .
From the point of view of mathematics, we take a filter of a fixed size, superimpose it on the image and calculate the scalar product of the filter and a piece of the input image. The results of the work are placed in the final property map. Then we move the filter to the right and repeat the operation, also adding the result of the calculation to the property map. After convolution of the whole image with the help of a filter, we get a property map, which is a set of explicit signs and is fed as input to the next layer.
Strides
Stride is the amount of filter offset. In the above illustration, we shift the filter by a factor of 1. But sometimes you need to increase the size of the offset. For example, if neighboring pixels strongly correlate with each other (especially on the lower layers), then it makes sense to reduce the size of the output using the appropriate stride. But if the stride is made too large, then a lot of information will be lost, so be careful.
Stride is 2. Source .
Padding
Padding single layer. Source .
One of the side effects of striding is the consistent decrease in the property map as more and more new convolutions are performed. This may be undesirable since “reduction” means loss of information. To make it clearer, pay attention to the number of times the filter is applied to the cell in the middle and in the corner. It turns out that for no reason the information in the middle part is more important than at the edges. And to extract useful information from earlier layers, you can surround the matrix with layers of zeros.
Parameter Sharing
Why do we need convolutional networks if we already have good deep learning neural networks? It is noteworthy that if we use deep learning networks to classify images, the number of parameters on each layer will be a thousand times larger than that of a convolutional neural network.
Sharing parameters in a convolutional neural network.
