May 20, 2019 at 16:03What the neural network saw in the first photo of a black hole
- Transfer
Friends, many probably remember the images of the black hole that shocked everyone in April of this year. We found very interesting material in which we will talk about what artificial intelligence algorithms “think” about the image of a black hole. By translating this material, we continue a series of publications dedicated to the launch of the Python Neural Networks course . We warn you that the material turned out to be more entertaining than informative, but these images are definitely worth seeing. Go.
On April 11, scientists and engineers from the Team of the Event Horizon telescope made a real breakthrough in understanding the processes that occur in outer space. They presented the first image (photograph) of a black hole. This further strengthened Einstein's general theory of relativity, namely the hypothesis that “massive objects cause distortion in space-time, which is reflected in the form of gravitational changes”.
Well, I'm not a physicist or an astronomer to understand or explain how this works, but I, like millions of people working in various fields, are fascinated by space and especially the black hole phenomenon. The first image of a black hole caused a wave of delight throughout the world. I am a specialist in deep learning, who mainly works with convolutional neural networks, and it became interesting to me that artificial intelligence algorithms “think” about the image of a black hole. This is what we will talk about in the article.
This excerpt from the Epoch Timesdescribes the black hole as follows: “Black holes consist of“ a large amount of matter packed into a very small space ”, mainly formed of“ the remains of a large star that died during a supernova explosion. ”Black holes have such a strong gravitational field that even light can't escape him. The resulting image of the M87 black hole is shown below. This phenomenon is well explained in the article “How to make sense of the black hole image, according to 2 astrophysicists” .
Black Hole - M87 - Event Horizon Telescope
Various areas of a black hole. Screenshot from vox video - Why this black hole photo is such a big deal
1. What CCN sees in the image of a black hole
CCN (Convolution Neural Network) - convolutional neural networks - a class of deep learning algorithms that is extremely effective in the recognition of real-world objects. CCNs are the best neural networks for interpreting and recognizing images. Such neural networks are trained on a million pictures and trained to recognize about 1000 different objects of the surrounding world. I thought about showing the image of a black hole to two trained convolutional neural networks and see how they recognize it, what object of the world around it looks like a black hole. This is not the wisest idea, since the image of the black hole was generated by combining various signals received from space using special equipment,
Neural network forecast VGG-16 - Match
Neural network forecast VGG-19 - Match
Neural network forecast ResNet-50 - Candle
As we see in the images above, trained VGG-16 and VGG-19 see a black hole as a match, and ResNet-50 thinks it's a candle. If we draw an analogy with these objects, we will understand that it makes some sense, since both the burning match and the candle have a dark center surrounded by dense bright yellow light.
2. What are the signs CCN extracted from the image of a black hole
I did one more thing, I visualized what the intermediate layers of VGG-16 generate. Deep learning networks are called deep, because they have a certain number of layers, and each of them processes the presentation and characteristics of the image at the input. Let's see what different layers of the network learn from the incoming image. The result is pretty beautiful.
64 feature cards of the first convolutional layer of VGG-16
If you look closely, you will see that a small bright area is a strong feature, and it is it that is absorbed after passing through most filters. Some interesting filter output are shown below, and they really look like some kind of celestial object.
4 out of 64 feature cards of the first convolutional layer
64 feature maps of the second convolutional layer VGG-16
Let's scale up some interesting feature maps of the second layer of the neural network.
6 of 64 feature cards of the second convolutional layer
Now we go down even deeper and look at the third convolutional layer.
128 feature cards of the third convolutional layer of VGG-16.
After approaching, we find a familiar pattern.
8 of the feature maps presented above, on the third layer
Moving deeper, we get something like this.
6 out of 128 feature cards from 4 convolutional layers of VGG-16 Going deeper
, we get higher-level abstract information, and when we visualize the 7th, 8th and 10th convolution layers, we will see only high-level information.
Feature map of the 7th convolutional layer
As we can see, many of the feature maps are dark and learn only the specific high-level features needed to recognize this class. In deeper layers, they become more noticeable. Now we zoom in and take a look at some filters.
6 feature cards
Now let's look at 512 feature cards of the 10th convolutional layer.
Feature cards 10 convolutional layer.
Now you see that in most of the received feature maps only the image area is accepted as a feature. These are high level signs that are visible to neurons. Let's take a closer look at some of the feature maps above.
Some of the feature cards of 10 convolutional levels, increased in size
Now that we have seen that CCN is trying to isolate a black hole from an image, let's try to pass this image to other popular neural network algorithms, such as Neural Style Transfer and DeepDream.
3. We try Neural Style Transfer and Deep Dream on the image of a black hole.
Neural style transfer is a smart neural network that gives the “style” of one image to another source image and ultimately creates an artistic image. If you still do not understand, then the images below will explain the concept. I used the website deepdreamgenerator.com to create various artistic images from the original image of a black hole. Pictures turned out pretty interesting.
Transmission style. Images generated using deepdreamgenerator.com
DeepDream is a computer vision program created by Google engineer Alexander Mordvintsev that uses a convolutional neural network to search and improve patterns in images using an algorithmic paired algorithm, thereby creating a hallucinogenic image from intentionally processed images.
Deep dream Images generated using deepdreamgenerator.com
In these videos about Deep Dream, you will see how hallucinating images she can create.
That's all! I was extremely shocked when I saw the first photograph of a black hole, and immediately wrote this small article. The information in it may not be so useful, but the images created during its writing and shown above are completely worth it. Enjoy the photos!
Write in the comments how you material. We are waiting for everyone at the open door at the course "Neural Networks in Python" .
On April 11, scientists and engineers from the Team of the Event Horizon telescope made a real breakthrough in understanding the processes that occur in outer space. They presented the first image (photograph) of a black hole. This further strengthened Einstein's general theory of relativity, namely the hypothesis that “massive objects cause distortion in space-time, which is reflected in the form of gravitational changes”.
Well, I'm not a physicist or an astronomer to understand or explain how this works, but I, like millions of people working in various fields, are fascinated by space and especially the black hole phenomenon. The first image of a black hole caused a wave of delight throughout the world. I am a specialist in deep learning, who mainly works with convolutional neural networks, and it became interesting to me that artificial intelligence algorithms “think” about the image of a black hole. This is what we will talk about in the article.
This excerpt from the Epoch Timesdescribes the black hole as follows: “Black holes consist of“ a large amount of matter packed into a very small space ”, mainly formed of“ the remains of a large star that died during a supernova explosion. ”Black holes have such a strong gravitational field that even light can't escape him. The resulting image of the M87 black hole is shown below. This phenomenon is well explained in the article “How to make sense of the black hole image, according to 2 astrophysicists” .
Black Hole - M87 - Event Horizon Telescope
Various areas of a black hole. Screenshot from vox video - Why this black hole photo is such a big deal
1. What CCN sees in the image of a black hole
CCN (Convolution Neural Network) - convolutional neural networks - a class of deep learning algorithms that is extremely effective in the recognition of real-world objects. CCNs are the best neural networks for interpreting and recognizing images. Such neural networks are trained on a million pictures and trained to recognize about 1000 different objects of the surrounding world. I thought about showing the image of a black hole to two trained convolutional neural networks and see how they recognize it, what object of the world around it looks like a black hole. This is not the wisest idea, since the image of the black hole was generated by combining various signals received from space using special equipment,
Neural network forecast VGG-16 - Match
Neural network forecast VGG-19 - Match
Neural network forecast ResNet-50 - Candle
As we see in the images above, trained VGG-16 and VGG-19 see a black hole as a match, and ResNet-50 thinks it's a candle. If we draw an analogy with these objects, we will understand that it makes some sense, since both the burning match and the candle have a dark center surrounded by dense bright yellow light.
2. What are the signs CCN extracted from the image of a black hole
I did one more thing, I visualized what the intermediate layers of VGG-16 generate. Deep learning networks are called deep, because they have a certain number of layers, and each of them processes the presentation and characteristics of the image at the input. Let's see what different layers of the network learn from the incoming image. The result is pretty beautiful.
64 feature cards of the first convolutional layer of VGG-16
If you look closely, you will see that a small bright area is a strong feature, and it is it that is absorbed after passing through most filters. Some interesting filter output are shown below, and they really look like some kind of celestial object.
4 out of 64 feature cards of the first convolutional layer
64 feature maps of the second convolutional layer VGG-16
Let's scale up some interesting feature maps of the second layer of the neural network.
6 of 64 feature cards of the second convolutional layer
Now we go down even deeper and look at the third convolutional layer.
128 feature cards of the third convolutional layer of VGG-16.
After approaching, we find a familiar pattern.
8 of the feature maps presented above, on the third layer
Moving deeper, we get something like this.
6 out of 128 feature cards from 4 convolutional layers of VGG-16 Going deeper
, we get higher-level abstract information, and when we visualize the 7th, 8th and 10th convolution layers, we will see only high-level information.
Feature map of the 7th convolutional layer
As we can see, many of the feature maps are dark and learn only the specific high-level features needed to recognize this class. In deeper layers, they become more noticeable. Now we zoom in and take a look at some filters.
6 feature cards
Now let's look at 512 feature cards of the 10th convolutional layer.
Feature cards 10 convolutional layer.
Now you see that in most of the received feature maps only the image area is accepted as a feature. These are high level signs that are visible to neurons. Let's take a closer look at some of the feature maps above.
Some of the feature cards of 10 convolutional levels, increased in size
Now that we have seen that CCN is trying to isolate a black hole from an image, let's try to pass this image to other popular neural network algorithms, such as Neural Style Transfer and DeepDream.
3. We try Neural Style Transfer and Deep Dream on the image of a black hole.
Neural style transfer is a smart neural network that gives the “style” of one image to another source image and ultimately creates an artistic image. If you still do not understand, then the images below will explain the concept. I used the website deepdreamgenerator.com to create various artistic images from the original image of a black hole. Pictures turned out pretty interesting.
Transmission style. Images generated using deepdreamgenerator.com
DeepDream is a computer vision program created by Google engineer Alexander Mordvintsev that uses a convolutional neural network to search and improve patterns in images using an algorithmic paired algorithm, thereby creating a hallucinogenic image from intentionally processed images.
Deep dream Images generated using deepdreamgenerator.com
In these videos about Deep Dream, you will see how hallucinating images she can create.
That's all! I was extremely shocked when I saw the first photograph of a black hole, and immediately wrote this small article. The information in it may not be so useful, but the images created during its writing and shown above are completely worth it. Enjoy the photos!
Write in the comments how you material. We are waiting for everyone at the open door at the course "Neural Networks in Python" .