Researchers Visualized Neural Network Calculations

Visualization of the neural network learning cycle A

team of Graphcore project engineers built graphs of the activity of neural network nodes and their connections during the pattern recognition training process, which the researchers described in their blog .

The image above demonstrates the full cycle of training and recognition of the Microsoft Research RESNET-34 neural network in December 2016. The system itself was deployed on the basis of IPU - an intelligent graphics processor, as the creators call it, in the middle of 2016. The obtained data was colored in order to isolate the various density of calculations produced by the neural network.

All the images obtained by researchers were not only very complex, but also frighteningly similar to real biological objects. The aim of the engineers was to visually show what is happening inside the neural network and why even some scientists are confused by the principle of their work.

Graphcore images constructed are technical graphs of Microsoft's RESNET neural network. In 2015, RESNET won an image recognition competition called ImageNet.

The following image was taken after conducting 50 cycles of Graphcore image recognition neural network training:



The Graphcore IPU system works with the Poplar framework. The framework is written in C ++ and is focused on working with graphs in the course of machine learning of a neural network. Poplar libraries are open source-development, which in the future can be used in conjunction with TensorFlow and MXNet, which can work out of the box with the Graphcore IPU. A set of debugging and analysis tools can be customized using both C ++ and Python.

IPC Graphcore is applicable not only for image recognition, but also for processing a large array of data. For example, developers provide visualization of astrophysical data processing at their IPU under the control of a neural network:



Or here is an image of the AlexNet deep neural network, built using TensorFlow:



AlexNet is also the winner of ImageNet, but 2012. For comparison, the structure of the neural network based on Microsoft Research RESNET is given:



IPU was developed specifically for work with neural networks, and the developers hope that the result of their work will initiate a new stage in machine learning. The Graphcore team notes the greater efficiency of networks on the IPU, as well as a higher learning rate than the competition.