Neural network determines image recall

This photo you will soon forget

Researchers at the Massachusetts Institute of Technology’s Laboratory for Computer Science and Artificial Intelligence have developed an algorithm that determines image recall. Moreover, based on this algorithm, an application for improving photos has been created so that they are well imprinted in the brain.

As you might guess, the algorithm was developed by training the neural network. Training took place in public. During training, each photograph from the album was shown 600 milliseconds, then a pause of 800 milliseconds, then dozens of other pictures followed. After a random time, the first photo was repeated. Some other photographs were also repeated later. At any time, the user could press a button and signal that he recognized the photo. Thus, the neural network received information about which photos are better remembered.

As a result of training, the neural network learned to literally predict human memory. Moreover, it highlights areas with high and low memorability in the frame, due to which the photo can be optimized.



As a result, the researchers compiled a large database of 60,000 photographs with an index of recallability - LaMem (Large-scale Memorability Dataset).

The algorithm is called MemNet, there is a demo page online .

A pre-prepared neural network model can be downloaded and tested offline using the Caffe framework .

“Understanding memorability will help in creating systems that store the most important information or, conversely, keep that information that people are likely to forget,” says Aditya Khosla, graduate student at the Laboratory for Computer Science and Artificial Intelligence, lead author of the science project. “It's like having an instantly accessible focus group that speaks of the likelihood of remembering a particular visual message.”

Such a system can be applied in various fields: from improving the efficiency of advertisements and messages on a social network to developing more effective training materials. Or as a personal assistant who brings information to you in the most memorable way.

The project developers not only published the design of the neural network, but are also ready to provide everyone with a LaMem database with 60,000 annotated photos ( on request ).

A more detailed discussion of convolutional neural network training and the operation of the MemNet algorithm is described in a scientific paper . It describes how the base was compiled, how the visual memory game was played during the training of the neural network and, in the end, how the results of the work of the neural network were checked after its training.