Neural network predicts the first impression of a person on his face

The result of processing photos of Julian Assange and Benedict Cumberbatch by the neural network to predict the first impression of a person. There is a slight lack of subjective intelligence and dominance of the actor compared with the hero he played in the film “The Fifth Power”. The rest of the image Assange passed quite correctly. Illustration: University of Notre Dame, Harvard University

Our brain is able to quickly form an impression of a stranger on his face. For a few seconds, the key characteristics of a person are determined: his attractiveness, intelligence, age, reliability, social status (dominance), sociability and level of morality. It is clear that this impression is inaccurate, and sometimes completely wrong. Absolute accuracy is not required here. Speed ​​is important. This is an absolutely necessary social mechanism, without which it is difficult to survive in society, a vital skill. Therefore, the first impression is so strong and important. Change it then very difficult.

Leaving behind the obvious lack of objectivity of the first impression, it is important to teach computers to evaluate faces in the same way that other people do. Such attempts have been repeatedly made. Researchers tried to algorithmically categorize various emotions across a person’s face, including using neural networks . Published work on the automatic determination of the attractiveness of the person , as well as the presence of a sense of humor and other personal characteristics of a person using neural networks.

All these studies are an absolutely necessary part of the work on creating artificial intelligence at the human level. The AI ​​should be able to read faces no worse than a person, and make subjective evaluations in the face no worse than a person.

Now a group of researchers from the University of Notre Dame (USA) and Harvard University (USA) has conducted a new study that will complement the scientific developments in the field of social psychology. Their work is devoted to the automatic compilation of the first impression that arises about a person among others. The analysis is performed by a neural network trained on sample photos.

According to psychologistswho specialize in studying the system of social assessments of people, at the first assessment of a stranger in the face are two key characteristics: reliability and dominance. According to Alexander Todorov, all other characteristics can be derived from these two main ones. Psychologists have long been a set of basic characteristics that demonstrate the dominance of a person over others, including the inclination of the head, gestures of eyebrows and mouth.

Scientists from the University of Notre Dame and Harvard University have trained the neural network on a sample of 6300 photographs from the publicly available Annotated Facial Landmarks in the Wild . These photos were annotated by users of the crowdsourcing platform TestMyBrain.orgin three social attributes: dominance, reliability and IQ, as well as an additional attribute - age.

6000 photos were used for training, another 200 - for fine-tuning and fine-tuning, and the last 100 - for testing the neural network.


Almost perfect match profiles of Edward Snowden and playing his actor Joseph Gordon-Levitt in the film "Snowden". Illustration: University of Notre Dame, Harvard University

Scientists note that the attributes IQ and age they could make annotations with real, not subjective characteristics. But then the neural network would learn to determine the real IQ and age of a person on his face. As mentioned above, the task was completely different, so for learning the neural network of the category IQ and age were also assigned using crowdsourcing.

The illustration shows individual photos from the base that was used for training. Samples are selected among holders of minimum, medium and maximum characteristics in all categories. It is easy to see that young children have the lowest scores in the IQ, dominance and age categories. A person with an ornate face is one of the minimum marks for reliability.



The distribution of indicators in the training dataset is close to the normal distribution.



The table shows the average ratings of photos in four categories, based on user ratings. The minimum and maximum marks, as well as the average number of marks given are indicated. Each individual received, on average, 32 ratings for dominance and reliability, as well as 15 ratings for age and IQ.



After training, the neural network showed a fairly accurate assessment of persons close to the results of the assessment of site users.



The authors hope that their neural network will be useful in the development of computer vision programs. Such a system is able to analyze the video stream in real time. For example, to show on the screen in the form of a graph how the subjectively perceived intelligence of the person of the host of a television program changes over time. That is, how clever his face looks at certain points in time. Imagine that the program will compile a digest of the most interesting moments of the transfer, when the presenter's face looked the most stupid.



During video processing, the neural network processes each frame separately.

You can think of other applications of this technology. For example, to select the actors who most accurately convey the subjective impression of the appearance of their characters in real life, as shown in the previously mentioned examples with Julian Assange and Edward Snowden.

Probably, such a neural network will find application in personal growth programs: by changing the facial expression in front of the camera, a person will practice how to become more solid and intelligent in the eyes of others - by observing the assessment of his face by the neural network.

One can imagine that some firms will begin to recruit employees, taking into account the results of the subjective assessment of candidates' faces. That is, when training a neural network, the boss chooses several dozen people that he likes according to IQ and reliability characteristics, and the program finds candidates with exactly those characteristics that the boss needs.

Researchers only warn that the accuracy of the neural network directly depends on the quality of the initial data, that is, the assessment of individuals. It is important that this assessment is carried out by representatives of the target audience. The fact is that the impression of a person strongly depends on the personality of the person who assesses: his culture and nationality. Roughly speaking, a very authoritative and wise person in the Zulu tribe will not be perceived as the dominant leader in the community of Oxford aristocrats.

The scientific work was published on October 25, 2016 on the site arXiv.org .