To see the world through the eyes of animals: new horizons of IT tracking
When we talk about technologies for recognizing emotions, by default we mean a person, that is, the whole array of data - verbal and non-verbal - that is generated and used by him for a comprehensive assessment of his condition. However, the farther, the more attention is paid to animals: in recent years, studies of the "emotional" world of horses or dogs, for example, have been regularly conducted. In our article, we will discuss one of the turns of this topic: how IT tracking technologies are used to study the characteristics of some representatives of the fauna living side by side with us. So how do you look around with the eyes of a dog, peacock or even a mouse?
Probably every dog lover wants to know how the friends of a person perceive us and their relatives. Therefore, this direction of recording eye movements of animals leads in terms of the variety of experimental non-invasive techniques.
For dogs, as for other social animals, gaze is an important social signal. The predominant function of dog-directed gaze in dogs is a signal of dominance or ritualized aggression (Schenkel, 1967). However, such oculomotor behavior demonstrated by the dog in relation to the owners is interpreted as initialization of communication (Kis, Hernádi, Miklósi, Kanizsár, & Topál, 2017). Also, an analysis of the oculomotor activity of dogs is a promising area for studying not only communication, but also recognition of faces and emotional facial expressions of people.
In the course of studies using the method of IT tracking on dogs, it was shown that the latter can not only recognize people by their faces, but also distinguish individual human emotions: joy, sadness, anger. When viewing images of people's faces, dogs give preference to the eye area, regardless of facial expression of the stimulus (Kis et al., 2017).
Tornqvist et al. (Törnqvist et al., 2015) conducted an interesting comparative study of dog and human image viewing. As stimulus material, photographs depicting social interaction or avoiding social interaction between two people or two dogs were chosen. So, both people and dogs have demonstrated a longer time viewing objects in images with social interaction. It should be noted that people looked at pictures with social interaction of dogs more than time than people, while dogs - on the contrary.
As for the study of human eye movements, for dogs there are two main types of experimental facilities: stationary and wearable eye trackers.
Fixed installations are a tracker mounted under a monitor or projector, on which incentives are presented. In such installations, the dog is either preliminarily trained to stay for a long time (for example, Tornqvist et al. (Törnqvist et al., 2015) used this version of the experimental setup, it is also used in Clever Dog Lab - the experiments can be performed using the videooculography technique in this laboratory read here), or the owner fixes it in the right position, as was done in a study by Kis et al. (Kis et al., 2017). Regarding the preliminary training of dogs in staying in an experimental setup, there is an opinion that the training itself creates an additional cognitive task for the dog, which distorts the distribution of visual attention (Kis et al., 2017). In addition, it is logical to assume that retention in the installation by the owner can also introduce certain distortions into the results obtained, and they are fundamentally important in studying the psychophysiological state of animals.
Therefore, the most promising method for studying the eye movements of dogs in more natural conditions is the system of eyewear-glasses. So, at the University of Lincoln (UK), Williams and colleagues (Williams, Mills, & Guo, 2011) developed, based on the IT tracker VisionTrak (60 Hz), tracker glasses for dogs that work with an accuracy of 2.25-2.71 °. The whole structure is fixed to the muzzle: between the ears is the scene camera, in front of the left eye - a dichroic mirrorabove the left eye - a camera for recording eye movements and infrared illumination. During calibration, a calibration structure is attached to the center of the muzzle (in the nose), which consists of two crossed axes, the ends of which correspond to the boundaries of the scene camera frame. At the end of each calibration axis - there is an area where something tasty is fixed during the calibration - the task of the dog is to look at the desired piece of food, with successful calibration it receives it as a reward. Thus, this tracker does not require special training of the dog’s behavior during the experiment, but involves only a short training for calibration.
Another project for creating a tracker-glasses for dogs is DogCam (Rossi, Parada, & Allen, 2010).
Many amazing facts about studying the behavior of dogs can be found on the CleverDogLab page .
Birds have extremely sharp eyesight, and among them there are no species that would have no vision as such, so studying their visual perception is of particular interest. A feature of focusing the gaze on the object in birds is that they have practically no vestibulo - ocular reflex , that is, the head can move faster than the eyes. For example, in pigeons, eye movements account for only 10-20% of eye movement (Yorzinski, Patricelli, Platt, & Land, 2015).
Today there are tracking systems for recording eye movements of only large birds. Such an IT tracker consists of a helmet on which a camera for recording eye movements (30 Hz) with infrared illumination and a scene camera, and a backpack with a battery and a transmitter are mounted. Similar trackers only perform monocular recordings (while the second eye of the bird is closed), since the design with two cameras for recording eyes would be too heavy.
For example, during the use of the described system in the study of the visual attention of peacock females, significant results were obtained: when females see a male peacock at close range, their visual attention is directed to the lower half of the tail, while the male peacock spreads its tail, far from the female , - her attention is directed to the upper half of the tail (Yorzinski, Patricelli, Babcock, Pearson, & Platt, 2013).
Currently, adequate equipment is not available to study the eye movements of small birds (such as starlings), so the problem of the visual attention of small birds so far comes down to tracking head movements. So, in a study of Butler and Fernandez-Juricic (Butler & Fernandez-Juricic, 2018) it was shownfoveal vision when viewing a predator (the authors suggest that at this time the bird receives detailed information that may be required to successfully avoid the enemy).
H. Payne and JL Raymond (Payne & Raymond, 2017) developed a method for recording eye movements on a freely moving mouse with a very high resolution of 0.1 °. To do this, they applied the method of magnetic IT tracking. A cylindrical magnet (0.75 x 1 mm in size, 6.8 mg in weight) is implanted in mice in a sheath of a biocompatible polymer from the temporal side under the conjunctiva, the incision in which is healed using tissue adhesive. The magnet is perpendicular to the axis of the horizontal saccades . Above the implanted magnet, a small angular magnetic sensor (4.8 x 5.8 mm in size, weighing 76 mg) is attached to the skull from above. The authors also conducted pilot experiments with smaller implantable disk magnets (1.5 x 0.5 mm), but they gave too weak a signal.
Calibration of this system is a synchronous recording of the magnetic tracker signal and video using the dual-angle video-oculography method on a mouse, whose head is fixed. The essence of dual-angle video-oculografy is the video recording of eye movements with two fixed cameras with infrared illumination, located at an angle of 40 ° from each other and at an equal distance from the center of the pupil (5 cm). The position of the eye is calculated for each frame by comparing images from both cameras, the position of the eye is attached to the signal from the magnetic tracker. Calibration is carried out only once and subsequently the magnetic tracker can be used on a freely moving mouse in complete darkness. The original article is available online, you can read it here .
In our article, of course, it is impossible to ignore the closest relatives of a person from the animal kingdom - primates. There are much more successful IT tracking studies on primates than on other animals, since monkeys are a classic model object for studying the neurophysiology of the visual system. Most often, the method of electromagnetic aytracking is used to record the movements of the eyes of primates (we wrote about the use of the indicated method of recording eye movements on humans in our previous article ), but in the case of monkeys, the movable element is attached not to the lens, but is implanted. Often, this method is used in combination with invasive neurophysiological techniques that allow you to analyze in detail the features of the brain and visual attention.
You can read about studying the eye movements of monkeys in Russia here .
Material author:
Maria Konstantinova, Researcher at the Neurodata Lab , biologist, physiologist, specialist in the visual sensory system, oculography and oculomotorics.
Dogs
Probably every dog lover wants to know how the friends of a person perceive us and their relatives. Therefore, this direction of recording eye movements of animals leads in terms of the variety of experimental non-invasive techniques.
For dogs, as for other social animals, gaze is an important social signal. The predominant function of dog-directed gaze in dogs is a signal of dominance or ritualized aggression (Schenkel, 1967). However, such oculomotor behavior demonstrated by the dog in relation to the owners is interpreted as initialization of communication (Kis, Hernádi, Miklósi, Kanizsár, & Topál, 2017). Also, an analysis of the oculomotor activity of dogs is a promising area for studying not only communication, but also recognition of faces and emotional facial expressions of people.
In the course of studies using the method of IT tracking on dogs, it was shown that the latter can not only recognize people by their faces, but also distinguish individual human emotions: joy, sadness, anger. When viewing images of people's faces, dogs give preference to the eye area, regardless of facial expression of the stimulus (Kis et al., 2017).
Tornqvist et al. (Törnqvist et al., 2015) conducted an interesting comparative study of dog and human image viewing. As stimulus material, photographs depicting social interaction or avoiding social interaction between two people or two dogs were chosen. So, both people and dogs have demonstrated a longer time viewing objects in images with social interaction. It should be noted that people looked at pictures with social interaction of dogs more than time than people, while dogs - on the contrary.
As for the study of human eye movements, for dogs there are two main types of experimental facilities: stationary and wearable eye trackers.
Fixed installations are a tracker mounted under a monitor or projector, on which incentives are presented. In such installations, the dog is either preliminarily trained to stay for a long time (for example, Tornqvist et al. (Törnqvist et al., 2015) used this version of the experimental setup, it is also used in Clever Dog Lab - the experiments can be performed using the videooculography technique in this laboratory read here), or the owner fixes it in the right position, as was done in a study by Kis et al. (Kis et al., 2017). Regarding the preliminary training of dogs in staying in an experimental setup, there is an opinion that the training itself creates an additional cognitive task for the dog, which distorts the distribution of visual attention (Kis et al., 2017). In addition, it is logical to assume that retention in the installation by the owner can also introduce certain distortions into the results obtained, and they are fundamentally important in studying the psychophysiological state of animals.
Therefore, the most promising method for studying the eye movements of dogs in more natural conditions is the system of eyewear-glasses. So, at the University of Lincoln (UK), Williams and colleagues (Williams, Mills, & Guo, 2011) developed, based on the IT tracker VisionTrak (60 Hz), tracker glasses for dogs that work with an accuracy of 2.25-2.71 °. The whole structure is fixed to the muzzle: between the ears is the scene camera, in front of the left eye - a dichroic mirrorabove the left eye - a camera for recording eye movements and infrared illumination. During calibration, a calibration structure is attached to the center of the muzzle (in the nose), which consists of two crossed axes, the ends of which correspond to the boundaries of the scene camera frame. At the end of each calibration axis - there is an area where something tasty is fixed during the calibration - the task of the dog is to look at the desired piece of food, with successful calibration it receives it as a reward. Thus, this tracker does not require special training of the dog’s behavior during the experiment, but involves only a short training for calibration.
Another project for creating a tracker-glasses for dogs is DogCam (Rossi, Parada, & Allen, 2010).
Many amazing facts about studying the behavior of dogs can be found on the CleverDogLab page .
Birds
Birds have extremely sharp eyesight, and among them there are no species that would have no vision as such, so studying their visual perception is of particular interest. A feature of focusing the gaze on the object in birds is that they have practically no vestibulo - ocular reflex , that is, the head can move faster than the eyes. For example, in pigeons, eye movements account for only 10-20% of eye movement (Yorzinski, Patricelli, Platt, & Land, 2015).
Today there are tracking systems for recording eye movements of only large birds. Such an IT tracker consists of a helmet on which a camera for recording eye movements (30 Hz) with infrared illumination and a scene camera, and a backpack with a battery and a transmitter are mounted. Similar trackers only perform monocular recordings (while the second eye of the bird is closed), since the design with two cameras for recording eyes would be too heavy.
For example, during the use of the described system in the study of the visual attention of peacock females, significant results were obtained: when females see a male peacock at close range, their visual attention is directed to the lower half of the tail, while the male peacock spreads its tail, far from the female , - her attention is directed to the upper half of the tail (Yorzinski, Patricelli, Babcock, Pearson, & Platt, 2013).
Currently, adequate equipment is not available to study the eye movements of small birds (such as starlings), so the problem of the visual attention of small birds so far comes down to tracking head movements. So, in a study of Butler and Fernandez-Juricic (Butler & Fernandez-Juricic, 2018) it was shownfoveal vision when viewing a predator (the authors suggest that at this time the bird receives detailed information that may be required to successfully avoid the enemy).
Mice
H. Payne and JL Raymond (Payne & Raymond, 2017) developed a method for recording eye movements on a freely moving mouse with a very high resolution of 0.1 °. To do this, they applied the method of magnetic IT tracking. A cylindrical magnet (0.75 x 1 mm in size, 6.8 mg in weight) is implanted in mice in a sheath of a biocompatible polymer from the temporal side under the conjunctiva, the incision in which is healed using tissue adhesive. The magnet is perpendicular to the axis of the horizontal saccades . Above the implanted magnet, a small angular magnetic sensor (4.8 x 5.8 mm in size, weighing 76 mg) is attached to the skull from above. The authors also conducted pilot experiments with smaller implantable disk magnets (1.5 x 0.5 mm), but they gave too weak a signal.
Calibration of this system is a synchronous recording of the magnetic tracker signal and video using the dual-angle video-oculography method on a mouse, whose head is fixed. The essence of dual-angle video-oculografy is the video recording of eye movements with two fixed cameras with infrared illumination, located at an angle of 40 ° from each other and at an equal distance from the center of the pupil (5 cm). The position of the eye is calculated for each frame by comparing images from both cameras, the position of the eye is attached to the signal from the magnetic tracker. Calibration is carried out only once and subsequently the magnetic tracker can be used on a freely moving mouse in complete darkness. The original article is available online, you can read it here .
Primates
In our article, of course, it is impossible to ignore the closest relatives of a person from the animal kingdom - primates. There are much more successful IT tracking studies on primates than on other animals, since monkeys are a classic model object for studying the neurophysiology of the visual system. Most often, the method of electromagnetic aytracking is used to record the movements of the eyes of primates (we wrote about the use of the indicated method of recording eye movements on humans in our previous article ), but in the case of monkeys, the movable element is attached not to the lens, but is implanted. Often, this method is used in combination with invasive neurophysiological techniques that allow you to analyze in detail the features of the brain and visual attention.
You can read about studying the eye movements of monkeys in Russia here .
Literature
List of references
Butler, SR, & Fernandez-Juricic, E. (2018). European starlings use their acute vision to check on feline predators but not on conspecifics. PLoS ONE, 1–12. doi.org/10.1371/journal.pone.0188857
C. Pallus, A., & G. Freedman, E. (2016). Target position relative to the head is essential for predicting head movement during head-free gaze pursuit. Experimental Brain Research, 234 (8), 2107–2121. doi.org/10.1007/s00221-016-4612-x
Kis, A., Hernádi, A., Miklósi, B., Kanizsár, O., & Topál, J. (2017). The Way Dogs (Canis familiaris) Look at Human Emotional Faces Is Modulated by Oxytocin. An Eye-Tracking Study. Frontiers in Behavioral Neuroscience, 11 (October), 1–11. doi.org/10.3389/fnbeh.2017.00210
Payne, HL, & Raymond, JL (2017). Magnetic eye tracking in mice. eLife, 6, 1-24. doi.org/10.7554/eLife. 29222
Rossi, A., Parada, FJ, & Allen, C. (2010). DogCam: a Way to Measure Visual Attention in Dogs. 7Th International Conference on Methods and Techniques in Behavioral Research.
Schenkel, R. (1967). Submission: Its features and function in the wolf and dog. Integrative and Comparative Biology, 7 (2), 319–329. doi.org/10.1093/icb/7.2.319
Törnqvist, H., Somppi, S., Koskela, A., Krause, CM, Vainio, O., & Kujala, MV (2015). Comparison of dogs and humans in visual scanning of social interaction. Royal Society Open Science, 2 (9), 150341. doi.org/10.1098/rsos.150341
Williams, FJ, Mills, DS, & Guo, K. (2011). Development of a head-mounted, eye-tracking system for dogs. Journal of Neuroscience Methods, 194 (2), 259-265. doi.org/10.1016/j.jneumeth.2010.10.0.022
Yorzinski, JL, Patricelli, GL, Babcock, JS, Pearson, JM, & Platt, ML (2013). Through their eyes: selective attention in peahens during courtship. Journal of Experimental Biology, 216 (16), 3035-3046. doi.org/10.1242/jeb.087338
Yorzinski, JL, Patricelli, GL, Platt, ML, & Land, MF (2015). Eye and head movements shape gaze shifts in Indian peafowl. Journal of Experimental Biology, 218 (23), 3771-3776. doi.org/10.1242/jeb.129544
C. Pallus, A., & G. Freedman, E. (2016). Target position relative to the head is essential for predicting head movement during head-free gaze pursuit. Experimental Brain Research, 234 (8), 2107–2121. doi.org/10.1007/s00221-016-4612-x
Kis, A., Hernádi, A., Miklósi, B., Kanizsár, O., & Topál, J. (2017). The Way Dogs (Canis familiaris) Look at Human Emotional Faces Is Modulated by Oxytocin. An Eye-Tracking Study. Frontiers in Behavioral Neuroscience, 11 (October), 1–11. doi.org/10.3389/fnbeh.2017.00210
Payne, HL, & Raymond, JL (2017). Magnetic eye tracking in mice. eLife, 6, 1-24. doi.org/10.7554/eLife. 29222
Rossi, A., Parada, FJ, & Allen, C. (2010). DogCam: a Way to Measure Visual Attention in Dogs. 7Th International Conference on Methods and Techniques in Behavioral Research.
Schenkel, R. (1967). Submission: Its features and function in the wolf and dog. Integrative and Comparative Biology, 7 (2), 319–329. doi.org/10.1093/icb/7.2.319
Törnqvist, H., Somppi, S., Koskela, A., Krause, CM, Vainio, O., & Kujala, MV (2015). Comparison of dogs and humans in visual scanning of social interaction. Royal Society Open Science, 2 (9), 150341. doi.org/10.1098/rsos.150341
Williams, FJ, Mills, DS, & Guo, K. (2011). Development of a head-mounted, eye-tracking system for dogs. Journal of Neuroscience Methods, 194 (2), 259-265. doi.org/10.1016/j.jneumeth.2010.10.0.022
Yorzinski, JL, Patricelli, GL, Babcock, JS, Pearson, JM, & Platt, ML (2013). Through their eyes: selective attention in peahens during courtship. Journal of Experimental Biology, 216 (16), 3035-3046. doi.org/10.1242/jeb.087338
Yorzinski, JL, Patricelli, GL, Platt, ML, & Land, MF (2015). Eye and head movements shape gaze shifts in Indian peafowl. Journal of Experimental Biology, 218 (23), 3771-3776. doi.org/10.1242/jeb.129544
Material author:
Maria Konstantinova, Researcher at the Neurodata Lab , biologist, physiologist, specialist in the visual sensory system, oculography and oculomotorics.