From Alibaba to Pyaterochka: who and how uses face recognition in business
Pay for a burger without a card, do not wait for check-in at the hotel, do not stand in line at the cashier — all this is possible with the help of face recognition technology. In recent years, such solutions have been actively tested by many large Russian and foreign retailers. We selected the five most interesting examples.
Learn more about how to use facial recognition, neural networks, and machine learning in business at the AI School intensive course .
KFC, along with Ant Financial (Alibaba’s “daughter”), launched Smile to Pay service - a “smile-like” payment at its restaurant in Hangzhou in 2017. The head of the Alibaba Group, Jack Ma, first presented it in 2015 at the CEBIT trade show in Hannover, but it came to stores only two years later.
How it works: To buy a burger, you must enter the phone number tied to your Alipay wallet and smile at the camera embedded in a special biometric terminal. After that, the system confirms the identity of the owner of the wallet and allows you to make a payment.
Video from the official account of Alibaba Group on YouTube.
A smile is needed for the system to understand that it is a living person, not a photograph. Two or three seconds are enough for recognition. It is impossible to deceive the system, even by changing the hair color, makeup or wearing a wig: it uses a complex of distinctive features, taking into account both the geometry of the face, and the location of certain points on it.
In July 2018, technology also began to be tested.in two Marriott hotels - in Hangzhou and in Sanya County on Hainan Island. Waiting at the reception desk is no longer necessary: guests of the hotels for registration check in and enter the contact details in the self-service terminal. After that, the terminal checks the information with the reservation number and prints the key card to enter the room. Registration time was reduced three times and takes only a minute (earlier - three).
Source: Walmart official website.
The world's largest retailer in 2017 developed a technology that helps determine how satisfied the customer is with the store.
How it works: if the system detects a buyer with an unhappy face, it should signal the store employees. According to Walmart management, this will help improve customer service before they complain massively about any problems.
In addition, the system should help analyze consumer behavior over longer periods - for this, it will connect their emotions with how much they spend and what they buy. The client’s biometric data is matched with the data of his transactions to detect changes in consumer habits due to dissatisfaction.
Earlier, in 2015, Walmart had already tried to introduce a facial recognition system - to prevent theft in stores - but the experiment eventually failed . The system was considered unprofitable, how many thieves could be detected with its help, Walmart management did not report.
The largest Russian retailer - X5 (Pyaterochka, Karusel, and Perekrestok) - also began testing face recognition technologies in 2017. According to representatives of the company, the technology should , in particular, help reduce waiting in queues at the box office and optimize the retail space.
How it works: remember the signs at the checkout - “If there are more than five people in the queue, call the number ...”? The face recognition system does the same thing automatically: if the queues of more than five customers, and some of the store’s cash desks do not work, the director receives an alert: “We need to open a cashier.” And with the help of video analytics, you can find out where more people go to the store, what they pay attention to, in order to properly position the goods and promotional materials.
At the same time, facial recognition systems do not store photos, explained Vasily Gromov , director of information technology at the Karusel retail chain (part of the X5 Retail Group). The system builds a function that determines the location of points on the face - cheekbones, eyes, ears, and so on (face hash). The next time, getting in the field of view of the cameras connected to the module for recognition, the system checks the image with the hash directory and gives a signal about who is in the video. Vasily Gromov also pointed out that “from the point of view of the law, a person’s hash is not personal information, since it does not allow identifying a person without additional information”. In the official blog of X5 on Habré, the company recently wrote that it continues to compare technologies for recognizing faces from different companies.
Source: AddReality official site.
In Dixy, facial recognition technologies were decided to be tested simultaneously with the X5 Retail Group, but they chose another developer, AddReality. The goal was to determine the sex and age of customers and implement targeted advertising in the cashier zone and in the sales area. Dixy began testing biometric identification at Victoria stores and its headquarters in 2017.
How it works: On large screens installed in sales areas, a camera is mounted to recognize the face of a person who, passing by, draws attention to the screen.
“It is possible to determine whether a man is a woman or a woman, an approximate age, and taking into account these parameters, based on the existing pool of advertisers, the most suitable advertisement is broadcast on the monitor. For women, this could be an advertisement for cosmetics, and for a middle-aged man, for example, beer, ” explained Dixy's IT director Vladimir Muravyov.
Source: Yandex
Yandex in June 2018 launched the sale of targeted advertising on screens using the face recognition system at Asna pharmacies. At the core is the same technology that has already been used in Victoria stores, supplemented by the ability to select the most appropriate advertising for the user using a search engine.
How it works: the system consists of a screen and a camera that recognizes the viewer: his age, gender, appearance features - for example, glasses or a beard, and other characteristics. After that, with the help of Yandex.Direct, the most appropriate advertising for such a client is displayed on the screen.
According to a Yandex representative, the system can independently determine the types of several people in front of the screen, and the advertiser will pay only for targeted impressions. If there are three people in front of the screen, only two of which are advertising relevant, the system will count two impressions.
In Addreality, it was noted that the system “uses only impersonal user data, does not identify individual customers, does not use personal data, and does not record video of customers passing by”. The experiment began with 70 screens, another 2000 will appear in Asna pharmacies in case of successful testing of the system.
Learn more about how to use facial recognition, neural networks, and machine learning in business at the AI School intensive course.. Course speakers from Microsoft, “Nanosemantics” and Comparex LLC will tell you how to use different types of AI and what tools are available for this. The nearest intensive will take place on November 17 - December 15.
Learn more about how to use facial recognition, neural networks, and machine learning in business at the AI School intensive course .
1. Alibaba: payment using "smiles" in stores and check-in in hotels
KFC, along with Ant Financial (Alibaba’s “daughter”), launched Smile to Pay service - a “smile-like” payment at its restaurant in Hangzhou in 2017. The head of the Alibaba Group, Jack Ma, first presented it in 2015 at the CEBIT trade show in Hannover, but it came to stores only two years later.
How it works: To buy a burger, you must enter the phone number tied to your Alipay wallet and smile at the camera embedded in a special biometric terminal. After that, the system confirms the identity of the owner of the wallet and allows you to make a payment.
Video from the official account of Alibaba Group on YouTube.
A smile is needed for the system to understand that it is a living person, not a photograph. Two or three seconds are enough for recognition. It is impossible to deceive the system, even by changing the hair color, makeup or wearing a wig: it uses a complex of distinctive features, taking into account both the geometry of the face, and the location of certain points on it.
In July 2018, technology also began to be tested.in two Marriott hotels - in Hangzhou and in Sanya County on Hainan Island. Waiting at the reception desk is no longer necessary: guests of the hotels for registration check in and enter the contact details in the self-service terminal. After that, the terminal checks the information with the reservation number and prints the key card to enter the room. Registration time was reduced three times and takes only a minute (earlier - three).
2. Walmart: search for "unhappy" customers in stores
Source: Walmart official website.
The world's largest retailer in 2017 developed a technology that helps determine how satisfied the customer is with the store.
How it works: if the system detects a buyer with an unhappy face, it should signal the store employees. According to Walmart management, this will help improve customer service before they complain massively about any problems.
In addition, the system should help analyze consumer behavior over longer periods - for this, it will connect their emotions with how much they spend and what they buy. The client’s biometric data is matched with the data of his transactions to detect changes in consumer habits due to dissatisfaction.
Earlier, in 2015, Walmart had already tried to introduce a facial recognition system - to prevent theft in stores - but the experiment eventually failed . The system was considered unprofitable, how many thieves could be detected with its help, Walmart management did not report.
3. X5 Retail Group: the fight with queues at the box office
The largest Russian retailer - X5 (Pyaterochka, Karusel, and Perekrestok) - also began testing face recognition technologies in 2017. According to representatives of the company, the technology should , in particular, help reduce waiting in queues at the box office and optimize the retail space.
How it works: remember the signs at the checkout - “If there are more than five people in the queue, call the number ...”? The face recognition system does the same thing automatically: if the queues of more than five customers, and some of the store’s cash desks do not work, the director receives an alert: “We need to open a cashier.” And with the help of video analytics, you can find out where more people go to the store, what they pay attention to, in order to properly position the goods and promotional materials.
At the same time, facial recognition systems do not store photos, explained Vasily Gromov , director of information technology at the Karusel retail chain (part of the X5 Retail Group). The system builds a function that determines the location of points on the face - cheekbones, eyes, ears, and so on (face hash). The next time, getting in the field of view of the cameras connected to the module for recognition, the system checks the image with the hash directory and gives a signal about who is in the video. Vasily Gromov also pointed out that “from the point of view of the law, a person’s hash is not personal information, since it does not allow identifying a person without additional information”. In the official blog of X5 on Habré, the company recently wrote that it continues to compare technologies for recognizing faces from different companies.
4. "Dixie": women - cosmetics, men - beer
Source: AddReality official site.
In Dixy, facial recognition technologies were decided to be tested simultaneously with the X5 Retail Group, but they chose another developer, AddReality. The goal was to determine the sex and age of customers and implement targeted advertising in the cashier zone and in the sales area. Dixy began testing biometric identification at Victoria stores and its headquarters in 2017.
How it works: On large screens installed in sales areas, a camera is mounted to recognize the face of a person who, passing by, draws attention to the screen.
“It is possible to determine whether a man is a woman or a woman, an approximate age, and taking into account these parameters, based on the existing pool of advertisers, the most suitable advertisement is broadcast on the monitor. For women, this could be an advertisement for cosmetics, and for a middle-aged man, for example, beer, ” explained Dixy's IT director Vladimir Muravyov.
5. Yandex: special advertising for "bearded" and intellectuals in pharmacies
Source: Yandex
Yandex in June 2018 launched the sale of targeted advertising on screens using the face recognition system at Asna pharmacies. At the core is the same technology that has already been used in Victoria stores, supplemented by the ability to select the most appropriate advertising for the user using a search engine.
How it works: the system consists of a screen and a camera that recognizes the viewer: his age, gender, appearance features - for example, glasses or a beard, and other characteristics. After that, with the help of Yandex.Direct, the most appropriate advertising for such a client is displayed on the screen.
According to a Yandex representative, the system can independently determine the types of several people in front of the screen, and the advertiser will pay only for targeted impressions. If there are three people in front of the screen, only two of which are advertising relevant, the system will count two impressions.
In Addreality, it was noted that the system “uses only impersonal user data, does not identify individual customers, does not use personal data, and does not record video of customers passing by”. The experiment began with 70 screens, another 2000 will appear in Asna pharmacies in case of successful testing of the system.
Learn more about how to use facial recognition, neural networks, and machine learning in business at the AI School intensive course.. Course speakers from Microsoft, “Nanosemantics” and Comparex LLC will tell you how to use different types of AI and what tools are available for this. The nearest intensive will take place on November 17 - December 15.