Shop of the future. How the first supermarket without cash registers works in America

    In 2016, Amazon tested a “store without cash desks and queues” on its employees. And now such a supermarket was first opened to the public. It is equipped with cameras and sensors that look at what the buyer takes, and when you exit the store, money is automatically debited from the card. The opening was expected in early 2017, but the technology had to be honed an extra year. It was difficult for her to distinguish many buyers of similar builds, and when people came with children during the test period, they caused chaos by stacking goods incorrectly. But now, after five years of work, Amazon says the technology is ready, and "will be a new achievement in computer vision and machine learning."




    Before that, there were already similar projects - for example, Moby Mart , which was tested in China in 2017, and a “cafe without personnel” from Alibaba. But with Amazon, the system works on a completely different scale, pleases (quite numerous) customers, promises to be profitable, and is ready to expand. And we are watching how all this works, and why millions of people have taken up this technology in the USA.



    Big Brother is following


    At the entrance to the store you are greeted by rows of turnstiles. It may seem like you are going down to a metro station. You can only go through the turnstile using your smartphone. Access is opened when it scans the QR code generated by your Amazon Go application . At this point, the Amazon account is tied to your physical person, and cameras (THOUSAND cameras) begin to track your every action.


    The Amazon Go app is associated with an Amazon account, and money for all purchases will be debited from there. A couple of minutes after you leave the store, you will receive a receipt on your smartphone with what you bought. If suddenly the system entered something wrong, you can specify it, and the money will be returned back to your account. It seems that the system is vulnerable to those who want to cheat it, but Amazon representatives say that they have countermeasures for this. Which ones are not specifically reported to reporters, but the simplest thing you can think of is if a person constantly writes everything off incorrectly, Amazon Go is simply blocked for him. Like, why are you continuing to go to the store if you didn’t have this with him?




    The store itself, if you do not look up, looks like a very ordinary small American supermarket. Clean, spacious enough, with a maximum of wood in the interior (like Whole Foods) and shelves for chips, drinks, bread, salads, vegetables ... The most noticeable difference is that Amazon Go has no trolleys and shopping baskets. They simply are not needed: visitors immediately put goods in their bags (or pockets). At this moment, things are entered into the "virtual basket". Big Brother remembers what you took from the shelf. If then you decide not to buy the product and put it back, even in another place, the system will take this into account and automatically delete the record from your memory.


    And what about those who go shopping with the whole family? Here Amazon came up with two solutions. First, anyone can sign up for your Amazon Go account from their smartphone. Secondly, you can skip the whole family forward on the turnstile using your device. The result is the same: the system remembers everyone, and everything that they take with them is combined in a common account. If you are shopping with a friend and everyone has their own Amazon Go account, money is read from the person who takes the goods from the shelf.



    To the exit


    It is estimated that in the store you can take all the products that customers are used to. There are shelves with milk, yogurt, cookies, pastries, snacks. A separate section for the Whole Foods organic food chain (which Bezos bought last year for $ 13.7 billion), and a section with Fresh Meal Kits , sets launched on Amazon this summer, letting you cook for two at home. The store also has its own kitchen. You can take a soup or a sandwich or have a bite to eat in a small cafeteria. Prices for everything are the same as everywhere else, not more expensive and not cheaper. Obviously, Amazon does not expect to “recapture” all the costs of the technology at one supermarket, and average prices will allow it to compare activity and customer satisfaction.




    At the back in the corner of the supermarket is the only place where the buyer needs to interact with the person: a section with beer and wine. There, an Amazon employee checks your ID to see if you are 21 years old. After that, you can grab alcohol and leave the store, as well as with any other product.


    The exit is carried out through the same turnstiles, this time without authorization or any action on your part. There are no sound or light signals either. Just go about your business. No queues. And then a check arrives on the smartphone.


    The first customers who tested the first supermarket without cash desks say that the sensations are very strange (more precisely - “dumb”). When you go out the door, it seems that you have stolen products from the store, and then it is difficult to get rid of this feeling. Perhaps this will be the main trick of Amazon Go: when a person does not feel like paying, he is inclined to spend much more.



    Chamber work


    System developers say that the store uses “computer vision, deep learning algorithms and a combination of sensory data from various sources” as part of the Just Walk Out technology. The main "source" - hundreds of specials. cameras mounted in the ceiling. They track every point of the store from several angles. Basically, these are ordinary RGB cameras, only with built-in Amazon development boards, thanks to which they independently perform basic tasks: track movement, find an object, try to distinguish it from neighbors. Nearby are slightly more advanced cameras, capable of measuring depth, detecting the flight time of the signal to each point in the image.


    When you stand in the supermarket and look up, you hardly see these cameras: just gray blocks built into the dark ceiling. This may seem like a design element. In a regular store, cameras stand out much more.




    The resulting images are sent to the "central processor", about which Amazon does not say anything. He does the main job: identifying who is who, in real time. What kind of person picked up what subject, what put back on the shelf and so on. This processor is the main feature of the whole project, without which nothing would have happened. It is difficult for the human eye to know which of the similar people, dressed in a gray T-shirt and blue jeans, took which yogurt from a dozen almost identical white packages. And here the system should determine this with lightning speed, faultlessly, and in a hundred places at the same time. Previously, on such a scale and at such speeds, nobody even wanted to take on the task.


    Amazon, as usual, does not reveal its secrets, but you can look at similar projects - to present how it all works. For example, the startup Standard Cognition has recently developed a machine vision system that tracks the movement of customers and identifies the products they take. And then, for example, sending this data to the smartphone of the security guard. It turns out pretty simple when compared to the “Just Walk Out” on Amazon Go. But the company expects to sell its development to supermarkets and large retailers. To become successful, it is enough for her to be bought by one network for 2-3 thousand stores.



    Testing Standart Cognition

    Bodega is working on the same idea with kiosks. Three to four shelves, five cameras, one customer at a time. For office, cafeteria, dining room at the university. Well, in the very same direction as Amazon, Aipoly is moving - trying to create "fully autonomous stores" without sellers and cashiers through computer vision .



    Demo showing that Aipoly works even when visitors transfer goods to each other

    The main problem is the accuracy and speed of data processing. In the end, probably, the student could have made the basis of the system in a few weeks if he has enough time and resources. But she would correctly handle, say, 80% of the cases. And to bring it to an acceptable 99.9%, millions of dollars and many, many years of work are needed.


    Moreover, the Amazon system, according to developers, does not use face recognition. The company did not want to be accused of violating privacy, and tried to avoid criticism to the maximum. The faces of customers are not even remembered. Instead, “Just Walk Out” uses other visual signals (gait, limb length, etc.) and also monitors your movement through the cameras so that each of them “takes over” you from the previous ones. A person is not required here: your person is never lost sight of, which means that you do not need to be re-identified.




    If one of the cameras breaks, or something contaminates its lens, this does not affect the operation of the technology. The system has already been tested without a part of the cameras, and with a poor view. According to the calculations of engineers, 10% of broken cameras are needed so that the quality of recognition of customers and their purchases in the store somehow decreases significantly. In addition, weight sensors in the shelves help the cameras. They know the exact weight of each product, so grabbing two yoghurts at a time, hiding one in your sleeve and not fooling the system, even if you are David Blaine.



    How to Avoid Theft


    There are no employees in the store, except for people filling the shelves. But Amazon is not worried about attempts to steal goods. Theoretically, if you know the exact weight of the product you want to steal, you can do it in the style of Indiana Jones. Pour sand into a bag, and put it on a shelf the very second when you pick up the product from there. And at the same time, cover yourself with some extra large cloak with a hood. But this requires a lot more skill than ordinary shoplifters. And Amazon Go doesn't sell expensive things. The skills of such homegrown Jones will be spent more efficiently somewhere in a bank or jewelry boutique.


    The fact is that most of us do not steal. This is unpleasant for us. And with small losses of money due to kleptomaniacs or the poor, ordinary shops also put up with it, taking them into account when drawing up business plans. Most of the thefts in supermarkets are carried out by the employees themselves , and another 34% of the lost goods are due to errors (or abuses) of management and suppliers.




    You can steal goods in a regular supermarket. He took what you want, tore off the label, put it in his pocket, went to the exit. At Amazon, on the contrary, it’s more difficult to make it, wherever you put the product, you can’t deceive the system. If some critical vulnerability is not found, Just Walk Out can be advertised as the most “theft-resistant” model of supermarket operations.


    Another thing is that there are errors in the technology, and they were noticed on the very first day. Tweets began to appear from people from whom the store did not withdraw money for a particular product. Amazon says they can take it as a gift from the supermarket. The percentage is quite low, and the company does not have the opportunity to completely eliminate such errors, so it does not even bother. And buyers are still happy with unexpected “gifts” and share their impressions online.


    The technology did not exist when Amazon started its project five years ago. Everything was created by ourselves, from scratch. Therefore, in the early days, when the supermarket is full to the cue ball, a couple of mistakes can be forgiven (the developers admit that the system is most difficult to track many people standing together as a group). The complexity of monitoring does not increase with an increase in square meters, you only need more cameras and more computing power. And the technology will continue to hone.



    A world without people


    The American working class is worried that soon there will be no place left for them. There is a joke that "soon Bezos will work in America, and the rest of the tasks will be performed by robots." There is some truth in every joke: Amazon eliminates the human factor to the maximum, if possible. In 2012, she bought Kiva Systems, a car manufacturer, for $ 775 million. Now in her warehouses there are about 50 thousand of their robots, similar to huge orange rumbas. They gradually crowd out storekeepers. There are about 3.5 million cashiers in the United States, and their work in the world with Amazon Go, it seems, is no longer needed. What to do with the freed labor force?




    Amazon says the problem is far-fetched. In their "store without cash registers and cashiers" in fact, as many people will work as in a regular market, just the customers will not see them. These are people who fill shelves with food, cooks, preparing sandwiches and dishes that customers take with them, and, of course, technicians who support the work of webcams and turnstiles in good condition. Bezos love to blame for the fact that he deprives people of work, but in fact, Amazon is now expanding at a frantic pace, hiring thirty to forty thousand people every year, and another hundred thousand for the holidays.


    But the work of the cashier in the near future, apparently, will really disappear. Movements in this direction are being made even in Russia , where fast foods and restaurants have begun to replace people with self-service kiosks, vending machines and other digital platforms. Accuracy and speed are increasing, and systems do not have to pay salaries, which allows them to pay for themselves in less than a year.



    Who needs it


    Opponents of the idea say that the total number of jobs will go down, and the rich will become richer again, and the poor will become poorer, so that new technology will not bring benefits to society. The ubiquitous observation, hundreds of cameras that monitor your every movement, are not to everyone’s liking too. And all for the sake of saving “a couple of seconds” when leaving the store. A huge amount of resources and brain power of smart people is spent on what already worked fine.


    Supporters of Just Walk Out note that the strategy will reduce costs and increase store throughput. Saving is not only on salaries, but also on cash collection, on the work of cash registers and payment terminals. Overhead costs are reduced, which means that the cost of goods in the store is reduced compared to competitors who have live cashiers.



    One of the main advantages of Amazon Go, the lack of queues, has not yet worked: a giant queue has lined up in a new store

    Another important point is that simplifying the payment procedure in the store according to statistics increases the size of the average check. The easier it is to part with money, the more goods a person buys. At Amazon Go, you don’t have to imagine how each of the things you bought will be touched and punched by a person at the checkout.


    Plus - as with the giant Bezos clock , walled up in the mountain for 10,000 years, in the process of developing new technology, you can find solutions to other problems.


    Пока что Amazon не объявлял о планах массового внедрения «Just Walk Out», однако многие отраслевые эксперты не сомневаются, что она произведет революцию в сфере услуг. Сотни тысяч американцев, переживающие, что тратят на очереди в Walmart и Costco больше времени, чем на сам сбор продуктов, уже воодушевлены технологией. Это может быть огромный шаг в сфере оффлайн-продаж, его сравнивают с изобретением тележки для покупок в 1930-е. Чего в итоге будет больше, пользы или вреда, – покажет время.



    P. S.


    Из Amazon и других американских магазинов можно заказывать товары прямо в Россию с Pochtoy.com. Цены на технику в США, как правило, намного ниже, а выбора больше. Стоимость доставки за 0,5 кг начинается от $8,99. А при регистрации с кодом Geektimes вам на счет начисляется $7.


    Also popular now: