Amazon bases ‘Just Walk Out’ on AI innovation
Amazon is revealing some of the next-gen technologies supporting its “Just Walk Out” frictionless shopping experience.
In a new corporate blog post, the e-tailer explains how artificial intelligence (AI), along with proprietary computer vision and machine learning (ML) technology, make its Just Walk Out autonomous checkout solution possible. Customers can shop the store, pick out they want, and skip the checkout when they’re done.
Knowing what’s going on
Inside a Just Walk Out-enabled store, several specialized cameras mounted to the ceiling provide a view of the entire store area. Jon Jenkins, VP of Just Walk Out technology, Amazon, said in the blog post that a combination of computer vision and machine learning allows the system to know “who takes what and charge them correctly when they walk out.”
“Our tech is able to distinguish shoppers from one another, without tracking physical attributes beyond a single store visit,” Jenkins said. “Just Walk Out technology detects when a shopper’s hand interacts with a product on the shelf. When that happens, machine learning algorithms make sure the correct item is added to the virtual cart—all without any specific knowledge about the person.”
How it works
Just Walk Out technology operates independently from Amazon One palm-based payment (which is offered as a payment method in some Just Walk Out-equipped stores), and doesn’t use or collect any biometric information to track the identity of shoppers.
Instead, the solution links a customer with their payment instrument. When shoppers enter the store, the technology assigns them a temporary numeric code, which serves as the shopper’s unique digital signature for that shopping trip.
The system preserves the code throughout the shopper’s time in the store. When they exit, the code disappears, and if they come back, they get a new code.
In every image, the Just Walk Out solution has a number of pixels that belong to a person. If a shopper takes off their jacket, for example, their signature will update as they move around the store without collecting any biometrics.
The technology can also track groups of shoppers. If more than one person enters the store with a single credit card, the system tracks each individual, but associates the group with the same payment instrument. When the group walks out, the system knows which shoppers were using the same card and generates one receipt for the collective transaction.
Just Walk Out technology uses object recognition to accurately identify different types of merchandise. If an item is put back in the wrong place by mistake, the system adjusts accordingly and alerts a store attendant to put the misplaced item back in the right place.
For smaller, harder-to-see products, like chewing gum or lipstick, weight sensors on the shelf detect when customers pick something up. A fusion of input from the weight sensor, known as a “load-cell sensor,” on the shelves and the cameras helps the system figure out what item was picked up by the shopper.
Development with generative AI
Amazon research teams used generative AI to help develop Just Walk Out technology by creating photorealistic sets of synthetic data, such as video clips of synthetic shoppers performing similar activities, to train the system to make independent decisions. The e-tailer used a type of generative AI called generative adversarial network (GAN) to create the synthetic data for training the technology.