Amazon pilots self-navigating robots in fulfillment centers
Amazon is testing next-generation robots that use artificial intelligence (AI) and computer vision to roam freely throughout fulfillment centers.
The e-tail giant currently utilizes a fleet of more than 500,000 robots in its fulfillment centers, performing tasks such as stocking inventory, filling orders, and sorting packages for delivery. These robots follow directions provided by cloud-based algorithms to navigate along a grid of encoded markers that restrict their movement and interactions with people.
However, Amazon is now piloting a new class of self-navigating robots that help fulfillment center associates perform tasks such as transporting oversized and bulky items across a facility’s floor, which can cover several dozen football fields and contains numerous obstacles, including moving people and objects.
The new robots have an AI capability Amazon scientists call semantic understanding, which is the ability to understand and navigate a dynamic, three-dimensional environment and the objects in it. The robots use LIDAR (light detection and ranging) and computer vision technology to collect data about its surroundings and then define what occupies different points in space.
To teach the robots semantics, Amazon scientists collected thousands of images taken by the robots as they navigated. Then, they traced the shape of each object in each image and labeled it. Data scientists used this labeled data to train a machine learning (ML) model that segments and labels each object in the cameras’ field of view, a process known as semantic segmentation.
Layered on top of the semantic understanding are predictive models that teach the robot how to treat each object detected. When it detects a pillar, for example, it knows that pillars are static and will always be there. Amazon is working on another model to predict the paths of the people the robot encounters, and adjust course accordingly.
“When the robot takes a picture of the world, it gets pixel values and depth measurements,” said Lionel Guegen, an Amazon Robotics AI machine learning applied scientist, in a corporate blog post. “So, it knows at that distance, there are points in space — an obstacle of some sort. But that is the only knowledge the robot has without semantic understanding.”
“The navigation system does what we call semantically aware planning and navigation,” said Siddhartha Srinivasa, director of Amazon Robotics AI. “The intuition is very simple: The way a robot moves around a trash can is probably going to be different from the way it navigates around a person or a precious asset. The only way the robot can know that is if it’s able to identify, ‘Oh that’s the trash can or that’s the person.’ And that’s what our AI is able to do.”
Currently, Amazon is deploying its new robots in a few fulfillment centers where they are performing a narrow set of tasks.
“Our work is improving the representation of static obstacles in the present as well as starting to model the near future of where the dynamic obstacles are going to be,” said Gueguen. “And that representation is passed down in such a way that the robot can plan accordingly to, on one hand, avoid static obstacles and on the other hand avoid dynamic obstacles.”
In June 2022, Amazon announced “Proteus,” its first fully autonomous mobile robot. Proteus moves autonomously through Amazon’s fulfillment and sort facilities using advanced safety, perception, and navigation technology developed by Amazon.
[Read more: Amazon debuts first fully autonomous mobile robot in supply chain]
The Proteus robot was built to be automatically directed to perform its work and move around employees, meaning it has no need to be confined to restricted areas. Amazon is initially deploying Proteus in outbound handling areas for GoCarts (manual wheeled package transports) in its fulfillment centers and sort centers. The company seeks to automate GoCart handling throughout the network, which it says will help reduce the need for people to manually move heavy objects.