Amazon seeks to remove barcodes from inventory tracking
Amazon is researching a process to automate identification of items in its supply chain.
The e-tail behemoth is researching how to automate inventory identification using multimodal identification, or MMID. This process uses multiple modalities of information, such as extracting the appearance and dimensions of an item from an image of that item, to automate identification.
Amazon is already piloting MMID systems in fulfillment centers in Hamburg, Germany and Barcelona, Spain, where it is being used on conveyor belts to flag trays with what Amazon calls virtual-physical mismatches, or instances where the items in a tray don’t match the ones listed by the inventory system.
Amazon initially piloted MMID technology in a fulfillment center in Szczecin, Poland, with a camera positioned above a single conveyor line taking pictures of “singulated” trays — trays that contain only one item.
The first step of developing the MMID system was simply to pictures of products as they moved along conveyor belts in fulfillment centers, building up a library of images. Each image was then translated into a descriptive list of numbers, or a vector. The item’s dimensions also became a vector.
Researchers then developed machine learning (ML) algorithms to extract those vectors and to match them with the corresponding vectors of candidate items. Initial experiments produced match rates of 75% to 80%, according to Amazon.
The algorithm does not need to match an item against Amazon’s entire catalogue of hundreds of millions of products, currently an impossible task. Instead, the algorithm only has to match an item against the contents of a single tote.
In addition, cameras are continually adding to the library of images with each item that passes by. In the future, Amazon says it may integrate MMID into other components of its fulfillment process.
“Our north star vision is to use this in robotic manipulation,” Nontas Antonakos, an applied science manager in Amazon’s computer vision group in Berlin who led the MMID team when the concept was initially developed and deployed, in a corporate blog post. “Solving this problem, so robots can pick up items and process them without needing to find and scan a barcode, is fundamental. It will help us get packages to customers more quickly and accurately. And MMID is a cornerstone for achieving this.”
Amazon automates the supply chain
Amazon employs more than a dozen types of robotic systems in its supply chain facilities around the world, including sort centers and air hubs. In June 2022, Amazon announced “Proteus,” its first fully autonomous mobile robot. Proteus moves autonomously through Amazon’s fulfillment and sort facilities using advanced safety, perception, and navigation technology developed by Amazon.
Amazon also introduced Cardinal, a robotic lifting arm that uses artificial intelligence (AI) and computer vision to efficiently select one package out of a pile of packages, lift it, read the label, and precisely place it in a cart.
In addition, to reduce the need for employees to reach up, bend down, or climb ladders when retrieving items, the company is developing a robotic Containerized Storage System designed to deliver products to employees in a more ergonomic manner. The solution determines which pod has the container with the needed product, where that container is located in the pod, how to grab and pull the container to the employee, and how to pick it up once the employee has retrieved the product.
Most recently, the retailer acquired Belgium-based Cloostermans, which designs and manufactures mechatronics solutions, robotic technology that Amazon will use to help move and stack heavy palettes and totes, or package products together for customer delivery.