Amazon introduces Lens Live, an AI-powered visual shopping tool
Amazon is expanding the capabilities of its mobile image-based shopping solution.
The online giant is introducing Lens Live, a new artificial intelligence-based feature of its Amazon Lens tool, which lets app users search for products based on photos from their device’s camera roll. When customers with Lens Live open Amazon Lens, the Lens camera will instantly begin scanning products and show top-matching items in a swipeable carousel at the bottom of the screen that is designed to enable quick comparisons.
Customers will also now have the ability to tap an item in the camera view to focus in on a specific product, add products directly to their cart by tapping the "+" icon, and save to their wish lists by tapping the heart icon, all without leaving the camera view.
In addition, Amazon has integrated its Rufus generative AI-based expert shopping assistant into the Lens Live user experience. While in the camera view, customers will now see suggested questions and quick summaries of what makes a product stand out. These conversational prompts and summaries appear under the product carousel.
[READ MORE: Amazon introduces new visual search features in shopping app]
Tens of millions of U.S. Amazon customers now have access to Lens Live and it will roll out to all Amazon customers nationwide in the coming months. Customers can still use traditional Lens options such as taking a picture, uploading an image, or scanning a barcode if they prefer.
How it works
Lens Live runs on Amazon Web Services-managed Amazon OpenSearch and Amazon SageMaker services to deploy machine learning models at scale. It employs a lightweight computer vision object detection model running on-device to identify products in real time as customers pan their cameras across scenes or focus on specific items.
This real-time detection is designed to identify the primary objects automatically. Lens Live uses a deep learning visual embedding model to match the customer’s view against billions of Amazon products, retrieving exact or highly similar items. Lens Live also now leverages the Rufus large language model to offer relevant questions and answers and enable product discovery.
"We’ll continue to look for ways to build on the convenience of searching and shopping with Amazon Lens, helping customers find and shop for the items they need and want that much faster," said Trishul Chilimbi, VP and distinguished scientist, stores foundational AI, Amazon, in a corporate blog post.
