Tokyo-based startup Telexistence announced this week that it will deploy NVIDIA AI-powered robots to restock the shelves of hundreds of FamilyMart convenience stores in Japan.
There are 56,000 convenience stores in Japan, the third highest density in the world. About 16,000 of them are operated by FamilyMart. Telexistence aims to save these stores time by offloading repetitive tasks such as stocking shelves of drinks onto a robot, allowing retail staff to tackle more complex tasks such as interacting with customers. clients.
That’s just one example of what Telexistence’s robots, which run on NVIDIA Jetson’s cutting-edge AI and robotics platform, can do. The company is also developing AI-based systems for warehouse logistics with robots that sort and pick packages.
“We want to deploy robots in industries that support daily human life,” said Jin Tomioka, CEO of Telexistence. “The first space we are tackling this issue is convenience stores – a vast network that supports daily life, especially in Japan, but faces a labor shortage.”
The company, founded in 2017, then plans to expand into convenience stores in the United States, which is also plagued by labor shortages in the retail sector – and where more than half of consumers report visiting at least one of the country’s 150,000 convenience stores. once a month.
Telexistence robots source from FamilyMart
Telexistence will begin rolling out its replenishment robots, called TX SCARA, to 300 FamilyMart stores in August — and aims to bring the autonomous machines to other FamilyMart locations, as well as other major convenience chains, in the coming years .
“Staff members spend a lot of time in the store’s back room, restocking shelves, instead of going out with customers,” Tomioka said. “Robotics as a service can allow staff to spend more time with customers.”
TX SCARA operates on a track and includes multiple cameras to scan each shelf, using AI to identify drinks that are running out and plan a path to restock them. The AI system can successfully restock drinks automatically more than 98% of the time.
In the rare event that the robot misjudges drink placement or if a drink spills, store staff do not need to abandon their task to get the robot running again. Instead, Telexistence has remote operators on standby, who can quickly resolve the situation by taking manual control through a VR system that uses NVIDIA GPUs for video streaming.
Telexistence estimates that a busy convenience store needs to restock more than 1,000 beverages per day. TX SCARA’s cloud system maintains a database of product sales based on name, date, time and number of items stored by robots during operation. This allows the AI to prioritize which items to restock first based on past sales data.
Reach Edge AI with NVIDIA Jetson
TX SCARA has several AI models under the hood. An object detection model identifies beverage types in a store to determine which belongs on which shelf. It is combined with another model that helps to detect the movement of the robot arm, so that it can pick up a drink and place it precisely on the shelf between other products. A third is for anomaly detection: recognizing whether a drink has fallen or is off the shelf. Another detects beverages that are low in each display area.
The Telexistence team used custom pre-trained neural networks as base models, adding synthetic and annotated real-world data to refine the neural networks for their application. Using a simulation environment to create more than 80,000 synthetic images helped the team augment their data set so the robot could learn to detect drinks in any color, texture or environment. lighting.
For training the AI models, the team relied on an NVIDIA DGX station. The robot itself uses two NVIDIA Jetson on-board modules: the NVIDIA Jetson AGX Xavier for edge AI processing and the NVIDIA Jetson TX2 module for transmitting video streaming data.
On the software side, the team uses the NVIDIA JetPack SDK for advanced AI and the NVIDIA TensorRT SDK for high-performance inference.
“Without TensorRT, our models wouldn’t run fast enough to effectively detect objects in the store,” said Pavel Savkin, robotics automation lead at Telexistence.
Telexistence has further optimized its AI models using half precision (FP16) instead of single precision floating point (FP32).
Learn about the latest in AI and robotics at NVIDIA GTC, online September 19-22. Registration is free.
