DATE: 2021/03/18

SEER Will Show You How to Use Vision Technology to Unlock a New Collaborative Model of Auto Factory

With the continuous upgrading and implementation of smart manufacturing, more and more attention has been paid to smart handling and smart logistics in factories. The introduction of AI technology into various scenarios in factories where robots and AMR are deployed has gradually become a new trend. At this stage, how to deploy vision technology into applications more conveniently, practically, safely and stably will become the key point of combining AI with robot applications.

And RoboView, the smart AI logistics system developed by Shanghai Seer Intelligent Technology Corporation (SEER) has its own "insights" in solving these key points.

The visual applications of smart AI logistics system RoboView

Due to the complexity of the industrial scene, it is not easy to introduce cutting-edge visual technology into the work scene. The most critical point is how to easily implement the cutting-edge visual technology into practical applications and allow customers to quickly master the skills. Especially in 3D vision applications, it is required to accurately locate and evaluate objects in 3D scenes. However, due to the high difficulty of R&D and deployment at this stage, its application is often not as common as 2D technology. However, with the development of technology, 3D technology will eventually become the future trend and get more applications. The figure below shows the process of RoboView in a typical 3D application.

3D vision in the 3D capture task: The pose of the target needs to be recognized



For example, in AGV logistics and warehousing, in order to effectively manage the goods, 3D vision technology is needed to complete the handling and stacking tasks. However, due to the large differences in the actual factory environment (different lighting conditions), the scene changes and the large difference in the size of the goods (from a few hundred to several thousand millimeters) and large difference in the accuracy requirements for handling and stacking tasks (from millimeter-level to centimeter-level) and other reasons, it is necessary to formulate corresponding plans for specific customer scenarios, which will bring great difficulties for both R&D and deployment.

In order to make the deployment of such applications simpler and more efficient, SEER integrates its own developed vision algorithm library, camera library, and robot library into a vision application platform, so that you can flexibly choose and use the corresponding cameras and programs. For example, choose the camera that truly fits the application from hundreds or more produced by more than a dozen manufacturers, and directly select the corresponding algorithms and robots, so as to truly achieve the aim of acting according to actual circumstances.

This vision application platform is the RoboView, the smart AI logistics system developed by Shanghai Seer Intelligent Technology Corporation (SEER). Supported by its powerful algorithm capabilities, through one-click deployment, it not only reduces costs, but also greatly improves deployment efficiency.

The application library of RoboView

We all know that the emergence of visual technology is to solve practical problems. As far as the current workshop or factory is concerned, the biggest problem encountered is the low efficiency of storage location management, and RoboView can integrate location management more "smartly" into the system. For example, the AGV scheduling system can make the automated logistics system more streamlined, reduce management costs and improve management efficiency.

In fact, for the storage location management system implemented by RoboView, workers just need to deploy a few cameras and a vision server in the warehouse that needs storage location management and run the RoboView to complete storage location management. At this time RoboView will be responsible for all the tasks of storage location management, and it will be connected with the scheduling system to complete the storage location management tasks in "quiet", such as full/empty storage, out/in storage, error in storage and other rich management ability.

Schematic diagram of the storage location management system implemented by RoboView



In addition, in some dense storage areas, even if more sensors are installed on the automatic forklift, some dangerous situations cannot be avoided, such as the situation in the following figure:

Some dangerous situations that may be encountered in dense storage areas



As can be seen from the above figure, in the scene where personnel are mixed with automatic forklifts and AMR, RoboView can solve these problems in the unified management of equipment, personnel, and goods on some scenes for customers.

How is the RoboView developed?

RoboView is inspired by the RoboCup small group soccer robot competition. As the group that emphasizes team collaboration and confrontation most in RoboCup, the small group uses external visual services as the main way of sensing the position of the robot. The typical scenes are as follows:

The schematic diagram of the main sensing methods of the position of the RoboCup small group soccer robots



Through external visual perception and a unified brain, the robots in the scene are managed uniformly, so as to realize various complex collaborations and confrontations.

The competition site of world Robocup small group soccer robots



Starting from this, SEER has innovatively proposed RoboView, a smart AI logistics system, in addition to the SLAM-based navigation system.

1. RoboView architecture

RoboView is composed of a perception platform and perception nodes. As the core of the system, the perception platform is to provide four major parts including all visual services, necessary information for smart factories and logistics systems (including storage location management information, fleet tracking information and equipment operating status), forklift operation visual service request response and system safety monitoring information; Perception nodes include fixed or mobile image (2D)/point cloud (3D) acquisition equipment and marshalling.

The perception platform is an independent vision server, which mainly provides visual AI perception capabilities, and realizes the recognition, positioning and detection of targets in the perception area through 2D/3D vision technology. The perception platform only needs one input and one output definition (DOO, Definition Of Output) to complete a perception service. Therefore, it is a multi-task and independent information perception platform.

Perception nodes are deployed in a distributed manner and are divided into fixed nodes and mobile nodes. Fixed node refers to fixed installation of image (2D)/point cloud (3D) acquisition equipment. For example, ordinary surveillance cameras, ToF cameras, 3D lidar, etc. can be switched in as perception nodes; Mobile nodes refer to various types of visual image (2D)/point cloud (3D) acquisition equipment installed on mobile robots, which can be seamlessly connected with RoboView through the SRC core controller of SEER.

Perception nodes can be added to the SLAM map through one-stop implementation software, and the corresponding workflow can be configured, so as to realize the rapid setting and deployment of nodes. At the same time, the smart AI logistics system RoboView can work with the digital middle ground SEED of SEER to realize the deployment, planning and monitoring of various systems in the site, and realize the safety monitoring and operation planning of AMR and unmanned forklifts.

SEER SMART LOGISTICS SOLUTION



2. AI full perception technology

The AI full perception platform integrates a number of key technologies and provides a wealth of information for the logistics management system.

Enhanced display: Through several fixed perception nodes to "observe" in all directions, combined with multi-point vision registration, splicing, and fusion technologies, the real-time status of the entire warehouse can be seen at a glance, and AR enhancement information is added to the original surveillance video so as to make the on-site personnel more intuitive and clear about the equipment status.

Vehicle monitoring: Combine with the forklift's own information to achieve real-time positioning and tracking of each AMR and automatic forklift in the scene, capture and predict the moving trajectory of forklifts. Combine with the digital middleground SEED of SEER to timely send adjustment instructions to the forklifts deviating from the task and warning to the platform.

Target recognition and positioning: RoboView includes not only conventional visual perception algorithms, but also advanced deep learning algorithms, which greatly improves the anti-interference performance of the system. Target detection technology can detect target objects, such as cars or people, from the video stream in real time; 3D point cloud segmentation technology can directly extract target objects from the point cloud and provide important information such as size and location to make the algorithm more stable and reliable. In addition, through image semantic segmentation technology, it can also achieve one-time segmentation and extraction for warehouse control marks, road signs, and operating forklifts, providing another important information for the control platform.

Location Management: Through multiple fixed perception nodes deployed in the scene, the perception platform can obtain all the location information in the scene in real time, and use 2D recognition technology to identify and locate the effective information of the location, including but not limited to information such as whether there is goods, whether there is a car, whether there is foreign matter intrusion, etc., And it can publish this information (according to the configuration of the logistics system, information may be sent to the logistics system and the car). The logistics system or the forklift can make effective decisions based on this information, which greatly improves the efficiency of warehouse management.

Safety monitoring: It mainly involves real-time monitoring of all things' status in the scene, including: the intrusion monitoring of foreign objects (non-operating personnel and operators entering non-safe areas, forklifts leaving the preset task trajectory, non-operating forklifts, etc.), cargo status monitoring, obstacle detection in forklift route, monitoring of other interlocking equipment (such as large racks, elevators).

Real-time monitoring of all status of the robots in the scene