QR code looking patterns are a common occurrence in robotics. We see them used by various robotics companies, from Miso Robotic’s flippy and Agility Robotic’s Digit to Brain Corp’s AMRs:
While these images all look like QR Codes, they are not. Only one of the above images is a QR code, the others are ArUco codes!:
What are ArUco Codes?
The name ArUco is derived from the University of Cordoba in Spain, where the ArUco library was initially developed. The C++ library allows detecting and recognizing square markers (also known as ArUco markers) in images and videos. ArUco Codes are a specific type of fiducial marker, which in robotics is primarily used to provide reference points that robots can use to determine their position and orientation relative to the markers and to calibrate cameras. Different fiducial markers exist, but ArUco are common. Marker appearance need to be unique in all orientations to accurately know a given markers pose!
The distinctive feature of ArUco codes is that each code contains a unique pattern that can be easily identified by computer vision algorithms. This allows robots or other devices equipped with cameras to accurately determine the pose (position and orientation) of the code relative to the camera. By detecting multiple ArUco codes within the camera’s field of view, robots can localize themselves within an environment by calculating their camera position relative to the codes.
Why are ArUco codes used in robotics?
Because the image faces geometric distortion when the camera is at an angle compared to when it is viewed without an angle, if we know how the original image looks (i.e the original ArUco marker shape), we can compute the angle of the camera /marker using the distortion of the image:
Robot localization is important because the robot needs to know where it is in the environment relative to its internal map representation of the world. If an ArUco code existed in a known location in the real world, the robot can use this to be confident in its location within its own map!
How are ArUco codes detected?
- Apply Adaptive Thresholding to find the borders.
- Remove small borders with only a few points.
- Simplify the shape of the borders to rectangles with four corners.
- Arrange the corners of the rectangles in a specific order and remove rectangles that are too close to each other.
- Remove perspective distortion to make the rectangle look straight, identify the internal code of the marker
Overall, ArUco codes are a common form of fiducial marker used by robots and computer vision algorithms to understand where a camera/robot is relative to the marker in the environment, helping with localization so the robot knows where it is and camera calibration! To learn more about how robots user cameras to see, see our post on how robots see.