“The article delves into the development of a fascinating project that involves object detection, tracking, and control of a mechanical arm unit.
Introduction
When we saw a video on YouTube of someone using a robotic arm to achieve object tracking, it deeply inspired us. We became very interested in this project and decided to independently develop a similar program.
Our objective is to develop a precise object recognition and tracking robotic arm system that can effectively contribute to practical applications. This project involves a multitude of technologies and algorithms, including aspects such as visual recognition, hand-eye coordination, and mechanical arm control.
myCobot 280 Jetson Nano
The robotic arm used for the operation is the myCobot280-Jetson Nano. This is a small 6-axis robotic arm produced by Elephant Robotics, with Jetson Nano as the microprocessor, ESP32 as the auxiliary control, and a UR collaborative structure. The myCobot280 Jetson Nano has a body weight of 1030g, a payload of 250g, a working radius of 280mm, and a compact and portable design. Despite its small size, it is powerful and easy to operate, capable of collaborating with humans and working safely.
Jetson Nano
Jetson Nano is an embedded artificial intelligence computer introduced by NVIDIA, featuring a powerful NVIDIA Maxwell GPU and a quad-core ARM Cortex-A57 processor. Jetson Nano supports various artificial intelligence frameworks and tools, such as TensorFlow, PyTorch, Caffe, and MXNet, among others. In addition, Jetson Nano has a variety of input and output interfaces, including HDMI, USB, GPIO, and more, making it convenient for developers to connect and control hardware.
Due to its powerful computing performance and its design specifically for artificial intelligence development, Jetson Nano supports various deep learning frameworks such as TensorFlow, PyTorch, and Caffe, making it more convenient for developers to develop AI applications. As a result, Jetson Nano has become an ideal platform for developers to develop AI applications.
Process
The following image is a flowchart of the project development process.
Capture the target
Before beginning development, we conducted some research and experiments. We used a camera to capture images of objects and utilized the OpenCV library for recognition. We attempted various methods, but object recognition required machine learning for the target we wanted to identify, which would increase the project development time. Ultimately, we decided to use aruco codes for identification, which allowed us to quickly capture the aruco codes and proceed to the next stage of development.
Hand-eye calibration
Hand-eye calibration refers to the process of determining the position and orientation of the robot end effector (such as a mechanical arm) relative to the robot base coordinate system in the field of robotics. This process involves pairing the robot end effector with a camera and then determining its position and orientation in the robot base coordinate system by capturing its position and orientation in the camera’s field of view.
Hand-eye calibration typically involves a series of movements between the robot end effector and the camera to collect enough data to calculate the transformation matrix between them. This transformation matrix describes the position and orientation of the robot end effector relative to the camera, which can be used to control the robot’s motion and accurately perform the required tasks.
In “eye-to-hand” hand-eye calibration, the camera is considered a stationary observer (“eye”), while the robot end effector is considered a moving object in the camera’s field of view (“hand”). As the robot end effector moves around the camera, a series of images are collected that contain information about the end effector’s position and orientation at different locations and orientations. By analyzing these images, the position and orientation of the robot end effector relative to the camera can be calculated, completing the hand-eye calibration.
The following is the code for processing the coordinate transformation data.”