I have several images taken of a table setup with ArUco Markers. As end result, I would like to have a top-down coordinate system which defines the configuration of the markers on the tables. I managed to detect all the markers using the code: corners, ids = self.detect_aruco_markers(image) Afterwards, I used the real-life marker sizes to define the coordinates for the markers as well as their relative positions, and cluster them correctly to their tables. However, there is an issue which relates to the fact that the photos are not taken perfectly top-down. I have attached an example image below. The issue is that in the resulting coordinate system, markers are slightly rotated and moved from where they should be. For instance, markers that should be x- or y-aligned are not in the resulting coordinate system. To fix this issue, I wanted to use a perspective transformation, using a homography matrix. However, I am having trouble deciding which points exactly to use as "real world" reference points and source points for the cv2.getPerspectiveTransform method. I do not have access to the real world coordinates of the markers top-down, as this changes per input image. All that I know is that the markers are of fixed size (that is given) and that they all lie in the same plane, facing upwards. I tried something using the convex hull so far, but to no avail: # Flatten the list of corners and convert to a NumPy array src_points = np.concatenate(corners, axis=0).reshape(-1, 2) # Compute the bounding box of the detected markers x_min, y_min = np.min(src_points, axis=0) x_max, y_max = np.max(src_points, axis=0) # Define destination points for the perspective transform dst_points = np.array([ [0, 0], [x_max - x_min, 0], [x_max - x_min, y_max - y_min], [0, y_max - y_min] ], dtype=np.float32) # Ensure src_points has exactly four points by selecting the outermost corners if src_points.shape[0] > 4: # Compute the convex hull to find the outermost corners hull = cv2.convexHull(src_points) if hull.shape[0] > 4: # Approximate the hull to a quadrilateral epsilon = 0.02 * cv2.arcLength(hull, True) approx = cv2.approxPolyDP(hull, epsilon, True) if approx.shape[0] == 4: src_points = approx.reshape(4, 2) else: rect = cv2.minAreaRect(hull) src_points = cv2.boxPoints(rect) else: src_points = hull.reshape(-1, 2) elif src_points.shape[0] < 4: raise ValueError("Not enough points to compute perspective transform.") # Compute the perspective transform matrix matrix = cv2.getPerspectiveTransform(src_points.astype(np.float32), dst_points) # Warp the image using the transformation matrix width = int(x_max - x_min) height = int(y_max - y_min) warped_image = cv2.warpPerspective(image, matrix, (width, height)) Am I on the right track with this approach, and how should I proceed? Thanks for any help! Continue reading...
