[Python] How do to camera calibration using charuco board for OpenCV 4.10.0?

I am trying to do a camera calibration using OpenCV, version 4.10.0. I already got a working version for the usual checkerboard, but I can't figure out how it works with charuco. I would be grateful for any working code example.

What I tried: I tried following this tutorial: https://medium.com/@ed.twomey1/using-charuco-boards-in-opencv-237d8bc9e40d

It seems that essential functions like: cv.aruco.interpolateCornersCharuco and cv.aruco.interpolateCornersCharuco are missing. Even threw the documentation states an existing Python implementation, see: https://docs.opencv.org/4.10.0/d9/d6a/group__aruco.html#gadcc5dc30c9ad33dcf839e84e8638dcd1

I also tried following the official documentation for C++, see https://docs.opencv.org/4.10.0/da/d13/tutorial_aruco_calibration.html

The ArucoDetector in Python doesnt have the detectBoard method. So its also impossible to follow this tutorial in total.

But I guess from a hint in the documentation that the functions used by Medium are deprectated? But in no place marked as “removed”!

I already got the markers detected:



But then getting the object and image point fails:

`object_points_t, image_points_t = charuco_board.matchImagePoints( marker_corners, marker_ids)`


Any help or working code would be highly appreciated.

P.S.: My output of the “detectMarkers” method seems valid. Detected corners are of type

std::vector<std::vector<Point2f>.


(So translated to python an Array of Arrays containing the 4 points made by 2 coordinates each.) ID’s are

std::vector<int>


so in Python a list of Integers.

So I guess the python function “matchImagePoints” gets what it wants!

The marker detection seems succesful. I already tried changing the corner array: The detectMarkers method returns a tuple. I used the folloqing code to create the desired array of shape (X, 4, 2). (X beeing the number of deteced markers. Each has 4 corners with 2 coordinates x and y.)

marker_corners = np.array(marker_corners)
marker_corners = np.squeeze(marker_corners)


So I have the following:

marker_corners = [
[[8812. 5445.]
[8830. 5923.]
[8344. 5932.]
[8324. 5452.]],

[[7172. 5469.]
[7184. 5947.]
[6695. 5949.]
[6687. 5476.]],

[[3896. 5481.]
[3885. 5952.]
[3396. 5951.]
[3406. 5483.]],
...
]

marker_ids = [
[11],
[27],
[19],
...
]


Both, Passing the original return I get from

detector.detectMarkers


into the function and passing my modified array are failing. (Also not using squeeze and inputting a (X, 1, 4, 2) array fails!)

I can't get further anymore.

Minimum working example:

Use this picture:

import cv2 as cv
import numpy as np

image = cv.imread("charuco_board.png")
im_gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(11, 8),
squareLength=500,
markerLength=300,
dictionary=charuco_marker_dictionary
)

# Initial method of this question:
params = cv.aruco.DetectorParameters()
detector = cv.aruco.ArucoDetector(charuco_marker_dictionary, params)
marker_corners, marker_ids, rejected_candidates = detector.detectMarkers(im_gray)
marker_corners = np.array(marker_corners)
marker_corners = np.squeeze(marker_corners)

# Using cv.aruco.CharucoDetector as pointed out in the comments.
detector_charuco = cv.aruco.CharucoDetector(charuco_board)
result = detector_charuco.detectBoard(im_gray)
marker_corners_charuco, marker_ids_charuco = result[2:]

# Compare the two results
assert (marker_ids == marker_ids_charuco).all() # Detected ID's are identical.
# assert (marker_corners == marker_corners_charuco).all() # There seems to be a difference.
print(marker_corners[0:2], marker_corners_charuco[0:2]) # They seem to be in a different order.

# Proof of the different order statement:
def reshape_and_sort(array):
array_reshaped = array.copy().reshape(-1, 2)
return np.array(sorted(array_reshaped, key=lambda x: x[0]**2 + x[1]**2)) # Using geometric distance between each point and the point (0, 0). Leaving out the square.

marker_corners_reshaped = reshape_and_sort(marker_corners)
marker_corners_reshaped_charuco = reshape_and_sort(np.array(marker_corners_charuco))
assert (marker_corners_reshaped == marker_corners_reshaped_charuco).all()


# Trying with new resutls: # Still fails!
try:
object_points_t, image_points_t = charuco_board.matchImagePoints(
marker_corners_charuco,
marker_ids_charuco
)
except cv.error as err:
print(err)

Continue reading...