[Python] How to undistort an image by generating cameraMatrix and distortion coefficients...

Here are the docs I was following: https://docs.opencv.org/4.11.0/d9/d6a/group__aruco.html#ga9952135f6bf4ec17894a103c238f6979

Running OpenCV 4.11.0

The size of each aruco marker is 0.5 inches x 0.5 inches, or 0.0127 meters wide. To load the image, get the marker corners and ids...

cameraMatrix = np.matrix([[3000, 0, cx],[0, 3000, cy],[0, 0, 1]]) # Initial values, should be calculated/populated by calibrateCameraAruco
distCoeffs=np.zeros((1,5)) # Initial values, should be calculated/populated by calibrateCameraAruco
marker_length = 0.0127 # meters = .5 inches
dict_used = cv.aruco.DICT_4X4_50
ardict = cv.aruco.getPredefinedDictionary(dict_used)
img = cv.imread("Cayman.jpg")
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
det = cv.aruco.ArucoDetector(ardict, ARUCO_PARAMETERS := cv.aruco.DetectorParameters())
corners, ids, rejected = det.detectMarkers(gray)


The values of corners and ids returned by det.detectMarkers(gray) are

>>> corners
(array([[[1023., 4862.],
[1023., 5080.],
[ 806., 5080.],
[ 807., 4861.]]], dtype=float32), array([[[2339., 4865.],
[2340., 5083.],
[2125., 5085.],
[2124., 4868.]]], dtype=float32), array([[[1030., 299.],
[1028., 511.],
[ 815., 511.],
[ 817., 298.]]], dtype=float32), array([[[2333., 308.],
[2333., 518.],
[2119., 519.],
[2119., 306.]]], dtype=float32))



array([[12],
[13],
[14],
[11]], dtype=int32)


In order to undistort an image, I need the camera matrix and distance coefficients. From a comment in this stackoverflow question, camera matrix and distance coefficients can initially be None when generating with cv.aruco.calibrateCameraAruco(...).

Docs on calibrateCameraAruco can be found here. The inputs required are:

1. corners: vector of detected marker corners in all frames. The corners should have the same format returned by detectMarkers (see detectMarkers).
2. ids: list of identifiers for each marker in corners counter number of markers in each frame so that corners and ids can be split
3. board: Marker Board layout
4. imageSize: Size of the image used only to initialize the intrinsic camera matrix.
5. cameraMatrix (can be None initially or empty and it will be populated by the function)
6. distCoeffs (same as 5. cameraMatrix)

For the Board object input in point 3 above, it requires

1. objPoints: array of object points of all the marker corners in the board
2. dictionary: the dictionary of markers employed for this board
3. ids: vector of the identifiers of the markers in the board objPoints is a vector of vector of Point3D float32, in meters. From this tutorial, object points can be initialized with the Z values set to 0. However they use cv.solvePnP which requires camMatrix and distCoeffs, how am I supposed to generate camMatrix and distCoeffs using calibrateCameraAruco? Using their construction, my object points look like:

ml = marker_length # = 0.0127 meters = 0.5 inches
obj_points = np.array([np.array([-ml/2, ml/2, 0], dtype=np.float32),
np.array([ml/2, ml/2, 0], dtype=np.float32),
np.array([ml/2, -ml/2, 0], dtype=np.float32),
np.array([-ml/2, -ml/2, 0], dtype=np.float32)], dtype=np.float32)



However I have four markers in frame, I can't reuse the same object points can I? How do I initialize object points so I can create a board object? I tried filling it with dumby values just to see if I could get some matrix values from calibrateCameraAruco, but I get this error:

dumby_obj_points_list = [ obj_points, obj_points+[4*marker_length, 2*marker_length, 0], obj_points+[0, -2*marker_length, 0], obj_points+[4*marker_length, -2*marker_length, 0] ]

board = cv.aruco.Board([objPoints], ardict, ids)



>>> cv.aruco.Board(dumby_obj_points_list, ardict, ids)

cv2.error: OpenCV(4.11.0) /build/source/modules/objdetect/src/aruco/aruco_board.cpp:163: error: (-215:Assertion failed) corners_mat.total() == 4 in function 'Board'


If I instead try and use a GridBoard object instead, with dumby separation values, I get:

dumby_gridboard = cv.aruco.GridBoard((2,2), marker_length, 2*marker_length, ardict)
>>> cv.aruco.calibrateCameraAruco(corners, ids, 4, dumby_gridboard, gray.shape, cameraMatrix, distCoeffs)

cv2.error: OpenCV(4.11.0) /build/source/opencv_contrib/aruco/src/aruco_calib.cpp:33: error: (-215:Assertion failed) nMarkersInThisFrame > 0 in function 'calibrateCameraAruco'


Even if I generate a GridBoard directly without taking a photo, and use that exact gridboard in cv.calibrateCameraAruco I get the same error, how come opencv detects 0 markers in frame?

The Input image is provided here: The result of cv2.detectMarkers is here (the green/blue/red outline trace is thin w.r.t the image)

Continue reading...