[Python] Can I have a useful algorithm to find specifically 4 sides of a shape using cv2?

This is My code:

class PuzzlePieceDetector:
def __init__(self, binary_image: np.ndarray, original_image: np.ndarray, debug: bool = False):
self.binary_image = binary_image
self.original_image = original_image
self.debug = debug

def find_pieces(self, min_area: int = 500):
inverted = cv2.bitwise_not(self.binary_image)
contours, _ = cv2.findContours(inverted, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

if self.debug:
print(f"[INFO] Found {len(contours)} contours total.")

pieces = []
bounding_boxes = []
contour_image = self.original_image.copy()

for contour in contours:
area = cv2.contourArea(contour)
if area < min_area:
continue

# Get 4 corner points using rotated rectangle
rect = cv2.minAreaRect(contour)
box = cv2.boxPoints(rect)
box = np.intp(box)

if self.debug:
print(f"[DEBUG] Rectangle corner points: {box.tolist()}")

# Draw lines between the 4 corners
for i in range(4):
pt1 = tuple(box)
pt2 = tuple(box[(i + 1) % 4])
cv2.line(contour_image, pt1, pt2, (255, 0, 0), 2) # Blue lines

# Optional: Draw corner points
cv2.circle(contour_image, pt1, 5, (0, 255, 255), -1) # Yellow dots

# Crop the piece
x, y, w, h = cv2.boundingRect(contour)
piece_crop = self.original_image[y:y + h, x:x + w].copy()
pieces.append(piece_crop)
bounding_boxes.append((x, y, w, h))

# Harris corner detection
harris = cv2.cornerHarris(self.binary_image, 2, 3, 0.04)
harris = cv2.dilate(harris, None)


# Draw detected corners on binary image
corner_overlay = cv2.cvtColor(self.binary_image, cv2.COLOR_GRAY2BGR)
threshold = 0.01 * harris.max()
corner_points = np.argwhere(harris > threshold)
for y, x in corner_points:
cv2.circle(corner_overlay, (x, y), radius=4, color=(0, 0, 255), thickness=-1)

if self.debug:
print(f"[INFO] Extracted {len(pieces)} puzzle piece(s). Harris max: {harris.max()}, threshold: {threshold}")
print(bounding_boxes)

return pieces, bounding_boxes, contour_image, corner_overlay


Image I am testing on

For full code

I have tried many algorithms for this. I am currently using minRect, but the problem is that it is also looking at the bulges of the puzzle piece. As you can see, I have used harrisCorner detection, but it totally outlines the shape. When I change the values, It doesn't catch some of the main corners. When I used approxPolyDP, something like when I used minRect happened. Can I have some help with this?

Continue reading...