[Python] What are schemes for detecting a region with a variable value, shape, size, and...

Currently, I am trying to develop a program to help me measure the size of a laser keyhole. To do this, I need to take grayscale .tif files, isolate the keyhole, and then measure the area. The area measurement part is easy, but I am having problems isolating the keyhole. For example, here's a picture of a keyhole:



The keyhole is the faint circle in the middle, not the bright spots in the bottom right.

This keyhole is never offset more than a dozen or so pixels from the target area, but its (color) value changes, it's shape changes drastically, and while its size is mostly consistent, this changes as well. On top of this, the environment around the keyhole is also constantly changing due to welding gases and reflections (such as those in the bottom right of the picture above).

An example of a keyhole very different than the one above:



Here, the keyhole is not only much brighter, but it is surrounded by brightly glowing spots, some of which are part of the keyhole, and others which are not.

I have written a script which is fairly capable of sorting out the bottom image, but it struggles with the top one:

import cv2
import numpy as np
import matplotlib.pyplot as plt

#Import image and prepare it for processing
img = cv2.imread(r'C:\Users\J.Reilly\Desktop\Tiffs\Split_Video_A022_200fps_withRuler\2\20250522-144605905854_022.tif')
img = img[565:650, 450:550]
img = cv2.medianBlur(img, 5)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

#Structural Element declaration
big_strel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (7,7))
small_strel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2,2))
mid_strel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))


edges = cv2.Canny(img,30,100)

#This block finds large hotspots and small hotspots, large hotspots are assumed to be a part of the keyhole, small
#hotspots are removed
_, highs = cv2.threshold(gray, 200,255,cv2.THRESH_BINARY)
large_highs = cv2.morphologyEx(highs, cv2.MORPH_OPEN, big_strel)
large_highs = cv2.dilate(large_highs, big_strel)
highs = highs * cv2.bitwise_not(large_highs)
highs = cv2.dilate(highs, big_strel)
highs = (highs + 1) % 2 #binary_not produced non-binary images
edges = edges * highs

#Combines edges into a larger mass
edges = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, big_strel)
edges = cv2.dilate(edges, small_strel)

#This block cuts a circle shape out of the image before applying the floodFill to ensure the corners are empty
mask2 = np.zeros(edges.shape, dtype=np.uint8)
center = (gray.shape[0] // 2, gray.shape[1] // 2) # Center of the image
radius = 40
cv2.circle(mask2, center, radius, 255, -1) # 255 for white, -1 for filled circle
edges[mask2==0] = 0

#This section inverts "edges", then floods the image and inverts it to remove major internal voids
edges = (edges + 1) #This line inverts edges
cv2.floodFill(edges, None, (0,0), 255)
edges = cv2.bitwise_not(edges)
edges = cv2.morphologyEx(edges, cv2.MORPH_OPEN, big_strel)

#This section attempts to find the largest contour left, which hopefully is the keyhole
biggest_mask = np.zeros(gray.shape,np.uint8)
contours, hier = cv2.findContours(edges,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
if len(contours)>0:
biggest = max(contours, key = cv2.contourArea)
cv2.drawContours(biggest_mask, [biggest],0,1,-1)


#Display
plt.subplot(141),plt.imshow(img,cmap = 'gray')
plt.title('Original Image'), plt.xticks([]), plt.yticks([])
plt.subplot(142),plt.imshow(edges,cmap = 'gray')
plt.title('Edge Image'), plt.xticks([]), plt.yticks([])
plt.subplot(143),plt.imshow(highs,cmap = 'gray')
plt.title('Highs Image'), plt.xticks([]), plt.yticks([])
if len(contours)>0:
plt.subplot(144),plt.imshow(biggest_mask,cmap = 'gray')
plt.title('Biggest Image'), plt.xticks([]), plt.yticks([])
plt.show()


It produces the following figure when used on the second image:



(Here, there were not contours left beside the keyhole, so edge and biggest are the same)

However, on the first image, it produces this, as it is unable to find the keyhole, and it also erroneously picks up a hotspot.

I have also tried using cv2.inRange to group by value, but after testing all day, I couldn't find a way to make it work, since without edge detection, cases such as image 2 cannot be properly detected.

Continue reading...