[Python] Instance Segmentation of Curved Surfaces

I have a render of four identical objects and need to segment the curved surfaces and classify them (associate similar ones with each other).

All the objects are positioned at the same angle relative to the viewpoint. The task essentially boils down to identifying and matching corresponding 2D patterns.

I want to do this without deep learning, using traditional computer vision techniques or algorithms.





So far, I’ve tried extracting contours with Canny, but the results are noisy, and I’m unsure how to improve them.

import cv2
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt

image = np.array(image)

bil = cv2.bilateralFilter(image, d=2, sigmaColor=75, sigmaSpace=300)
edges = cv2.Canny(bil, 0, 20)

contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

min_length = 25
filtered_contours = [cnt for cnt in contours if cv2.arcLength(cnt, True) >= min_length]

filtered_edge_image = np.zeros_like(edges)
cv2.drawContours(filtered_edge_image, filtered_contours, -1, (255, 255, 255), 1)

image = Image.fromarray(filtered_edge_image.astype(np.uint8))
image




I also calculated the gradient magnitude, which seems closer to what I need, but I can’t figure out how to completely fill the masks of the desired surfaces in white without including other surfaces.

import cv2
import numpy as np
from PIL import Image
from skimage.util import invert

img = cv2.imread("img.png", cv2.IMREAD_GRAYSCALE)
image = invert(img)

sobelx = cv2.Sobel(image, cv2.CV_64F, 1, 0, ksize=3)
sobely = cv2.Sobel(image, cv2.CV_64F, 0, 1, ksize=3)

magnitude = np.sqrt(sobelx**2 + sobely**2)
magnitude = np.uint8(magnitude * 255 / np.max(magnitude))

thresh = cv2.threshold(magnitude, 0, 255, cv2.THRESH_BINARY)

Image.fromarray(thresh)




At this point, I’m stuck.

1. 
   How do I get accurate segmentation masks?
   
   
2. 
   How can I classify masks of identical surfaces? Should I use affine transformations?
   

Continue reading...