[Python] Similar object detection project not working as intended

I wrote a code to be able to identify similar/same objects. We have a set of stored images and we input an image of an object, so the code is supposed to box all the similar matches inside every image. This in my code:

import cv2
import numpy as np
import pathlib
import os
import sys
from PIL import Image
from sklearn.cluster import DBSCAN

def is_image(path):
if not os.path.exists(path):
sys.exit("\n\nThe path does not exist")
try:
with Image.open(path) as img:
img.verify()
print('\n\nThe path is valid, looking for similar images')
return True
except (IOError, SyntaxError):
sys.exit("\n\nFile exists but is not a valid image.")

# Image directory
img_path = 'pics/same'
data_dir = pathlib.Path(img_path)
extensions = ('.jpg', '.jpeg', '.png')
img_tpl = tuple(path for path in data_dir.rglob('*') if path.suffix.lower() in extensions)
print(img_tpl)

# Input image
input_img = input('\n\nEnter the image path: ')
is_image(input_img)

# Initialize ORB detector
ORB = cv2.ORB_create(nfeatures=2000)

# Read and resize input image
img1 = cv2.imread(input_img)
img1 = cv2.resize(img1, (250, 250)) # Resize to approximate logo size
gray1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
kp1, desc1 = ORB.detectAndCompute(gray1, None)

if desc1 is None or len(desc1) == 0:
print("No descriptors found in input image.")
sys.exit()

# Matcher
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=False)

# Matching with each stored image
for i in img_tpl:
img2 = cv2.imread(str(i))
gray2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
kp2, desc2 = ORB.detectAndCompute(gray2, None)

if desc2 is None or desc1 is None:
print(f"Skipping {i}, no descriptors found.")
continue

matches = bf.knnMatch(desc1, desc2, k=2)

# Ratio test
good_matches = [m for m, n in matches if m.distance < 0.80 * n.distance]

if len(good_matches) < 4:
print(f"Not enough matches for {i} - {len(good_matches)}/4")
continue

#DBSCAN
matched_pts = np.float32([kp2[m.trainIdx].pt for m in good_matches])
clustering = DBSCAN(eps=40, min_samples=2).fit(matched_pts)
labels = clustering.labels_

img_with_boxes = img2.copy()
h, w = gray1.shape
box_template = np.float32([[0, 0], [0, h], [w, h], [w, 0]]).reshape(-1, 1, 2)

for cluster_id in set(labels):
if cluster_id == -1:
continue

cluster_matches = [m for idx, m in enumerate(good_matches) if labels[idx] == cluster_id]
imges = cv2.drawMatchesKnn(img1, kp1, img2, kp2, [cluster_matches], None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
cv2.imshow("Matches", imges)
cv2.waitKey(0)

if len(cluster_matches) < 4:
continue

#Homography
pts1 = np.float32([kp1[m.queryIdx].pt for m in cluster_matches]).reshape(-1, 1, 2)
pts2 = np.float32([kp2[m.trainIdx].pt for m in cluster_matches]).reshape(-1, 1, 2)

H, mask = cv2.findHomography(pts1, pts2, cv2.RANSAC, 5.0)
if H is not None:
box_transformed = cv2.perspectiveTransform(box_template, H)
img_with_boxes = cv2.polylines(img_with_boxes, [np.int32(box_transformed)], True, (255, 0, 0), 3)
else:
print(f"Homography failed for cluster {cluster_id} in image {i}")

cv2.imshow("Detected Matches", img_with_boxes)
cv2.waitKey(0)

cv2.destroyAllWindows()



I used ORB to find all keypoints and bruteforce matcher to match keypoints and Lowe's paper ratio test to filter out noise to certain extent.

Then then later I added DBSCAN to cluster up matched points along with Homography to help identify the object and further filter out noise.

Everything seems to be alright with the code but when i tried to test it the results were not upto the mark, or they were rather way to bad. The program was not really working as expected.

After that I tried various ways trying to improve it but nothing worked. I also tried changing the default arguments of ORB/DBSCAN and others. But it was to no avail.

After long time of testing/editing code I figured that my code was just not giving enough matches for the 2 images were being shown.

Images have no issue. I confirmed that shortly after, and they were identifiable which I confirmed with the help of a code on github also for the same thing as me(they used SIFT, FLANN matcher, Meanshift for clustering and homography)

But just to be on the safer side I used other pair of pictures as well but in only 1 of the 5 pairs did i got satisfactory results(had some issues it it as well but did work)

So I think the issue is(probably) with my code but I am not able to figure it out can someone help me with it?





these are 2 of the images and as for the output it matches were not enough.

Continue reading...