[Python] Extracting the colors on the face of a colorless rubick's cube

I am tasked with the problem of finding the colors on the face of a stickerless rubick's cube using OpenCV but I can't figure out what the best way might be to do so.

I tried to look at various solutions for the same problem online but most of them deal with canny edge detection which works great if the rubick's cube has a face with stickers on black background like this one:



However, this fails miserably when presented with faces of a stickerless rubick's cube like this one:



So my question is, how do I use OpenCV in order to detect the colors on the faces for this stickerless rubick's cube?

This is what I have tried till now:

import cv2 as cv
import numpy as np
from google.colab.patches import cv2_imshow


image = cv.imread('cube.png')


grey_frame = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
noiseless_frame = cv.fastNlMeansDenoising(grey_frame, None, 20, 7, 7)
blurred_frame = cv.blur(noiseless_frame, (3, 3))
canny_frame = cv.Canny(blurred_frame, 30, 60, 3)
dilated_frame = cv.dilate(canny_frame, cv.getStructuringElement(cv.MORPH_RECT, (9, 9)))

contours, _ = cv.findContours(dilated_frame, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)

square_contours = []

for contour in contours:

approx = cv.approxPolyDP(contour, 0.1*cv.arcLength(contour, True), True)

if len(approx) == 4 or True:

x, y, w, h = cv.boundingRect(approx)
ratio = float(w) / h
area = cv.contourArea(approx)

if ratio >= 0.8 and ratio <= 1.2 and w >= 30 and w <= 80 and area >= 900:

square_contours.append({"x": x, "y": y, "w": w, "h": h})


new_img = image.copy()
for contour in square_contours:

x, y, w, h = contour["x"], contour["y"], contour["w"], contour["h"]
cv.rectangle(new_img, (x, y), (x + w, y + h), (0, 255, 0), 2)


cv2.imshow(new_img)


However, this fails to detect the cubes on the faces of a stickerless cube most of the times.

Furthermore, sometimes the boundary between two adjacent cubes which have the same color is not very distinct. This leads to it detecting both of them as part of the same contour.

Continue reading...