[Python] "Text Recognition Issues in License Plate Recognition Using YOLO and EasyOCR"

I am working on a license plate recognition system using YOLO for object detection and EasyOCR for text recognition. I’ve set up the following pipeline:

YOLO (Ultralytics YOLOv8) detects the license plate in the image. EasyOCR performs text recognition on the detected license plate. While the system correctly detects the license plate region, the OCR results sometimes produce incorrect or non-relevant text. For example:

OCR detects “اباز” (which seems to be an incorrect detection). OCR also detects “الراق”, which is a partial match to "العراق". For correct numbers like "٢٦٠٤٩٩", the confidence is high. I have attempted to use difflib.get_close_matches() to match the detected text to a list of known target words (e.g., cities like "العراق", "سلیمانی", "اربیل", etc.) but the system still produces incorrect matches like “اباز” with a low confidence score. What I’m trying to do:

Improve the accuracy of text recognition. Ensure that close matches (e.g., "الراق" to "العراق") are detected correctly. Filter out low-confidence text or non-relevant text. I’m looking for suggestions on:

Improving the text matching (e.g., using a better similarity measure). Improving OCR accuracy for Arabic text, especially when the detected text is close to my target words. Any other tips for improving the license plate recognition accuracy in this context.

import cv2 import easyocr from ultralytics import YOLO import matplotlib.pyplot as plt from difflib import get_close_matches

def show_image(title, image): """Display an image.""" if len(image.shape) == 3: image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) plt.imshow(image) plt.title(title) plt.axis('off') plt.show()

def detect_plate_with_yolo(image_path, model_path="yolov8n.pt"): """Detect the license plate using YOLO.""" # Load YOLO model model = YOLO(model_path)

# Read the image
img = cv2.imread(image_path)
if img is None:
raise FileNotFoundError(f"Image not found at {image_path}")

# Perform YOLO inference
results = model(img)

# Extract bounding boxes
detections = results[0].boxes.xyxy.cpu().numpy() # Bounding boxes
if len(detections) == 0:
print("No license plate detected.")
return None, img

# Crop the first detected license plate
x1, y1, x2, y2 = map(int, detections[0])
plate_img = img[y1:y2, x1:x2]

show_image("Detected License Plate", plate_img)
return plate_img, img


def perform_ocr_on_plate(plate_img): """Perform OCR on the license plate.""" reader = easyocr.Reader(['ar', 'en'], gpu=False)

# Preprocess plate image
plate_gray = cv2.cvtColor(plate_img, cv2.COLOR_BGR2GRAY)
plate_gray = cv2.resize(plate_gray, None, fx=2, fy=2, interpolation=cv2.INTER_CUBIC)

# OCR on the processed plate
results = reader.readtext(plate_gray, detail=1, text_threshold=0.4, min_size=10)

detected_text = []
for bbox, text, confidence in results:
detected_text.append((text, confidence))

return detected_text


def correct_text(detected_text, target_words, similarity_cutoff=0.9, confidence_threshold=0.7): """Correct detected text to the closest match from target words.""" corrected_results = []

for text, confidence in detected_text:
# Skip text with low OCR confidence
if confidence < confidence_threshold:
continue

# Find the closest match to the target words
closest_match = get_close_matches(text, target_words, n=1, cutoff=similarity_cutoff)

if closest_match:
corrected_results.append((closest_match[0], confidence))
else:
corrected_results.append((text, confidence)) # If no match, keep original text

return corrected_results


def recognize_plate(image_path, model_path="yolov8n.pt"): """Main function to recognize license plates.""" # Target place names target_words = ["العراق", "سلیمانی", "اربیل", "هەولیر", "کەرکوک", "دهوک"]

# Detect plate using YOLO
try:
plate_img, original_img = detect_plate_with_yolo(image_path, model_path)
except Exception as e:
raise RuntimeError(f"Error during YOLO detection: {e}")

if plate_img is not None:
try:
detected_text = perform_ocr_on_plate(plate_img)
except Exception as e:
raise RuntimeError(f"Error during OCR processing: {e}")

print("Detected License Plate Text:")
for text, confidence in detected_text:
print(f"Text: {text}, Confidence: {confidence:.2f}")

# Correct detected text
corrected_text = correct_text(detected_text, target_words)

print("\nCorrected License Plate Text:")
for text, confidence in corrected_text:
print(f"Match: {text}, Confidence: {confidence:.2f}")

return corrected_text
else:
print("No license plate detected.")
return None

Test the function


image_path = r'C:\Users\ZETTA-\Downloads\IBCvska.webp' # Replace with your image path try: result = recognize_plate(image_path) if result: print("Recognition Complete.") except FileNotFoundError as e: print(f"Error: {e}") except Exception as e: print(f"Unexpected error: {e}")

What did you try?

I’ve set up the license plate recognition pipeline using YOLOv8 for license plate detection and EasyOCR for text recognition. Here’s what I have done so far:

YOLOv8 correctly detects the license plate region in the image. EasyOCR performs OCR on the detected region and returns text (such as "٢٦٠٤٩٩", "اباز", "الراق"). I use the get_close_matches() function from Python’s difflib module to match detected text with known target words like "العراق", "سلیمانی", "اربیل", etc. I apply a confidence threshold to filter out low-confidence OCR results. What was I expecting?

I expected the system to:

Detect and recognize license plate numbers with high confidence. For partially detected or incorrect words (like "اباز" or "الراق"), the system should return the closest match from a predefined list of target words (such as "العراق" for "الراق"). Improve the accuracy of text recognition and matching, especially for Arabic text, by filtering out non-relevant or low-confidence results.

Continue reading...