[Python] Tesseract OCR Misses Some Text in PDF Processing

I'm processing a PDF by converting each page into an image and then using Tesseract OCR to extract and create boxes around the words. While most of the text is detected correctly, some words which should be easy to read are missed. Interestingly, when I take a screenshot of the same problematic text and run it through Tesseract, the text is detected perfectly.

Below is a simplified version of my code:

import io
from PIL import Image, ImageDraw
import fitz
import pytesseract
from pytesseract import Output
pytesseract.pytesseract.tesseract_cmd = (
r"C:\Program Files\Tesseract-OCR\tesseract.exe"
)

INPUT_PDF = "example.pdf"
OUTPUT_PDF = "boxed_example.pdf"

def process_pdf(pdf_path):
doc = fitz.open(pdf_path)
modified_images = []

for page in doc:
zoom = 300.0 / 72.0
mat = fitz.Matrix(zoom, zoom)
pix = page.get_pixmap(matrix=mat, alpha=False)
img = Image.open(io.BytesIO(pix.tobytes("png")))
draw = ImageDraw.Draw(img)

data = pytesseract.image_to_data(img, output_type=Output.DICT)
n_boxes = len(data["text"])

for i in range(n_boxes):
x = data["left"]
y = data["top"]
w = data["width"]
h = data["height"]
draw.rectangle([x, y, x + w, y + h], outline="red", width=2)

modified_images.append(img.convert("RGB"))

doc.close()

output_buffer = io.BytesIO()
modified_images[0].save(
output_buffer,
"PDF",
resolution=300,
save_all=True,
append_images=modified_images[1:],
)
output_buffer.seek(0)
return output_buffer.getvalue()

boxed_pdf_bytes = process_pdf(INPUT_PDF)
with open(OUTPUT_PDF, "wb") as f:
f.write(boxed_pdf_bytes)
print(f"Boxed PDF saved as {OUTPUT_PDF}")


What I've Tried So Far: Changing the DPI, different psm modes but nothing helped.

The PDF I used: https://www.pdf-tools.com/public/downloads/whitepapers/Whitepaper-PDF-Grundlagen-DE.pdf

Almost all text except on the full pages 2 and 11 is detected.

Continue reading...