[Python] Cannot Open Raspberry Pi Camera (IMX296 Global Shutter Camera) with OpenCV and...

Question:
I am using a Raspberry Pi 5 with an IMX296 global shutter camera, connected via a MIPI interface, to perform real-time object detection using YOLOv8 in Python via OpenCV and GStreamer.

1. How can I enable GStreamer inside my virtual environment? (I want OpenCV in yolovenv to recognize GStreamer.)
2. Could there be an issue with my GStreamer pipeline? (Should I modify video/x-raw, width=640, height=640 to another resolution or format?)
3. The big issue is that I want my camera to work while it performs the real-time objection detection.

Problem:
When I run the script, I get the following: ❌ Could not open the camera on Raspberry Pi!

Which means that cv2.VideoCapture() fails to initialize the camera.

1. The camera works in the terminal

• Running libcamera-hello successfully displays the camera feed.
• Running gst-launch-1.0 libcamerasrc ! video/x-raw, format=BGR, width=640, height=480, framerate=30/1 ! videoconvert ! autovideosink works.

1. Checked GStreamer installation

• Running python3 -c "import cv2; print(cv2.getBuildInformation())" | grep -i gstreamer shows that GStreamer is enabled (YES) in my local environment.

1. Virtual Environment (yolovenv)

• I installed ultralytics in a virtual environment (source yolovenv/bin/activate) because installing it directly on the system failed.
• However, when I run python3 -c "import cv2; print(cv2.getBuildInformation())" | grep -i gstreamer inside the virtual environment, I get GStreamer: NO, which might be causing the issue.

THE CODE BELOW:

from ultralytics import YOLO
import cv2

#Load YOLOv8 model
model = YOLO("yolov8n.pt")

#GStreamer pipeline for Raspberry Pi camera
gst_pipeline = (
"libcamerasrc ! video/x-raw, width=640, height=480, framerate=30/1 ! "
"videoconvert ! appsink"
)

#Open the camera using OpenCV and GStreamer
cap = cv2.VideoCapture(gst_pipeline, cv2.CAP_GSTREAMER)

if not cap.isOpened():
print("Could not open the camera on Raspberry Pi!")
exit()

while True:
ret, frame = cap.read()
if not ret:
print("Failed to read frame from camera!")
break

#Run YOLOv8 inference
results = model(frame)

#Draw detected objects
annotated_frame = results[0].plot()
cv2.imshow("YOLOv8 - Raspberry Pi Detection", annotated_frame)

#Press 'q' to exit
if cv2.waitKey(1) & 0xFF == ord("q"):
print("Closing the camera...")
break

#Release the camera and close OpenCV window
cap.release()
cv2.destroyAllWindows()

Continue reading...