I have Yolo object tracking model. My depth camera steps and takes photos, after that, I give my image to my model as input. I need to convert 2D pixel coordinates to 3D real-world coordinates, how do I do it effectively? I am trying it by using my camera's intrinsic parameters as shown in the image, I have a depth map, it's includes depth for each(distance from the camera) pixel .enter image description here import numpy as np # Load the depth map depth_map = np.load("2025-01-15_16_26_28/P1_1736944088829_depth_array.npy")/100 # Camera intrinsics f_x = 384.75 # Focal length in pixels (x-axis) f_y = 384.75 # Focal length in pixels (y-axis) c_x = 312 # Principal point x-coordinate c_y = 236 # Principal point y-coordinate # Intrinsic matrix K = np.array([ [f_x, 0, c_x], [0, f_y, c_y], [0, 0, 1] ]) # Generate the 3D coordinates from the depth map def depth_map_to_3d(depth_map, K): h, w = depth_map.shape # Dimensions of the depth map # Generate pixel grid i, j = np.meshgrid(np.arange(w), np.arange(h)) # Pixel coordinates (u, v) # Flatten the depth map and pixel grids for efficient computation u = i.flatten() v = j.flatten() Z = depth_map.flatten() # Compute 3D coordinates X = (u - K[0, 2]) * Z / K[0, 0] Y = (v - K[1, 2]) * Z / K[1, 1] # Stack into a single array of 3D coordinates points_3d = np.stack((X, Y, Z), axis=-1) # Shape: (N, 3), where N = h * w return points_3d # Get 3D coordinates for all pixels points_3d = depth_map_to_3d(depth_map, K) # Reshape back to the original image shape (H, W, 3) for convenience points_3d = points_3d.reshape(depth_map.shape[0], depth_map.shape[1], 3) # Output a sample of 3D coordinates print("3D Coordinates for pixel (360, 640):") print(points_3d[226, 346]) # Example pixel coordinates # Optional: Save the 3D points as a file np.save("points_3d.npy", points_3d) Continue reading...
