[Python] How can I obtain a 360 degree image from 3 non overlapping 120 degree fov images?

I am trying to stitch or project 3 images from 3 120 degree fov cameras that have been aligned to perfectly cover a 360 degree fov. They have no overlap so the common stitching techniques using opencv's Stitcher class will not work as relies on overlapping and detecting matching orb points between the images. Basic concatenation does not work either as they need to be warped or projected on to a cylinder in order to get the correct 360 degree image.

Current simple concatenation of the images:

def create_360_view(camera_left, camera_right, camera_rear):
left_img = camera_left.get_data()
right_img = camera_right.get_data()
rear_img = camera_rear.get_data()

combined_img = np.concatenate((left_img, right_img, rear_img), axis=1)
return combined_img

def display_image(display, image):
surface = pygame.surfarray.make_surface(image.swapaxes(0, 1))
display.blit(surface, (0, 0))
pygame.display.flip()


Camera Aspects: 120 degree fov, 1024x728

Resulting image: https://ibb.co/FnFhBcp

I have tried some transformation work as seen below but the images have gaps between them when projected:

def create_360_view(camera_left, camera_right, camera_rear):
left_img = camera_left.get_data()
right_img = camera_right.get_data()
rear_img = camera_rear.get_data()

height, width = left_img.shape[:2]

panorama_width = width * 3
panorama = np.zeros((height, panorama_width, 3), dtype=np.uint8)

place_image_cylindrical(panorama, left_img, 0, width)
place_image_cylindrical(panorama, right_img, width, width)
place_image_cylindrical(panorama, rear_img, 2*width, width)

return panorama

def place_image_cylindrical(panorama, img, start_x, width):
height, img_width = img.shape[:2]
cylinder_radius = width / (2 * np.pi / 3) # 120 degrees = 2π/3 radians

theta = np.linspace(-np.pi/3, np.pi/3, width)
h = np.arange(height) - height // 2
theta, h = np.meshgrid(theta, h)

x = np.round(cylinder_radius * np.tan(theta) + img_width / 2).astype(int)
y = np.round(h * np.cos(theta) + height / 2).astype(int)

mask = (x >= 0) & (x < img_width) & (y >= 0) & (y < height)

panorama_x = np.round(theta * width / (2*np.pi/3) + start_x).astype(int)
panorama_y = h + height // 2

valid_mask = mask & (panorama_x >= 0) & (panorama_x < panorama.shape[1]) & (panorama_y >= 0) & (panorama_y < panorama.shape[0])

panorama[panorama_y[valid_mask], panorama_x[valid_mask]] = img[y[valid_mask], x[valid_mask]]


Resulting image: https://ibb.co/Mk2FXtV

Continue reading...