[Python] Trying to apply mask to an image with similar dimension yet fails

Trying to implement mask in form of array from torch.Tensordata type, with this code

from transformers import OneFormerProcessor, OneFormerForUniversalSegmentation, SegformerImageProcessor, SegformerForSemanticSegmentation

from PIL import Image
import requests
import cv2

import numpy as np
from matplotlib import pyplot as plt

# importing image to use
image = Image.open("D:\img_1.jpg")


# Create mask from a pretrained model
# Loading a single model for all three tasks
processor_shilab = OneFormerProcessor.from_pretrained("shi-labs/oneformer_ade20k_swin_large")

model_shilab = OneFormerForUniversalSegmentation.from_pretrained("shi-labs/oneformer_ade20k_swin_large")

panoptic_inputs = processor_shilab(images=image, task_inputs=["panoptic"], return_tensors="pt")

panoptic_outputs = model_shilab(**panoptic_inputs)

#predicted_panoptic_map in torch.Tensor type
predicted_panoptic_map = processor_shilab.post_process_panoptic_segmentation(panoptic_outputs, target_sizes=[image.size[::-1]])[0]["segmentation"]

#color array
color = np.array([0, 255, 0], dtype="uint8")

masked_image = np.where(predicted_panoptic_map, image)



Then has this error

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[37], line 3
1 color = np.array([0, 255, 0], dtype="uint8")
----> 3 masked_image = np.where(predicted_panoptic_map, color, grey)

ValueError: operands could not be broadcast together with shapes (1080,1080) (3,) (1080, 1080, 3)


before it has different dimension with the image that is (1080, 1080, 3) so I immediately make it grayscale to make it (1080, 1080) similar to the mask

But after reduce the dimension, I received this error

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[37], line 3
1 color = np.array([0, 255, 0], dtype="uint8")
----> 3 masked_image = np.where(predicted_panoptic_map, color, grey)

ValueError: operands could not be broadcast together with shapes (1080,1080) (3,) (1080,1080)


Removing the color array gives me

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[38], line 3
1 color = np.array([0, 255, 0], dtype="uint8")
----> 3 masked_image = np.where(predicted_panoptic_map, grey)

ValueError: either both or neither of x and y should be given


The dimensions are the same, the mask is generated from the said image, what else did I missed?

Note the mask is already in color? something like this [![plotted-mask with matplotlib][1]][1]

Edit 4: The mask isn't in color, I guess it's from matplotlib

Edit 5: editing only for posterity, will read the numpy docs as suggested

Continue reading...