I'm trying to use Facebook's Mask2Former to automatically generate masks for an image. By using the code they have at this page: with torch.no_grad(): outputs = model(**inputs) .... How do I interprete this "outputs" object into a useful label+segment+mask information? Ideally I want to generate a PNG with the mask. class_queries_logits = outputs.class_queries_logits masks_queries_logits = outputs.masks_queries_logits # you can pass them to processor for postprocessing result = processor.post_process_panoptic_segmentation(outputs, target_sizes=[image.size[::-1]])[0] I'm not sure how to convert these tensors to something useful. https://huggingface.co/docs/transformers/main/en/model_doc/mask2former describes the parameters but I still don't have a clue on what to do. In the huggingface API, the result is label (cat), score (say, 0.9) and mask - that's what I also want. Edit 1: I got the scores and labels: segs = result["segments_info"] for s in segs: score = s["score"] segment_label_id = s['label_id'] segment_label = model.config.id2label[segment_label_id] Masks to png, how? Edit 2: Seems I got it: import numpy as np mask = (result['segmentation'].numpy() == 1) # == x up to what we want to save visual_mask = (mask * 255).astype(np.uint8) visual_mask = Image.fromarray(visual_mask) visual_mask.save("r:/1.png"); Continue reading...
