[Python] Different SSIM Results for 3D Microscopy Slices Using torchmetrics and skimage: Which...

I am working with 3D microscopy data (e.g., stacks of 2D images) and trying to compute the Structural Similarity Index (SSIM) between two 3D images. The goal is to evaluate the similarity between an input stack and a ground truth stack. They both have the same dimensions.

Here is a MRE with random tensors:

import torch
import numpy as np
from skimage.metrics import structural_similarity
from torchmetrics.image import StructuralSimilarityIndexMeasure

# Simulating 10 256x256 numpy images randint from 0 to 2**16
input_array = np.random.randint(0, 2**16, (10, 256, 256), dtype=np.uint16)
output_array = np.random.randint(0, 2**16, (10, 256, 256), dtype=np.uint16)
input_tensor = torch.tensor(input_array, dtype=torch.float32)
output_tensor = torch.tensor(output_array, dtype=torch.float32)

# Computing SSIM
data_range = max(input_tensor.max(), output_tensor.max()) - min(input_tensor.min(), output_tensor.min())

ssim_metric = StructuralSimilarityIndexMeasure(data_range=data_range)
ssim_score = ssim_metric(
input_tensor.unsqueeze(0), # Add batch dimension
output_tensor.unsqueeze(0) # Add batch dimension
)
print(f"torchmetrics - SSIM score: {ssim_score}")

all_slices_ssim = []
for i in range(input_tensor.shape[0]):
slice_ssim = ssim_metric(
input_tensor.unsqueeze(0).unsqueeze(0), # Add channel and batch dimensions
output_tensor.unsqueeze(0).unsqueeze(0) # Add channel and batch dimensions
)
all_slices_ssim.append(float(slice_ssim))
mean_ssim = sum(all_slices_ssim) / len(all_slices_ssim)
print(f"torchmetrics - Mean SSIM score (slice-wise): {mean_ssim}")

ssim_3d_scikit = structural_similarity(
input_array,
output_array,
data_range=data_range.item(),
channel_axis=None
)
print(f"skimage - 3D SSIM score: {ssim_3d_scikit}")

ssim_3d_scikit_slices: list[float] = []
for i in range(input_tensor.shape[0]):
slice_ssim = structural_similarity(
input_array,
output_array,
data_range=data_range.item(),
channel_axis=None
)
ssim_3d_scikit_slices.append(slice_ssim)
print(f"skimage - Mean SSIM score (slice-wise): {sum(ssim_3d_scikit_slices) / len(ssim_3d_scikit_slices)}")


When I run this code, I get different results for the SSIM scores:

torchmetrics - SSIM score: 0.00599062442779541
torchmetrics - Mean SSIM score (slice-wise): 0.005990624928381294
skimage - 3D SSIM score: 0.006143948174795598
skimage - Mean SSIM score (slice-wise): 0.005819291556411987


I'm not really sure, which one I should use (I feel like slice-wise mean is a no-go).

Continue reading...