[Python] tf.keras: Why my UNet with final layer Conv3D with sigmoid activation gives...

I am really confused because this should never happen according to common sense and to all things that I found on the internet.

In short. My UNet with sigmoid activation as last layer gives probabilities of less than 0 and more than 1.

I have a UNet for semantic segmentation of 3-dimensional brain MRIs. My last layer is a Conv3D with activation function sigmoid. I have confirmed that the last activation is sigmoid.

u_cnn.layers[-1]
<Conv3D name=conv3d_20, built=True>
u_cnn.layers[-1].activation
<function keras.src.activations.activations.sigmoid(x)>


I know that any number (even infinity and minus infinity) should transform into something between 0 and 1 when passed through a sigmoid function. I know it is impossible to get anything out of that range. However, when I use the trained model to predict on the train data (which appeared to learn reasonably well but did not generalize well to the validation data), the result I got is:

Y_predictions_train = u_cnn.predict(X_train, batch_size=16)
np.min(Y_predictions_train)
-676.87195
np.max(Y_predictions_train)
20591.613


To say that I am extremely confused about this result is an understatement. How can a number be passed through the sigmoid function and give something outside of the 0-1 interval?

Any help appreciated.

I am using tensorflow 2.17.0 with keras 3.3.3

The full UNet architecture is presented below

# Create functions to define CNN blocks

def double_conv_block(x, n_filters):
# Conv3D then ReLU activation
x = layers.Conv3D(n_filters, kernel_size=5, strides=1, padding ='same', activation = 'relu', data_format="channels_last")(x)
# Batch normalization
x = layers.BatchNormalization()(x)
# Conv3D then ReLU activation
x = layers.Conv3D(n_filters, kernel_size=3, strides=1, padding ='same', activation = 'relu')(x)
return x


def downsample_block(x, n_filters):
f = double_conv_block(x, n_filters)
p = layers.MaxPool3D(pool_size=2)(f)
return f, p


def upsample_block(x, conv_features, n_filters):
# upsample
x = layers.Conv3DTranspose(n_filters, kernel_size=3, strides=2, padding='same')(x)
# concatenate
x = layers.concatenate([x, conv_features])
# Conv3D twice with ReLU activation
x = double_conv_block(x, n_filters)
return x


# Define the U CNN model
def create_u_cnn(SX, SY, SZ):
# inputs
inputs = layers.Input(shape=(SX, SY, SZ, 2))
# encoder: contracting path - downsample
# 1 - downsample
f1, p1 = downsample_block(inputs, 64)
# 2 - downsample
f2, p2 = downsample_block(p1, 128)
# 3 - downsample
f3, p3 = downsample_block(p2, 256)
# 4 - downsample
f4, p4 = downsample_block(p3, 512)
# 5 - bottleneck
bottleneck = double_conv_block(p4, 1024)
# decoder: expanding path - upsample
# 6 - upsample
u6 = upsample_block(bottleneck, f4, 512)
# 7 - upsample
u7 = upsample_block(u6, f3, 256)
# 8 - upsample
u8 = upsample_block(u7, f2, 128)
# 9 - upsample
u9 = upsample_block(u8, f1, 64)
# Final convolutional and dropout layers
u10 = layers.Conv3D(64, kernel_size=3, strides=1, padding ='same', activation = 'relu')(u9)
u11 = layers.Dropout(rate=0.4)(u10)
u12 = layers.Conv3D(64, kernel_size=3, strides=1, padding ='same', activation = 'relu')(u11)
# outputs
outputs = layers.Conv3D(filters=1, kernel_size=1, strides=1, padding='same', activation = 'sigmoid')(u12)
# unet model with Keras Functional API
u_cnn = tf.keras.Model(inputs, outputs, name="U-Net")
return u_cnn

# Mirrored strategy for using GPUs in parallel
mirrored_strategy = tf.distribute.MirroredStrategy()

with mirrored_strategy.scope():
u_cnn = create_u_cnn(SX, SY, SZ)


u_cnn.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
loss=FocalTverskyLoss,
metrics=['accuracy', tf.keras.metrics.BinaryIoU(target_class_ids=[1], threshold=0.5), dice_coef])

u_cnn.summary()


I repeated this experiment with the same data and different architectures (including a ResNet152 UNet from segmentation_models_3D. I have created a model with all layers except the last one and then added the last one as a conv3D with sigmoid activation. I still get the same result of probabilities less than 0 and more than 1.

Continue reading...