[Python] “ValueError: Invalid input shape when combining image, mask, and CSV data in Keras...

I’m working on a deep learning model using Keras where I’m trying to combine three inputs: images, masks, and CSV data. The goal of my model is to predict the presence and type of brain hemorrhage from medical images (CT scans) and CSV data. I have structured the model using an Attention U-Net for image segmentation and a dense layer for CSV data processing.

The input data consists of:

1. 
   Images: Grayscale CT scans with shape (256, 256, 1).
   
   
2. 
   Masks: Binary masks corresponding to the segmentation with shape (256, 256, 1).
   
   
3. 
   CSV Data: Numerical data for various hemorrhage types with shape (6,).
   

Problem:

I’m encountering the following error when trying to train the model using model.fit():

ValueError: Invalid input shape for input Tensor("functional_1/Cast:0", shape=(None, 6), dtype=float32). Expected shape (None, 256, 256, 1), but input has incompatible shape (None, 6)

It seems that the CSV data, which has a shape of (None, 6), is being fed into the model where the image data (with shape (None, 256, 256, 1)) is expected. I’ve double-checked the input shapes and used train_test_split to split the data, but the issue persists.

Code:

Here’s a simplified version of my code:

# Model definition
inputs, outputs = attention_unet() # UNet model for images

csv_input = layers.Input(shape=(6,), name='csv_input') # CSV data input

mask_input = layers.Input(shape=(256, 256, 1), name='mask_input') # Mask input

# CSV data processing
csv_x = layers.Dense(64, activation='relu')(csv_input)csv_x = layers.Dense(32, activation='relu')(csv_x)

# Combine CSV with U-Net output
flatten_outputs = layers.Flatten()(outputs)combined = layers.Concatenate()([flatten_outputs, csv_x])

# Final output
final_output = layers.Dense(1, activation='sigmoid', name='final_output')(combined)

# Model compilation
model = models.Model(inputs=[inputs, csv_input, mask_input], outputs=[outputs, final_output])model.compile(optimizer=Adam(), loss={'final_output': 'binary_crossentropy', 'outputs': 'categorical_crossentropy'}, metrics=['accuracy', dice_coef, jaccard_index])

# Data shapes
print(f"Shape of X_train: {X_train.shape}") # Shape of images: (batch_size, 256, 256, 1)print(f"Shape of csv_train: {csv_train.shape}") # Shape of CSV: (batch_size, 6)print(f"Shape of y_train: {y_train.shape}") # Shape of masks: (batch_size, 256, 256, 1)

# Model training
model.fit({'input_layer': X_train, 'csv_input': csv_train, 'mask_input': y_train},{'final_output': csv_train, 'outputs': y_train},validation_data=({'input_layer': X_val, 'csv_input': csv_val, 'mask_input': y_val},{'final_output': csv_val, 'outputs': y_val}),epochs=50,batch_size=32,callbacks=callbacks)


Steps I’ve Tried:

• I’ve printed the shapes of the training data, and they all seem correct:
  • Images: (batch_size, 256, 256, 1)
  • CSV: (batch_size, 6)
  • Masks: (batch_size, 256, 256, 1)
• I’ve checked the order of data input in model.fit() to ensure the data is being passed to the correct layers.

Question:

How can I resolve this shape mismatch issue and correctly combine the image, mask, and CSV data in the model? Any suggestions on what I might be doing wrong or how I can fix this?

Continue reading...