[Python] Why is my model's validation accuracy stuck while training a neural network?

I'm training a neural network for a classification task using TensorFlow/Keras. The training accuracy improves over epochs, but the validation accuracy remains constant and low. I suspect it may be overfitting, but regularization and dropout didn't help. I'm not sure what else could be causing this issue.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import SparseCategoricalCrossentropy

model = Sequential([
Dense(128, activation='relu', input_shape=(X_train.shape[1],)),
Dropout(0.5),
Dense(64, activation='relu'),
Dropout(0.5),
Dense(num_classes, activation='softmax')
])

model.compile(optimizer=Adam(learning_rate=0.001),
loss=SparseCategoricalCrossentropy(),
metrics=['accuracy'])

history = model.fit(X_train, y_train,
epochs=30,
batch_size=32,
validation_data=(X_val, y_val))



I tried adding dropout, L2 regularization, and reducing model complexity. I also shuffled the data, normalized inputs, and checked for data leakage. Despite these changes, validation accuracy doesn't improve beyond 60%, while training reaches over 90%. I expected at least moderate validation accuracy improvement after these adjustments.

Continue reading...