[Python] Keras ResNet model trains but does not evaluate

I am trying to use Keras' implementation of ResNet50 and adapting it to work on the tiny-imagenet dataset from huggingface.

I can use the model to train on the data and collect training and validation metrics without much trouble, but when I try to test it the model simply returns 0% accuracy and 0.0 loss.

I found this thread with the same issue, but this user is making a custom model from scratch so the comments suggest manually overriding the test_step function to fix the problem which I have not been able to do.

Here is the code for how I create the adapted model:

from tensorflow.keras.applications.resnet import ResNet50, preprocess_input
class ResNet_Interface:
def __init__(self, incoming_data_shape):
self.incoming_data_shape = incoming_data_shape
self.model = ResNet50(input_shape=self.incoming_data_shape,
include_top=True,
weights=None, classes=200)
self.model.trainable = True
self.pre_process = preprocess_input

def get_model(self):
return self.model


Here is the code for how I train and evaluate the same model:


def evaluate_model_imagenet(dataset, resnet_interface):

model = resnet_interface.get_model()
pre_process = resnet_interface.pre_process

opt = Adam()

model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])

score = model.fit(pre_process(dataset['x_train']), dataset['y_train'],
batch_size=64,
epochs=1,
verbose=2,
validation_data=(pre_process(dataset['x_val']), dataset['y_val']))

test_score = model.evaluate(data_process(dataset['x_test']), verbose=2)


This is a sample output showcasing the problem:

1563/1563 - 227s - loss: 5.0075 - accuracy: 0.0418 - val_loss: 4.5963 - val_accuracy: 0.0833 - 227s/epoch - 145ms/step
94/94 - 1s - loss: 0.0000e+00 - accuracy: 0.0000e+00 - 569ms/epoch - 6ms/step


I am using python 3.8.12 and tensorflow 2.9.1

Do you know what is causing this and how I can fix it? I am very confused as to why the model evaluates without errors but produce this strange output

Continue reading...