[Python] BERT model: model.fit throws "Invalid dtype: object" error

I been trying everything to make the following code work:

from tf_keras.src.layers.serialization import activation
import keras_nlp

# bert layers
text_input = tf.keras.layers.Input(shape=(), dtype=tf.string ,name="text")
preprocessor = keras_nlp.models.BertPreprocessor.from_preset("bert_base_en_uncased",trainable=True)
preprocessed_text = preprocessor(text_input)
encoder = keras_nlp.models.BertBackbone.from_preset("bert_base_en_uncased")
outputs = encoder(preprocessed_text)

# neural network layers
l = tf.keras.layers.Dropout(0.1, name='dropout')(outputs['pooled_output'])
l = tf.keras.layers.Dense(1, activation='sigmoid', name='output')(l)

# construct final model
model = tf.keras.Model(inputs=[text_input], outputs=[l])

model.summary()

METRICS = [
tf.keras.metrics.BinaryAccuracy(name='accuracy'),
tf.keras.metrics.Precision(name='prediction'),
tf.keras.metrics.Recall(name='recall')
]

model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=METRICS)



It runs up to here, but when I try to fit it, if gives me the following error:

model_bert=model.fit(X_train,
y_train,
epochs=10)

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/var/folders/my/psclr1fj1jq4cwk4tl2w9wzr0000gn/T/ipykernel_68633/736105977.py in <module>
1 # X_train = tf.data.Dataset.from_tensor_slices(X_train)
2 # y_train = tf.data.Dataset.from_tensor_slices(y_train)
----> 3 model_bert=model.fit(X_train,
4 y_train,
5 epochs=10)

~/opt/anaconda3/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py in error_handler(*args, **kwargs)
120 # To get the full stack trace, call:
121 # `keras.config.disable_traceback_filtering()`
--> 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

~/opt/anaconda3/lib/python3.9/site-packages/optree/ops.py in tree_map(func, tree, is_leaf, none_is_leaf, namespace, *rests)
750 leaves, treespec = _C.flatten(tree, is_leaf, none_is_leaf, namespace)
751 flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
--> 752 return treespec.unflatten(map(func, *flat_args))
753
754

ValueError: Invalid dtype: object


I been looking in multiple articles, but I can't seem to find an answer. I am just testing to understand how LLMs work and this would be the last step. Any help is welcome.

Continue reading...