[Python] TensorFlow/Transformers Compatibility Issue: ImportError from tensorflow.keras Causes...

The following code is from two years ago and throws an error on my current setup. I was wondering if it is related to the change to Keras 3 or what't is going on here. I can get it to work

• by excluding all from tensorflow.keras import something statements and exclusively using tf.keras.something
• or using the old setup.

The code works with the following (old) setup:

• tf version: 2.13.0
• keras version: 2.13.1
• transformers version: 4.33.3
• python: 3.8.10

But throws an error with the current setup and using from tensorflow.keras import something

• tf version: 2.16.1
• keras version: 3.3.2
• transformers version: 4.39.3
• python 3.11.5

import tensorflow as tf
from transformers import ElectraTokenizer, TFElectraModel
from tensorflow.keras.optimizers import Adam # if I include this line (regardless if I use Adam in the code) I get the error.

model = TFElectraModel.from_pretrained('deepset/gelectra-base')

inputs = tf.keras.layers.Input(shape=(40,), dtype='int32')

electra_output = model(inputs)[0]


I always get the same error once I include tensorflow.keras.something with the current setup

ValueError Traceback (most recent call last)
Cell In[3], line 7
4 inputs = tf.keras.layers.Input(shape=(40,), dtype='int32')
5 # inputs = Input(shape=(40,), dtype='int32')
----> 7 electra_output = model(inputs)[0]

File ~/.local/anaconda3/lib/python3.11/site-packages/tf_keras/src/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
67 filtered_tb = _process_traceback_frames(e.__traceback__)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb

File ~/.local/anaconda3/lib/python3.11/site-packages/transformers/modeling_tf_utils.py:436, in unpack_inputs.<locals>.run_call_with_unpacked_inputs(self, *args, **kwargs)
433 else:
434 config = self.config
--> 436 unpacked_inputs = input_processing(func, config, **fn_args_and_kwargs)
437 return func(self, **unpacked_inputs)

File ~/.local/anaconda3/lib/python3.11/site-packages/transformers/modeling_tf_utils.py:566, in input_processing(func, config, **kwargs)
564 output[main_input_name] = main_input
565 else:
--> 566 raise ValueError(
567 f"Data of type {type(main_input)} is not allowed only {allowed_types} is accepted for"
568 f" {main_input_name}."
569 )
571 # Populates any unspecified argument with their default value, according to the signature.
572 for name in parameter_names:

ValueError: Exception encountered when calling layer 'tf_electra_model_1' (type TFElectraModel).

Data of type <class 'keras.src.backend.common.keras_tensor.KerasTensor'> is not allowed only (<class 'tensorflow.python.framework.tensor.Tensor'>, <class 'bool'>, <class 'int'>, <class 'transformers.utils.generic.ModelOutput'>, <class 'tuple'>, <class 'list'>, <class 'dict'>, <class 'numpy.ndarray'>) is accepted for input_ids.

Call arguments received by layer 'tf_electra_model_1' (type TFElectraModel):
• input_ids=<KerasTensor shape=(None, 40), dtype=int32, sparse=None, name=keras_tensor>
• attention_mask=None
• token_type_ids=None
• position_ids=None
• head_mask=None
• inputs_embeds=None
• encoder_hidden_states=None
• encoder_attention_mask=None
• past_key_values=None
• use_cache=None
• output_attentions=None
• output_hidden_states=None
• return_dict=None
• training=False

Continue reading...