[Python] ValueError: The model did not return a loss from the inputs, but `model` exists in...

A complete error is:

ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_ids,attention_mask.


However, dataset contains label and train_dataset in the argument has label in column_names.

from transformers import Trainer, TrainingArguments
batch_size = 64
logging_steps = len(emotions_encoded["train"])

training_args = TrainingArguments(output_dir = "model_out",
num_train_epochs=2,
learning_rate = 2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
weight_decay=0.01,
evaluation_strategy="epoch",
disable_tqdm=False,
logging_steps=logging_steps,
push_to_hub=False,
log_level="error")

from transformers import Trainer
trainer = Trainer(model=model, args=training_args,
compute_metrics=compute_metrics,
train_dataset=emotions_encoded["train"], # column_names shows <class 'list'>: ['text', 'label', 'input_ids', 'attention_mask']
eval_dataset=emotions_encoded["validation"],
tokenizer=tokenizer)

trainer.train()


This is what I found: train_dataset in the following trainer.py function has column_names value <class 'list'>: ['label', 'input_ids', 'attention_mask']

def get_train_dataloader(self) -> DataLoader:
return DataLoader(
train_dataset,
batch_size=self._train_batch_size,
sampler=train_sampler,
collate_fn=data_collator,
drop_last=self.args.dataloader_drop_last,
num_workers=self.args.dataloader_num_workers,
pin_memory=self.args.dataloader_pin_memory,
worker_init_fn=seed_worker,
)


but datasets arrow_dataset.py function:

def _getitem(self, key: Union[int, slice, str, ListLike[int]], **kwargs) -> Union[Dict, List]:
format_columns = kwargs["format_columns"] if "format_columns" in kwargs else self._format_columns


because kwargs is None, thereafter self._format_columns is used and its value is <class 'list'>: ['input_ids', 'attention_mask']. Here model is lost.

And that finally caused dataloder.py class _BaseDataLoaderIter function:

def __next__(self) -> Any:
with torch.autograd.profiler.record_function(self._profile_name):
if self._sampler_iter is None:
# TODO(https://github.com/pytorch/pytorch/issues/76750)
self._reset() # type: ignore[call-arg]
data = self._next_data()


data contains only 'input_ids'and 'attention_mask'.

I am a beginner with the boot NLP with Transformers

Continue reading...