[Python] Issue with Multi-Target Configuration in Temporal Fusion Transformer (PyTorch...

I am trying to train a Temporal Fusion Transformer (TFT) model using the pytorch-forecasting library for a multi-target prediction task. My dataset contains multiples one-hot encoded target columns , and I am encountering an error during the model creation step. The error message is as follows:

'int' object is not iterable


Despite following the documentation and adjusting the configuration, the issue persists. Interestingly, when I create a simplified dataset with only 1 target columns , the script works fine. Below, I will provide both the working script (with random data) and the non-working script (with random data as well).

Working Script (Simplified Dataset): Here is a script that works with a simplified dataset containing 1 target columns, whis is directly copied from https://pytorch-forecasting.readthe...orecasting.data.encoders.MultiNormalizer.html

import numpy as np
import pandas as pd
from pytorch_forecasting import TimeSeriesDataSet, TemporalFusionTransformer
from torch.nn import BCEWithLogitsLoss

# Create a test dataset with 5 target columns
test_data = pd.DataFrame(
dict(
**{f'target_{i}': np.random.randint(0, 2, 30) for i in range(1, 6)}, # 5 target columns
group=np.repeat(np.arange(3), 10),
time_idx=np.tile(np.arange(10), 3),
)
)

# Configure the TimeSeriesDataSet
dataset = TimeSeriesDataSet(
test_data,
group_ids=["group"],
target=[f'target_{i}' for i in range(1, 6)], # List of target columns
time_idx="time_idx",
min_encoder_length=5,
max_encoder_length=5,
min_prediction_length=2,
max_prediction_length=2,
time_varying_unknown_reals=[f'target_{i}' for i in range(1, 6)],
)

# Define the loss function
loss = [BCEWithLogitsLoss()] * len(dataset.target)

# Create the TFT model
model = TemporalFusionTransformer.from_dataset(
dataset,
hidden_size=16,
attention_head_size=1,
dropout=0.1,
hidden_continuous_size=8,
output_size=len(dataset.target), # Number of target columns
loss=loss, # Loss function
learning_rate=0.01,
)

print("Model created successfully!")


This script works perfectly with the simplified dataset.

Non-Working Script : However, when I apply the same logic to a dataset with 5 one-hot encoded target columns , the model creation fails with the following error:

Errore durante la creazione del modello: 'int' object is not iterable


Here is the non working script:


import numpy as np
import pandas as pd
from pytorch_forecasting import TimeSeriesDataSet, TemporalFusionTransformer
from torch.nn import BCEWithLogitsLoss

# Crea un dataset di prova con 5 colonne target
test_data = pd.DataFrame(
dict(
**{f'target_{i}': np.random.randint(0, 2, 30) for i in range(1,6)}, # 5 colonne target
group=np.repeat(np.arange(3), 10),
time_idx=np.tile(np.arange(10), 3),
)
)

# Configura il TimeSeriesDataSet
dataset = TimeSeriesDataSet(
test_data,
group_ids=["group"],
target=[f'target_{i}' for i in range(1,6)], # Lista di colonne target
time_idx="time_idx",
min_encoder_length=5,
max_encoder_length=5,
min_prediction_length=2,
max_prediction_length=2,
time_varying_unknown_reals=[f'target_{i}' for i in range(1, 6)],
)

# Verifica il numero di colonne target
print("Numero di colonne target:", len(dataset.target)) # Dovrebbe essere 5

# Crea il modello TFT
try:
model = TemporalFusionTransformer.from_dataset(

dataset,
hidden_size=16,
attention_head_size=1,
dropout=0.1,
hidden_continuous_size=8,
output_size=len(dataset.target), # Numero di colonne target
loss=[BCEWithLogitsLoss()] * len(dataset.target), # Una perdita per ciascuna colonna target
learning_rate=0.01,

)
print("Modello creato correttamente!")
except Exception as e:
print(f"Errore durante la creazione del modello: {e}")


When running the above script, I get the following error:

Error during model creation: 'int' object is not iterable


Additionally, I receive the following warnings:

Attribute 'loss' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['loss'])`.
Attribute 'logging_metrics' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['logging_metrics'])`.```

Environment:
• Python Version: 3.12
• PyTorch Forecasting Version: [1.3.0]
• PyTorch Lightning Version: [1.9.4]

Question:
How can I correctly configure the TemporalFusionTransformer for a multi-target prediction task with X one-hot encoded target columns ? Is there a specific way to define the loss parameter or handle multi-targets in this scenario?

Continue reading...