[Python] Does Temporal Fusion Transformer Learn Global Trends Across the Entire Time Series?

I'm using the Temporal Fusion Transformer (TFT) to train on time series data, aiming to make real-time forecasts for a specific process unit at any point in time during operation.

However, for training and validation, I'm using complete historical time series data from previously finished process runs — meaning each sample spans from the start to the end of a process. Based on this setup, I have a few questions:

(1) Does the model learn the overall/global trend of each entire time series group? In other words, if I feed real-time data during a live process and make predictions from a mid-point, will the model still perform well, even though it was trained on full-range samples?

(2) During training, is validation performed only on the last decoder segment of each series? Or is it conducted over the entire series range?

(3) Should the validation dataset consist of unseen groups (i.e., different from the training data)? Or can it reuse group IDs from training?

Here is a simplified snippet of how I configure the dataset (feature names removed for brevity):

max_prediction_length = 60
max_encoder_length = 300
training_cutoff = df["time_idx"].max() - max_prediction_length

training = TimeSeriesDataSet(
df_train,
time_idx="time_idx",
target="",
group_ids=[""],
min_encoder_length=max_encoder_length // 2,
max_encoder_length=max_encoder_length,
min_prediction_length=1,
max_prediction_length=max_prediction_length,
static_categoricals=[],
static_reals=[],
time_varying_known_categoricals=[],
variable_groups={},
time_varying_known_reals=[],
time_varying_unknown_categoricals=[],
time_varying_unknown_reals=[],
allow_missing_timesteps=True,
add_relative_time_idx=True,
add_target_scales=True,
add_encoder_length=True,
)

validation = TimeSeriesDataSet.from_dataset(
training, df_valid, predict=False, stop_randomization=True
)

# create dataloaders
batch_size = 128
train_dataloader = training.to_dataloader(
train=True, batch_size=batch_size, num_workers=0
)
val_dataloader = validation.to_dataloader(
train=False, batch_size=batch_size * 10, num_workers=0
)

Continue reading...