I am trying to understand the code for temporal embedding inside autoformer implementation using pytorch. https://github.com/thuml/Autoformer/blob/main/layers/Embed.py class TemporalEmbedding(nn.Module): def __init__(self, d_model, embed_type='fixed', freq='h'): super(TemporalEmbedding, self).__init__() minute_size = 4 hour_size = 24 weekday_size = 7 day_size = 32 month_size = 13 Embed = FixedEmbedding if embed_type == 'fixed' else nn.Embedding if freq == 't': self.minute_embed = Embed(minute_size, d_model) self.hour_embed = Embed(hour_size, d_model) self.weekday_embed = Embed(weekday_size, d_model) self.day_embed = Embed(day_size, d_model) self.month_embed = Embed(month_size, d_model) def forward(self, x): x = x.long() minute_x = self.minute_embed(x[:, :, 4]) if hasattr(self, 'minute_embed') else 0. hour_x = self.hour_embed(x[:, :, 3]) weekday_x = self.weekday_embed(x[:, :, 2]) day_x = self.day_embed(x[:, :, 1]) month_x = self.month_embed(x[:, :, 0]) return hour_x + weekday_x + day_x + month_x + minute_x Why is day_size set to 32 and month_size to 13? I know I should post this under the discussion tab of the above repository link but I believe it has not been active since 2021. I tried to research it but I am getting more confused, because multiple repos use the same values for minutes and days and month. https://github.com/AI4Science-WestlakeU/beno/blob/main/transformer.py Continue reading...
