[Python] no_grad causing problem in Pytorch NN model

I'm building regression neural network model for my NLP data.

But can't get through this error: RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

What should I do?

ChatGPT says:

• it's requires_grad=False somewhere in the code which isn't true
• my data isn't float which isn't true
• my data isn't on same device as model which isn't true

Currently, the model with data looks like this:

1. 
   My data:
   
   X_val = torch.tensor(X_val_scaled, dtype=torch.float32)
   X_test = torch.tensor(X_test_scaled, dtype=torch.float32)
   
   y_train = torch.tensor(y_train, dtype=torch.float32)
   y_val = torch.tensor(y_val, dtype=torch.float32)
   y_test = torch.tensor(y_test, dtype=torch.float32)
2. 
   My model:
   
   def __init__(self, in_features=780, h1=8, h2=6, output=1):
   super().__init__()
   self.fc1 = nn.Linear(in_features, h1)
   self.fc2 = nn.Linear(h1, h2)
   self.out = nn.Linear(h2, output)
   
   def forward(self, x):
   x = F.relu(self.fc1(x))
   x = F.relu(self.fc2(x))
   x = self.out(x)
   return x
   
   model_nn = Model(in_features=780)
   optimizer = torch.optim.Adam(model_nn.parameters(), lr=0.001)
   criterion = nn.MSELoss()
   
   num_epochs = 100
   loss_list = []
   
   for epoch in range(num_epochs):
   model_nn.train()
   
   # Forward + loss + backward + step
   y_pred = model_nn(X_train)
   loss = F.mse_loss(y_pred, y_train)
   
   optimizer.zero_grad()
   loss.backward()
   optimizer.step()
   
   loss_list.append(loss.item())
   
   # Evaluate on val set
   model_nn.eval()
   with torch.no_grad():
   val_pred = model_nn(X_val)
   val_loss = F.mse_loss(val_pred, y_val)
   
   print(f"Epoch {epoch+1:03d}/{num_epochs} | Train Loss: {loss.item():.4f} | Val Loss: {val_loss.item():.4f}")

Continue reading...