[Python] Learnable parameter only updating non-zero values and identically during training...

I have the following code in Python3.11 using PyTorch:

arr = np.array([[0, 0, 0],
[0, -1, 0],
[0, 1, 0]])
arr_tensor = torch.tensor(arr, dtype=torch.float32, device=device).unsqueeze(0).repeat(5, 1, 1, 1, 1)


learnable_parameter = torch.nn.Parameter(arr_tensor)
optimizer = torch.optim.Adam([learnable_parameter, lr=1e-3)


During training, only the non-zero elements are being updated. Moreover, up to a minus sign, the non-zero elements' values are the same.

If I initiate arr with all zeros, learnable_parameter stays constantly zero. What I want is for the 9 numbers (that come from arr) and make up learnable_parameter are updated during training. And I want to initiate them as you see there. How can I achieve that?

Note that I need arr_tensor in that particular shape for other reasons.

Continue reading...