[Python] pytorch Module B=A, A.to('cpu'), but the tensor in B is still in GPU, why?

After converting module A to CPU, the origin parameter tensor still stays on the GPU? When it is released? Is it wrong if I reuse the parameter?

My code:

import torch.nn as nn

class A(nn.Module):
def __init__(self):
super().__init__()
self.fc = nn.Linear(10, 5)
self.relu = nn.ReLU()

def forward(self, x):
return self.relu(self.fc(x))

a = A().to('cuda')

weight = {}
for key, value in a.state_dict().items():
weight[key] = value

a.to('cpu')
print("a.state_dict() device:", [t.device for t in a.state_dict().values()]) # in CPU
print("weight device:", [t.device for t in weight.values()]) # still in GPU


Result:

a.state_dict() device: [device(type='cpu'), device(type='cpu'), device(type='cpu'), device(type='cpu')]

weight device: [device(type='cuda', index=0), device(type='cuda', index=0), device(type='cuda', index=0), device(type='cuda', index=0)]


Why are the tensors in weight still on the GPU?

Continue reading...