[Python] What is the difference between 'torch.backends.cudnn.deterministic=True' and...

My network includes torch.nn.MaxPool3d which throw a RuntimeError when the cudnn deterministic flag is on according to the PyTorch docs (version 1.7 - https://pytorch.org/docs/stable/generated/torch.set_deterministic.html#torch.set_deterministic).

However, when I inserted the code torch.backends.cudnn.deterministic=True at the beginning of my code, there was no RuntimeError. Why doesn't that code throw a RuntimeError?

I wonder whether that code guarantees the deterministic computation of my training process.

Continue reading...