[Python] How can I empty the GPU memory when I get a error like OOM?

sometimes,when I got a OOM error,but the parameters of LLM has been load in GPU,and it cannot be cleared automatically.

So,I try this

torch.cuda.empty_cache()


but it did't work.So,everytime I must restart my GPU to clear the cache.

Continue reading...