[Python] Cannot Access GPU in PyTorch: torch.cuda.is_available() is False and CUDA Error

I am working on an EC2 instance with the following setup:

NVIDIA Driver Version: 560.35.03 CUDA Version (nvidia-smi): 12.6 nvcc Version: 12.4 GPU Model: NVIDIA A10G

I installed PyTorch and CUDA using the following command:

conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia


However, when I run my PyTorch code, I cannot access the GPU, and torch.cuda.is_available() returns False.

Here is the output of nvidia-smi:

Sun Jan 19 09:02:48 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03 Driver Version: 560.35.03 CUDA Version: 12.6 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 |
| 0% 23C P8 8W / 300W | 1MiB / 23028MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+


What I’ve Tried:

1. 
   Verified the NVIDIA driver is properly installed.
   
   
2. 
   Confirmed CUDA compatibility with the driver and PyTorch.
   
   
3. 
   Restarted the machine after installation.
   
   
4. 
   Installed pytorch-cuda=12.4 to match the nvcc version.
   
   
5. 
   Ensured pytorch and pytorch-cuda are installed from pytorch and nvidia channels, respectively.
   
   
6. 
   Why is PyTorch unable to detect the GPU?
   
   
7. 
   How can I fix this error and ensure the GPU is accessible for PyTorch?
   

Any guidance would be greatly appreciated. Thank you!

Continue reading...