[Python] Torch CUDA initialization: Unexpected error from cudaGetDeviceCount()

i want to do speaker diarization using pyannote/speaker-diarization-3.1 but when i did this code

import torch

torch.cuda.is_available()


it throws this error

>>> import torch
>>> torch.cuda.is_available()
/root/.pyenv/versions/3.10.15/lib/python3.10/site-packages/torch/cuda/__init__.py:129: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
return torch._C._cuda_getDeviceCount() > 0
False


i have something like this in my dockerfile, the installation works fine - i used the latest base image with cuda toolkit 12.6.2

FROM nvidia/cuda:12.6.2-cudnn-devel-ubuntu24.04

RUN bash -i -c "source ~/.bashrc && python -m pip install llvmlite torch torchaudio --extra-index-url https://download.pytorch.org/whl/cu121"


here are my nvidia-smi result (rtx 3060ti) - i update the driver to latest

# nvidia-smi
Mon Nov 11 03:48:10 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.51.01 Driver Version: 565.90 CUDA Version: 12.7 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3060 Ti On | 00000000:01:00.0 On | N/A |
| 0% 44C P8 9W / 200W | 810MiB / 8192MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+


i am not sure if this is due to the cuda version and cuda toolkit missmatch, my cuda version is 12.7 but my cuda toolkit is 12.6.2 - (i cant find the 12.7 version)

i am running this through docker using windows wsl, nvidia-smi in wsl and inside docker have same result

Continue reading...