[Python] Why is GPU execution time much higher than CPU execution time?

I just installed new versions of CuPy and CUDA and I observed that the execution time for some code I used to run increased dramatically. The execution time on GPU is much longer than that on CPU.

Do you have any advice what I can do to reverse this situation?

Here is an example:

import cupy as cp
from cupyx.profiler import benchmark


xi_tr = cp.random.random( size = (20,5) , dtype = 'float32')
xi_te = cp.random.random( size = (20,5) , dtype = 'float32')
n_tr_samples = xi_te.shape[0]

def test_func( tr , te , n_samples):
inputs_3d = cp.tile( te , (n_samples , 1 , 1))
tr_3d = tr[: , cp.newaxis , :]
cl_dif = (tr_3d - inputs_3d)
tot_d = (cl_dif * cl_dif).sum(axis=2)
return tot_d

result = benchmark( test_func , args = ( xi_tr , xi_te , n_tr_samples ) , n_repeat = 10)

print( result )

test_func :
CPU: 139.340 us +/- 24.145 (min: 127.600 / max: 210.800) us
GPU-0: 260.102 us +/- 49.430 (min: 238.592 / max: 407.552) us



cp.cuda.is_available()

Out[96]: True

cp.cuda.runtime.getDeviceCount()

Out[97]: 1

cp.show_config()

OS : Windows-10-10.0.19045-SP0

Python Version : 3.12.3

CuPy Version : 13.3.0

CuPy Platform : NVIDIA CUDA

NumPy Version : 1.26.4

SciPy Version : 1.13.1

Cython Build Version : 0.29.37

Cython Runtime Version : None

CUDA Root : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8

nvcc PATH : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8\bin\nvcc.EXE

CUDA Build Version : 12060

CUDA Driver Version : 12080

CUDA Runtime Version : 12060 (linked to CuPy) / RuntimeError("CuPy failed to load cudart64_12.dll: FileNotFoundError: Could not find module 'cudart64_12.dll' (or one of its dependencies). Try using the full path with constructor syntax.") (locally installed)

CUDA Extra Include Dirs : ['C:\\ProgramData\\anaconda3\\Library\\include']

cuBLAS Version : (available)

cuFFT Version : 11303

cuRAND Version : 10309

cuSOLVER Version : (11, 7, 2)

cuSPARSE Version : (available)

NVRTC Version : (12, 8)
Thrust Version : 200600

CUB Build Version : 200600

Jitify Build Version : <unknown>

cuDNN Build Version : None

cuDNN Version : None

NCCL Build Version : None

NCCL Runtime Version : None

cuTENSOR Version : None

cuSPARSELt Build Version : None

Device 0 Name : NVIDIA GeForce GTX 1080 Ti

Device 0 Compute Capability : 61

Device 0 PCI Bus ID : 0000:02:00.0


Additional info: I used CUDA v11.2 and CuPy v12.3 before. I haven’t benchmarked GPU vs CPU than. Just execution time using cupy was 5-10 times lower than using numpy and overall was OK. But after this update execution time increased dramatically > 10 times.

The size of the arrays that I usually use is about (5000 , 30 ). In the example code I put small numbers just for illustration. Here are results for (5000 , 30 ): test_func :
CPU: 222.990 us +/- 25.716 (min: 169.800 / max: 258.000) us
GPU-0: 210689.948 us +/- 22235.564 (min: 201992.447 / max: 277377.014) us

Continue reading...