[Python] Using evaluate library to evaluate BertScore only uses 1 busy GPU

I'm using evaluate library to evaluate the BertScore. Here are my codes:

import evaluate
bertscore = evaluate.load("bertscore")
bertscore_result = bertscore.compute(predictions=[sentence], references=references_sentence)
bertscore_avg = np.mean(bertscore_result["f1"])

print("BertScore:", bertscore_avg)


I am using GeForce 4090 GPU, which has two GPU units, each with 24GB RAM. Here is the output of the nvidia-smi:

Wed May 28 00:11:09 2025
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 Off | 00000000:81:00.0 Off | Off |
| 0% 31C P8 21W / 450W | 14MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce RTX 4090 Off | 00000000:C1:00.0 Off | Off |
| 30% 28C P8 30W / 450W | 24004MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 1720 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 1720 G /usr/lib/xorg/Xorg 15MiB |
| 1 N/A N/A 1896 G /usr/bin/gnome-shell 10MiB |
| 1 N/A N/A 619637 C /opt/tljh/user/bin/python 23962MiB |
+---------------------------------------------------------------------------------------+


As you can see the GPU unit 1 is almost occupied, but the GPU unit 0 is idle. However, the code insists using GPU unit 1 and results in:

RuntimeError: CUDA error: out of memory
​

Of course I can kill other GPU-intensive process, but I would like to explore whether the evaluate library codes are smart enough to utlize the idle GPU unit.

Is this ever possible? Thanks in advance.

Continue reading...