[Python] How can I say in which order should CUDA use the memory

Iam trying to run a model which is arround 13GB of size. I have hardware configuartions like so.

Device 0 - GPU 0 - 12 GB Vram
Device 1 - GPU 1 - 12 GB Vram
Device 2 - GPU 2 - 8 GB Vram.
Device 3 - GPU 3 - 8 GB Vram.
IGPU(Integrated GPU) - GPU comes with processor - CPU


My issue is as model is 13Gb which defaulty starts with GPU0 and then offloads to GPU1 if it reached threshold iam doing like so.

set CUDA_VISIBLE_DEVICES=0,1,2,3


But I want to say cuda to use a IGPU instead of offloading to a new GPU for only that 1GB of extra usage. which ultimatly wastage of resource and that GPU cannot be assigned if new user is requesting a resource.

Like.

set CUDA_VISIBLE_DEVICES=0,IGPU,1,IGPU,2,IGPU,3. #IPGU is a not a valid keyword


Iam looking for

1. Is there is any keyword or parameter that I can use to say cuda that i want to use IGPU. I thought 0 is a IGPU but it's not.

Continue reading...