GPU index
1,307 views
Skip to first unread message
Artem Molchanov
Nov 9, 2015, 10:56:37 PM11/9/15
to Caffe Users
Hi guys,
a simple question. I have 2 GPUs on my Ubuntu 14.04 machine, how can I actually figure out which index in caffe (-gpu X parameter) corresponds to which card ?
Jan C Peters
Nov 10, 2015, 3:13:02 AM11/10/15
to Caffe Users
One way to do it would be running the build/tools/caffe device_query once with -gpu 0 and once with -gpu 1. The outputs should tell you enough about the card to tell which is which. If they are both the same build it will be harder, though. But then it probably does not matter that much which card you use.
Actually I think the X you give to -gpu corresponds directly to the device id, which you can also look up in the nvidia-settings.
Jan
Actually I think the X you give to -gpu corresponds directly to the device id, which you can also look up in the nvidia-settings.
Jan
Artem Molchanov
Jan 13, 2016, 7:19:58 PM1/13/16
to Caffe Users
Thanks for the reply btw!
Here is another solution (in case someone interested):nvidia-smi is nvidia tool allowing you to see which processes use which GPU. So it is enough to run caffe and see which card it is running. For example, in my case:
+------------------------------------------------------+
| NVIDIA-SMI 352.55 Driver Version: 352.55 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 470 Off | 0000:02:00.0 N/A | N/A |
| 40% 62C P12 N/A / N/A | 61MiB / 1279MiB | N/A Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX TIT... Off | 0000:03:00.0 Off | N/A |
| 22% 41C P2 79W / 250W | 211MiB / 12287MiB | 34% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 Not Supported |
| 1 4444 C caffe 186MiB |
+-----------------------------------------------------------------------------+
And no, system indexing is not the same as caffe indexing. For example, by default caffe uses gpu=0, but, as one can see, it has index in the output.
Felix Abecassis
Jan 13, 2016, 9:24:27 PM1/13/16
to Caffe Users
That's right, the CUDA ordering is different from the nvidia-smi ordering.
The ordering used by Caffe should be the same as the one reported by "deviceQuery".
Or, you could also use the environment variable CUDA_DEVICE_ORDER to match the nvidia-smi ordering, something like:
CUDA_DEVICE_ORDER=PCI_BUS_ID caffe train [...]
See:
http://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars
The ordering used by Caffe should be the same as the one reported by "deviceQuery".
Or, you could also use the environment variable CUDA_DEVICE_ORDER to match the nvidia-smi ordering, something like:
CUDA_DEVICE_ORDER=PCI_BUS_ID caffe train [...]
See:
http://docs.nvidia.com/cuda/cuda-c-programming-guide/#env-vars
Reply all
Reply to author
Forward
0 new messages