Selecting which device to run Session

[Kriti Tulsian]

Hi, 
I created 2 virtual GPU's from a physical GPU. I want to run two sessions each on 1 virtual GPU. But I'm not able to select the GPU I want to use. 
In python something like this is used - tf.device('/gpu:2')

How can I achieve the same behavior in rust?

[Adam Crume]

How are you building the op?  Are you using the low-level code (Graph::new_operation) or the high-level code (e.g. ops::add)?  This isn't supported by the high-level code yet (although it shouldn't be too difficult to add), but you can call OperationDescription::set_device when building the op with the low-level code.

If you don't need to place individual ops on GPUs and want to run an entire session on a particular GPU (which is what it sounds like), you can try creating a  ConfigProto proto, setting gpu_options.visible_device_list (see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto), serializing it, and passing that to SessionOptions::set_config.

--
You received this message because you are subscribed to the Google Groups "Rust for TensorFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rust+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/rust/7f41c05f-764d-4323-9f8a-ca55936538fco%40tensorflow.org.

[Kriti Tulsian]

Kriti Tulsian	11:34 PM (0 minutes ago)
to Adam

Yes, I want to run an entire session on a particular GPU. I have tried the ConfigProto proto option. 

I try to use the below (GPU Options in Config Proto)-

let mut config = ConfigProto::new();

let mut gpu_option = GPUOptions::new();

gpu_option.set_visible_device_list("/device:GPU:1".to_string());

config.set_gpu_options(gpu_option);

But I get this error - 

InvalidArgument: Could not parse entry in 'visible_device_list'

When I try to print the device list -  [Device { name: "/job:localhost/replica:0/task:0/device:CPU:0", device_type: "CPU", memory_bytes: 268435456, incarnation: 11303449111836618748 }, Device { name: "/job:localhost/replica:0/task:0/device:GPU:0", device_type: "GPU", memory_bytes: 1073741824, incarnation: 2389579074634507097 }, Device { name: "/job:localhost/replica:0/task:0/device:GPU:1", device_type: "GPU", memory_bytes: 1073741824, incarnation: 127789045278819517 }] device list

GPU:1 is a part of the device list? Where am I going wrong?

On Wednesday, July 8, 2020 at 11:20:58 PM UTC+8, Adam Crume wrote:

How are you building the op?  Are you using the low-level code (Graph::new_operation) or the high-level code (e.g. ops::add)?  This isn't supported by the high-level code yet (although it shouldn't be too difficult to add), but you can call OperationDescription::set_device when building the op with the low-level code.

If you don't need to place individual ops on GPUs and want to run an entire session on a particular GPU (which is what it sounds like), you can try creating a  ConfigProto proto, setting gpu_options.visible_device_list (see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto), serializing it, and passing that to SessionOptions::set_config.

On Wed, Jul 8, 2020 at 4:50 AM Kriti Tulsian <kriti.t...@gmail.com> wrote:

Hi, 
I created 2 virtual GPU's from a physical GPU. I want to run two sessions each on 1 virtual GPU. But I'm not able to select the GPU I want to use. 
In python something like this is used - tf.device('/gpu:2')

How can I achieve the same behavior in rust?

--
You received this message because you are subscribed to the Google Groups "Rust for TensorFlow" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ru...@tensorflow.org.

[Adam Crume]

From the docs on visible_device_list, I think you want to set it to just "1".

To unsubscribe from this group and stop receiving emails from it, send an email to rust+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/rust/67f8ff18-7e72-42aa-8298-f9455c9deff5o%40tensorflow.org.

[Kriti Tulsian]

I still get an error. 

InvalidArgument: 'visible_device_list' listed an invalid GPU id '1' but visible device count is 1'

My system has 1 physical GPU from which I'm creating 2 virtual GPU's as shown in the device list in my previous comment. I think I'm trying to access the GPU before it is created. Here is my code - 

let mut session_option = SessionOptions::new();

let mut gpu_option = GPUOptions::new();

let mut gpu_exp_option = GPUOptions_Experimental::new();

let mut gpu_virtual_option = GPUOptions_Experimental_VirtualDevices::new();

gpu_virtual_option.set_memory_limit_mb(vec![1024.0,1024.0]);

let mut gpu_virtual_vec: Vec<GPUOptions_Experimental_VirtualDevices> = Vec::new();

gpu_virtual_vec.push(gpu_virtual_option);

let rep_field = protobuf::RepeatedField::from_vec(gpu_virtual_vec);

gpu_exp_option.set_virtual_devices(rep_field);

gpu_option.set_experimental(gpu_exp_option);

gpu_option.set_visible_device_list("1".to_string());

let mut config = ConfigProto::new();

config.set_gpu_options(gpu_option);

config.set_log_device_placement(true);

let config_bytes = config.write_to_bytes().unwrap();

session_option.set_config(&config_bytes)?;

[Adam Crume]

Ah, I see.  virtual_devices gets applied *after* visible_device_list.  Will device_filters do what you want?  If not, I'm afraid I'm out of my depth here. You might want to try asking the wider TensorFlow community.  I can help with the Rust pieces, but I don't see how to change the specific configuration to do what you want.  It looks like you could set the device in the graph nodes to where they should be, but of course that's less flexible.  I'm working on a Scope::with_device implementation which would allow that.

[Kriti Tulsian]

I could probably try using device_filters. Could you link me with how I can use it? I basically want to do what this python code does to set a particular device to run a session/operation on - `tf.device("GPU:0")` 
Also, is there any other way to make my session run on multiple GPU's? Because even though I have created two virtual GPU's, my sessions run only on the single physical GPU. 
I set the `log_device_placement` in `ConfigProto` to check which device is being used

[鄭凱仁]

Hi, everyone. I'm new here. I would like to know where do you get all this "ConfigProto" struct and "GPUOption" struct to set the GPU option in rust.

I can't find any clue in rust TensorFlow binding documentation. Thanks in advance!

Cedric

Kriti Tulsian 在 2020年7月11日 星期六上午2:00:24 [UTC+8] 的信中寫道：