Selecting which device to run Session

116 views
Skip to first unread message

Kriti Tulsian

unread,
Jul 8, 2020, 7:50:44 AM7/8/20
to Rust for TensorFlow

Hi, 
I created 2 virtual GPU's from a physical GPU. I want to run two sessions each on 1 virtual GPU. But I'm not able to select the GPU I want to use. 
In python something like this is used - tf.device('/gpu:2')
How can I achieve the same behavior in rust?

Adam Crume

unread,
Jul 8, 2020, 11:20:58 AM7/8/20
to Kriti Tulsian, Rust for TensorFlow
How are you building the op?  Are you using the low-level code (Graph::new_operation) or the high-level code (e.g. ops::add)?  This isn't supported by the high-level code yet (although it shouldn't be too difficult to add), but you can call OperationDescription::set_device when building the op with the low-level code.

If you don't need to place individual ops on GPUs and want to run an entire session on a particular GPU (which is what it sounds like), you can try creating a  ConfigProto proto, setting gpu_options.visible_device_list (see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto), serializing it, and passing that to SessionOptions::set_config.

--
You received this message because you are subscribed to the Google Groups "Rust for TensorFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rust+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/rust/7f41c05f-764d-4323-9f8a-ca55936538fco%40tensorflow.org.

Kriti Tulsian

unread,
Jul 8, 2020, 11:35:51 AM7/8/20
to Rust for TensorFlow, kriti.t...@gmail.com

Kriti Tulsian

11:34 PM (0 minutes ago)
to Adam
Yes, I want to run an entire session on a particular GPU. I have tried the ConfigProto proto option. 

I try to use the below (GPU Options in Config Proto)-

let mut config = ConfigProto::new();
let mut gpu_option = GPUOptions::new();
gpu_option.set_visible_device_list("/device:GPU:1".to_string());
config.set_gpu_options(gpu_option);

But I get this error - 

InvalidArgument: Could not parse entry in 'visible_device_list'


When I try to print the device list -  [Device { name: "/job:localhost/replica:0/task:0/device:CPU:0", device_type: "CPU", memory_bytes: 268435456, incarnation: 11303449111836618748 }, Device { name: "/job:localhost/replica:0/task:0/device:GPU:0", device_type: "GPU", memory_bytes: 1073741824, incarnation: 2389579074634507097 }, Device { name: "/job:localhost/replica:0/task:0/device:GPU:1", device_type: "GPU", memory_bytes: 1073741824, incarnation: 127789045278819517 }] device list


GPU:1 is a part of the device list? Where am I going wrong?









On Wednesday, July 8, 2020 at 11:20:58 PM UTC+8, Adam Crume wrote:
How are you building the op?  Are you using the low-level code (Graph::new_operation) or the high-level code (e.g. ops::add)?  This isn't supported by the high-level code yet (although it shouldn't be too difficult to add), but you can call OperationDescription::set_device when building the op with the low-level code.

If you don't need to place individual ops on GPUs and want to run an entire session on a particular GPU (which is what it sounds like), you can try creating a  ConfigProto proto, setting gpu_options.visible_device_list (see https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto), serializing it, and passing that to SessionOptions::set_config.

On Wed, Jul 8, 2020 at 4:50 AM Kriti Tulsian <kriti.t...@gmail.com> wrote:

Hi, 
I created 2 virtual GPU's from a physical GPU. I want to run two sessions each on 1 virtual GPU. But I'm not able to select the GPU I want to use. 
In python something like this is used - tf.device('/gpu:2')
How can I achieve the same behavior in rust?

--
You received this message because you are subscribed to the Google Groups "Rust for TensorFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ru...@tensorflow.org.

Adam Crume

unread,
Jul 8, 2020, 12:35:02 PM7/8/20
to Kriti Tulsian, Rust for TensorFlow
From the docs on visible_device_list, I think you want to set it to just "1".

To unsubscribe from this group and stop receiving emails from it, send an email to rust+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/rust/67f8ff18-7e72-42aa-8298-f9455c9deff5o%40tensorflow.org.

Kriti Tulsian

unread,
Jul 8, 2020, 12:48:32 PM7/8/20
to Adam Crume, Rust for TensorFlow
I still get an error. 

InvalidArgument: 'visible_device_list' listed an invalid GPU id '1' but visible device count is 1'


My system has 1 physical GPU from which I'm creating 2 virtual GPU's as shown in the device list in my previous comment. I think I'm trying to access the GPU before it is created. Here is my code - 
let mut session_option = SessionOptions::new();
let mut gpu_option = GPUOptions::new();
let mut gpu_exp_option = GPUOptions_Experimental::new();
let mut gpu_virtual_option = GPUOptions_Experimental_VirtualDevices::new();
gpu_virtual_option.set_memory_limit_mb(vec![1024.0,1024.0]);
let mut gpu_virtual_vec: Vec<GPUOptions_Experimental_VirtualDevices> = Vec::new();
gpu_virtual_vec.push(gpu_virtual_option);

let rep_field = protobuf::RepeatedField::from_vec(gpu_virtual_vec);
gpu_exp_option.set_virtual_devices(rep_field);
gpu_option.set_experimental(gpu_exp_option);
gpu_option.set_visible_device_list("1".to_string());
let mut config = ConfigProto::new();
config.set_gpu_options(gpu_option);
config.set_log_device_placement(true);
let config_bytes = config.write_to_bytes().unwrap();
session_option.set_config(&config_bytes)?;

Adam Crume

unread,
Jul 10, 2020, 12:14:13 PM7/10/20
to Kriti Tulsian, Rust for TensorFlow
Ah, I see.  virtual_devices gets applied *after* visible_device_list.  Will device_filters do what you want?  If not, I'm afraid I'm out of my depth here. You might want to try asking the wider TensorFlow community.  I can help with the Rust pieces, but I don't see how to change the specific configuration to do what you want.  It looks like you could set the device in the graph nodes to where they should be, but of course that's less flexible.  I'm working on a Scope::with_device implementation which would allow that.

Kriti Tulsian

unread,
Jul 10, 2020, 2:00:24 PM7/10/20
to Adam Crume, Rust for TensorFlow
I could probably try using device_filters. Could you link me with how I can use it? I basically want to do what this python code does to set a particular device to run a session/operation on - `tf.device("GPU:0")` 
Also, is there any other way to make my session run on multiple GPU's? Because even though I have created two virtual GPU's, my sessions run only on the single physical GPU. 
I set the `log_device_placement` in `ConfigProto` to check which device is being used

鄭凱仁

unread,
Mar 8, 2021, 5:30:13 AM3/8/21
to Rust for TensorFlow, Kriti Tulsian, Rust for TensorFlow, acr...@google.com
Hi, everyone. I'm new here. I would like to know where do you get all this "ConfigProto" struct and "GPUOption" struct to set the GPU option in rust.
I can't find any clue in rust TensorFlow binding documentation. Thanks in advance!

Cedric
Kriti Tulsian 在 2020年7月11日 星期六上午2:00:24 [UTC+8] 的信中寫道:
Reply all
Reply to author
Forward
0 new messages