Tensorflow not detecting GTX 1080 Ti

1,749 views
Skip to first unread message

Watts

unread,
Mar 18, 2017, 4:55:53 PM3/18/17
to Discuss
HI,

I just got a GTX 1080 Ti for deep learning. I have installed it on a Ubuntu 14.04 machine along with Tensorflow 1.0.1 and Python 2.7. I cant get tensorflow to detect the card. 
nvidia-smi detects it as a "Graphics Device"

Any pointers are much appreciated.

Here's the output of nvidia-smi

Sun Mar 19 02:16:01 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 378.13                 Driver Version: 378.13                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Graphics Device     Off  | 0000:03:00.0      On |                  N/A |
| 28%   47C    P8    12W / 250W |    324MiB / 11169MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1351    G   /usr/lib/xorg/Xorg                             183MiB |
|    0      2378    G   compiz                                          44MiB |
|    0      2587    G   ...s-passed-by-fd --v8-snapshot-passed-by-fd    94MiB |
+-----------------------------------------------------------------------------+


And output of lspci | grep -i nvidia

watts@Magnus:~/NVIDIA_CUDA-8.0_Samples/1_Utilities/deviceQuery$ lspci | grep -i nvidia
03:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)

Pablo de Azevedo

unread,
Mar 18, 2017, 5:01:12 PM3/18/17
to Watts, Discuss
Check your gpu drive version.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/b5f9c4cf-a43e-4b6b-a23d-9577f5e1510e%40tensorflow.org.

Watts

unread,
Mar 18, 2017, 5:16:07 PM3/18/17
to Discuss
I have installed CUDA 8.0 with cuDNN 5.1 The display driver version is 378.13 (which is the latest at the time of writing)
deviceQuery works correctly. 

Where am I going wrong?
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

Isaac Gerg

unread,
Mar 18, 2017, 5:38:40 PM3/18/17
to Watts, Discuss
nvidia-smi can dump out much more than what you've written.  Do you have CUDA compute enabled for the device?

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
Message has been deleted

Watts

unread,
Mar 18, 2017, 6:05:40 PM3/18/17
to Discuss
How do I enable CUDA compute for the device?

Dump of nvidia-smi -q is below

nvidia-smi --gom=1 gives the following. Perhaps because gom is set to 0 (all enabled)
watts@Magnus:~/Downloads$ sudo nvidia-smi --gom=1
GOM mode cannot be changed on GPU 0000:03:00.0.
Treating as warning and moving on.
All done.

watts@Magnus:$ nvidia-smi -q
==============NVSMI LOG==============

Timestamp                           : Sun Mar 19 03:17:05 2017
Driver Version                      : 378.13

Attached GPUs                       : 1
GPU 0000:03:00.0
    Product Name                    : Graphics Device
    Product Brand                   : GeForce
    Display Mode                    : Enabled
    Display Active                  : Enabled
    Persistence Mode                : Disabled
    Accounting Mode                 : Disabled
    Accounting Mode Buffer Size     : 1920
    Driver Model
        Current                     : N/A
        Pending                     : N/A
    Serial Number                   : 0320817035626
    GPU UUID                        : GPU-cdb04047-0685-d250-09e6-cd7324a9a7ac
    Minor Number                    : 0
    VBIOS Version                   : 86.02.39.00.01
    MultiGPU Board                  : No
    Board ID                        : 0x300
    GPU Part Number                 : 900-1G611-0050-000
    Inforom Version
        Image Version               : G001.0000.01.04
        OEM Object                  : 1.1
        ECC Object                  : N/A
        Power Management Object     : N/A
    GPU Operation Mode
        Current                     : N/A
        Pending                     : N/A
    GPU Virtualization Mode
        Virtualization mode         : None
    PCI
        Bus                         : 0x03
        Device                      : 0x00
        Domain                      : 0x0000
        Device Id                   : 0x1B0610DE
        Bus Id                      : 0000:03:00.0
        Sub System Id               : 0x85E21043
        GPU Link Info
            PCIe Generation
                Max                 : 3
                Current             : 1
            Link Width
                Max                 : 16x
                Current             : 16x
        Bridge Chip
            Type                    : N/A
            Firmware                : N/A
        Replays since reset         : 0
        Tx Throughput               : 0 KB/s
        Rx Throughput               : 4000 KB/s
    Fan Speed                       : 28 %
    Performance State               : P8
    Clocks Throttle Reasons
        Idle                        : Active
        Applications Clocks Setting : Not Active
        SW Power Cap                : Not Active
        HW Slowdown                 : Not Active
        Sync Boost                  : Not Active
        Unknown                     : Not Active
    FB Memory Usage
        Total                       : 11169 MiB
        Used                        : 327 MiB
        Free                        : 10842 MiB
    BAR1 Memory Usage
        Total                       : 256 MiB
        Used                        : 8 MiB
        Free                        : 248 MiB
    Compute Mode                    : Default
    Utilization
        Gpu                         : 0 %
        Memory                      : 2 %
        Encoder                     : 0 %
        Decoder                     : 0 %
    Ecc Mode
        Current                     : N/A
        Pending                     : N/A
    ECC Errors
        Volatile
            Single Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                Total               : N/A
            Double Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                Total               : N/A
        Aggregate
            Single Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                Total               : N/A
            Double Bit            
                Device Memory       : N/A
                Register File       : N/A
                L1 Cache            : N/A
                L2 Cache            : N/A
                Texture Memory      : N/A
                Texture Shared      : N/A
                Total               : N/A
    Retired Pages
        Single Bit ECC              : N/A
        Double Bit ECC              : N/A
        Pending                     : N/A
    Temperature
        GPU Current Temp            : 46 C
        GPU Shutdown Temp           : 96 C
        GPU Slowdown Temp           : 93 C
    Power Readings
        Power Management            : Supported
        Power Draw                  : 13.23 W
        Power Limit                 : 250.00 W
        Default Power Limit         : 250.00 W
        Enforced Power Limit        : 250.00 W
        Min Power Limit             : 125.00 W
        Max Power Limit             : 300.00 W
    Clocks
        Graphics                    : 139 MHz
        SM                          : 139 MHz
        Memory                      : 405 MHz
        Video                       : 40 MHz
    Applications Clocks
        Graphics                    : N/A
        Memory                      : N/A
    Default Applications Clocks
        Graphics                    : N/A
        Memory                      : N/A
    Max Clocks
        Graphics                    : 1911 MHz
        SM                          : 1911 MHz
        Memory                      : 5505 MHz
        Video                       : 1708 MHz
    Clock Policy
        Auto Boost                  : N/A
        Auto Boost Default          : N/A
    Processes
        Process ID                  : 1351
            Type                    : G
            Name                    : /usr/lib/xorg/Xorg
            Used GPU Memory         : 183 MiB
        Process ID                  : 2378
            Type                    : G
            Name                    : compiz
            Used GPU Memory         : 53 MiB
        Process ID                  : 2587
            Type                    : G
            Name                    : /opt/google/chrome/chrome --type=gpu-process --field-trial-handle=1 --enable-crash-reporter=63587760-f26d-4381-9a76-e07bb5071ee1, --supports-dual-gpus=false --gpu-driver-bug-workarounds=7,19,23,42,50,54,63,74,85,87,93 --disable-gl-extensions=GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent --gpu-vendor-id=0x10de --gpu-device-id=0x1b06 --gpu-driver-vendor=NVIDIA --gpu-driver-version=378.13 --gpu-driver-date --enable-crash-reporter=63587760-f26d-4381-9a76-e07bb5071ee1, --service-request-channel-token=59BBB85158EA8E42E8C2B60BD363897E --v8-natives-passed-by-fd --v8-snapshot-passed-by-fd
            Used GPU Memory         : 87 MiB

Watts

unread,
Mar 18, 2017, 6:17:56 PM3/18/17
to Discuss
Here's the output of deviceQuery -

watts@Magnus:~/NVIDIA_CUDA-8.0_Samples/1_Utilities/deviceQuery$ ./deviceQuery ./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "Graphics Device"
CUDA Driver Version / Runtime Version 8.0 / 8.0
CUDA Capability Major/Minor version number: 6.1
Total amount of global memory: 11169 MBytes (11711938560 bytes)
(28) Multiprocessors, (128) CUDA Cores/MP: 3584 CUDA Cores
GPU Max Clock rate: 1582 MHz (1.58 GHz)
Memory Clock rate: 5505 Mhz
Memory Bus Width: 352-bit
L2 Cache Size: 2883584 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 3 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = Graphics Device
Result = PASS

Vijay Vasudevan

unread,
Mar 18, 2017, 7:10:25 PM3/18/17
to Watts, Discuss
How did you install TensorFlow? (did you pip install 'tensorflow-gpu' or 'tensorflow' ?)

Startup logs from TF would be useful.  However, this is something that could be posted to StackOverflow tagged with 'tensorflow', since we try to avoid making this list a help forum.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.

Watts

unread,
Mar 18, 2017, 7:16:19 PM3/18/17
to Discuss, v...@google.com

That was it! Muchos gracias.

Isaac Gerg

unread,
Mar 18, 2017, 7:32:25 PM3/18/17
to Watts, Discuss
Dumb question but how did you install tensorflow?  If by pip, did you use pip install tensorflow-gpu?

Watts

unread,
Mar 18, 2017, 7:34:19 PM3/18/17
to Discuss
Yes, pip install tensorflow-gpu 

lms...@gmail.com

unread,
Mar 19, 2017, 4:35:49 PM3/19/17
to Discuss, v...@google.com
Just to make it clear: you was able to use 1080 Ti by issuing pip install tensorflow-gpu  ? 

Watts

unread,
Mar 19, 2017, 5:29:09 PM3/19/17
to Discuss, lms...@gmail.com
yes, the card is being detected now and I ran some simple programs using the gpu.
Reply all
Reply to author
Forward
0 new messages