gtx1070 caffe train digit error (error == cudaSuccess (8 vs. 0) invalid device function)

342 views
Skip to first unread message

Li Zhang

unread,
Aug 4, 2016, 3:36:18 AM8/4/16
to Caffe Users


╭─root at 5420e0794e25 in ~ using
╰─○ ls -al /dev | grep nvidia
crw-rw-rw- 1 root root 243, 0 Aug 4 06:01 nvidia-uvm
crw-rw-rw- 1 root root 243, 1 Aug 4 06:07 nvidia-uvm-tools
crw-rw-rw- 1 root root 195, 0 Aug 4 06:01 nvidia0
crw-rw-rw- 1 root root 195, 255 Aug 4 06:01 nvidiactl
╭─root at 5420e0794e25 in ~ using
╰─○ nvidia-smi
Thu Aug 4 07:09:57 2016
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.35 Driver Version: 367.35 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 1070 Off | 0000:01:00.0 On | N/A |
| 0% 34C P8 10W / 230W | 514MiB / 8105MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+
╭─root at 5420e0794e25 in ~ using
╰─○ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Wed_May__4_21:01:56_CDT_2016
Cuda compilation tools, release 8.0, V8.0.26


F0804 07:20:57.251991 961 pooling_layer.cu:212] Check failed: error == cudaSuccess (8 vs. 0) invalid device function *** Check failure stack trace: *** @ 0x7f8b75c42daa (unknown) @ 0x7f8b75c42ce4 (unknown) @ 0x7f8b75c426e6 (unknown) @ 0x7f8b75c45687 (unknown) @ 0x7f8b76344410 caffe::PoolingLayer<>::Forward_gpu() @ 0x7f8b7619b748 caffe::Net<>::ForwardFromTo() @ 0x7f8b7619bac7 caffe::Net<>::Forward() @ 0x7f8b7633bfdc caffe::Solver<>::Test() @ 0x7f8b7633ca4e caffe::Solver<>::TestAll() @ 0x7f8b7633cb40 caffe::Solver<>::Step() @ 0x7f8b7633d4f9 caffe::Solver<>::Solve() @ 0x40b545 train() @ 0x40897c main @ 0x7f8b74440f45 (unknown) @ 0x409283 (unknown) @ (nil) (unknown) Aborted (core dumped)

Leon Truong

unread,
Aug 4, 2016, 7:08:47 PM8/4/16
to Caffe Users
Have you make sure to modify your makefile.config and set your cuda_arch for GTX 1070?

eg.  -gencode arch=compute_61,code=sm_61

Li Zhang

unread,
Aug 5, 2016, 6:50:35 AM8/5/16
to Caffe Users
Thanks. I solve my problem with cmake flags.

cmake -DCUDA_ARCH_NAME=Manual -DCUDA_ARCH_BIN="61" -DCUDA_ARCH_PTX="61" -DUSE_CUDNN=1



在 2016年8月5日星期五 UTC+8上午7:08:47，Leon Truong写道：

张超

unread,
Dec 6, 2016, 6:04:40 AM12/6/16
to Caffe Users

how do you know, gtx1070 will use 61? how to check it.

Przemek D

unread,
Dec 19, 2016, 4:01:25 AM12/19/16
to Caffe Users
Reply all
Reply to author
Forward
0 new messages