Skip to first unread message
Alberto Suárez
Mar 31, 2017, 1:40:18 PM3/31/17
to Caffe Users
Hello,
I'm trying to profiling a network with caffe using this:
nvprof caffe train -solver=4_PCAs_filtro10_Solver.prototxt
When I execute caffe without the nvprof profiler it works fine, but if I use the nvprof it tell me that caffe has received a SIGSEGV signal:
...
I0331 19:12:23.215083 21890 solver.cpp:244] Train net output #0: accuracy = 0.03125
I0331 19:12:23.215091 21890 solver.cpp:244] Train net output #1: loss = 2.92478 (* 1 = 2.92478 loss)
I0331 19:12:23.215111 21890 sgd_solver.cpp:106] Iteration 0, lr = 0.01
*** Aborted at 1490980350 (unix time) try "date -d @1490980350" if you are using GNU date ***
PC: @ 0x7fe7019ace78 (unknown)
*** SIGSEGV (@0x0) received by PID 21890 (TID 0x7fe72c135a40) from PID 0; stack trace: ***
@ 0x7fe729fdbcb0 (unknown)
@ 0x7fe7019ace78 (unknown)
@ 0x7fe70197ca33 (unknown)
@ 0x7fe7019812bb (unknown)
@ 0x7fe7024ab607 (unknown)
@ 0x7fe702515928 (unknown)
@ 0x7fe702515dc8 (unknown)
@ 0x7fe702555e9c (unknown)
@ 0x7fe7024b0af8 (unknown)
@ 0x7fe7024b1098 (unknown)
@ 0x7fe7023d57c2 (unknown)
@ 0x7fe7023d595a (unknown)
@ 0x7fe70250fad4 (unknown)
@ 0x7fe72b2110a2 (unknown)
@ 0x7fe72b1ed7c1 (unknown)
@ 0x7fe72b217412 (unknown)
@ 0x7fe72b610b30 caffe::caffe_copy<>()
@ 0x7fe72b7851e3 caffe::BasePrefetchingDataLayer<>::Forward_gpu()
@ 0x7fe72b742b45 caffe::Net<>::ForwardFromTo()
@ 0x7fe72b742eb7 caffe::Net<>::Forward()
@ 0x7fe72b5f5d97 caffe::Solver<>::Step()
@ 0x7fe72b5f6659 caffe::Solver<>::Solve()
@ 0x40873b train()
@ 0x405b3c main
@ 0x7fe729fc6f45 (unknown)
@ 0x4063ab (unknown)
@ 0x0 (unknown)
segmentation fault
Have anyone profiled a network with caffe using nvprof or nvvp? What is wrong?
Thanks a lot
Best regards,
Alberto.
I'm trying to profiling a network with caffe using this:
nvprof caffe train -solver=4_PCAs_filtro10_Solver.prototxt
When I execute caffe without the nvprof profiler it works fine, but if I use the nvprof it tell me that caffe has received a SIGSEGV signal:
...
I0331 19:12:23.215083 21890 solver.cpp:244] Train net output #0: accuracy = 0.03125
I0331 19:12:23.215091 21890 solver.cpp:244] Train net output #1: loss = 2.92478 (* 1 = 2.92478 loss)
I0331 19:12:23.215111 21890 sgd_solver.cpp:106] Iteration 0, lr = 0.01
*** Aborted at 1490980350 (unix time) try "date -d @1490980350" if you are using GNU date ***
PC: @ 0x7fe7019ace78 (unknown)
*** SIGSEGV (@0x0) received by PID 21890 (TID 0x7fe72c135a40) from PID 0; stack trace: ***
@ 0x7fe729fdbcb0 (unknown)
@ 0x7fe7019ace78 (unknown)
@ 0x7fe70197ca33 (unknown)
@ 0x7fe7019812bb (unknown)
@ 0x7fe7024ab607 (unknown)
@ 0x7fe702515928 (unknown)
@ 0x7fe702515dc8 (unknown)
@ 0x7fe702555e9c (unknown)
@ 0x7fe7024b0af8 (unknown)
@ 0x7fe7024b1098 (unknown)
@ 0x7fe7023d57c2 (unknown)
@ 0x7fe7023d595a (unknown)
@ 0x7fe70250fad4 (unknown)
@ 0x7fe72b2110a2 (unknown)
@ 0x7fe72b1ed7c1 (unknown)
@ 0x7fe72b217412 (unknown)
@ 0x7fe72b610b30 caffe::caffe_copy<>()
@ 0x7fe72b7851e3 caffe::BasePrefetchingDataLayer<>::Forward_gpu()
@ 0x7fe72b742b45 caffe::Net<>::ForwardFromTo()
@ 0x7fe72b742eb7 caffe::Net<>::Forward()
@ 0x7fe72b5f5d97 caffe::Solver<>::Step()
@ 0x7fe72b5f6659 caffe::Solver<>::Solve()
@ 0x40873b train()
@ 0x405b3c main
@ 0x7fe729fc6f45 (unknown)
@ 0x4063ab (unknown)
@ 0x0 (unknown)
segmentation fault
Have anyone profiled a network with caffe using nvprof or nvvp? What is wrong?
Thanks a lot
Best regards,
Alberto.
Alberto Suárez
Apr 3, 2017, 5:27:51 AM4/3/17
to Caffe Users
It seems to work reducing the max_iter number from 10000 to 1000, maybe 10000 is a bunch of iterations.
Reply all
Reply to author
Forward
0 new messages