why do I get the output: Restarting data prefetching from start??

[adi]

hi,

I run the training process on my own dataset through c++ code.

It runs pretty good and creates .solverstate and caffemodel filed, but during the training I get a lot of output "Restarting data prefetching from start"/

I don't understand why?? how can I fix it??

thanks

[el]

I also have the same issue..Please share any comments...

[siebe...@googlemail.com]

Hmm... interesting... this is a debug output that you should only see when you have compiled caffe in debug mode. Try to use (with cmake):

$ cmake . -DCMAKE_BUILD_TYPE=Release

The debug message is output, when the data for one epoch are completely used, and another epoch starts.

[Jan]

For the record: That message does not signal an error, it just informs you that now all provided training samples are consumed (the current epoch is complete), and that the data layer will rewind and start again with the first sample (start the next epoch). Which is completely intended and nothing to worry about.

Jan

[Marco Del Coco]

Is it normal that this output appears at the start of training in this way:

Inserisci qui il codI1124 12:04:18.671671  2508 solver.cpp:273] Learning Rate Policy: step
I1124 12:04:18.673341  2508 solver.cpp:330] Iteration 0, Testing net (#0)
I1124 12:04:18.970029  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:19.325554  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:19.660408  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:20.012720  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:20.366606  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:20.699510  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:21.054208  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:21.388109  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:21.744277  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:22.095875  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:22.431947  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:22.787510  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:23.138142  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:23.472215  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:23.825889  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:24.161841  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:24.514075  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:24.868482  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:25.201861  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:25.555625  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:25.912214  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:26.245303  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:26.597654  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:26.930994  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:27.285454  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:27.642153  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:27.974881  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:28.328384  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:28.683338  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:29.017910  2516 data_layer.cpp:73] Restarting data prefetching from start.
I1124 12:04:29.120671  2508 solver.cpp:384] Test loss: 87.3362
I1124 12:04:29.120693  2508 solver.cpp:397]     Test net output #0: accuracy = 1
I1124 12:04:29.120702  2508 solver.cpp:397]     Test net output #1: loss = 87.3362 (* 1 = 87.3362 loss)
I1124 12:04:29.391324  2508 solver.cpp:218] Iteration 0 (352000 iter/s, 10.7196s/1000 iters), loss = 1.45304
I1124 12:04:29.391356  2508 solver.cpp:237]     Train net output #0: loss = 1.45304 (* 1 = 1.45304 loss)
I1124 12:04:29.391402  2508 sgd_solver.cpp:105] Iteration 0, lr = 0.001
ice...