Training stops at iteration 0?

[Leon]

Hi There, 

    I am trying to supply my own data with the hdf5_classification sample net (train_val2.prototxt). 

    The training part seems to be able to complete, but when it does test, it simply stops at Iteration 0, Testing net (#0) with no warnings / errors. Does anyone might know whats going on?

    My data is HD5F with dimensions like (10000, 40, 1, 1), so it is thousands of 40 elements vectors, and each element is hashed into f64

The last couple lines of STDOUT is shown below:

1126 21:41:30.808037 20065 net.cpp:219] Network initialization done.

I1126 21:41:30.808043 20065 net.cpp:220] Memory required for data: 5048

I1126 21:41:30.808064 20065 solver.cpp:41] Solver scaffolding done.

I1126 21:41:30.808073 20065 solver.cpp:160] Solving Process Monitor

I1126 21:41:30.808086 20065 solver.cpp:247] Iteration 0, Testing net (#0)

[Evan Shelhamer]

Check your solver.prototxt settings to make sure test_iter is correct -- it could just be looping over your held-out data over and over.

Evan Shelhamer

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/92cc66b2-fc52-45df-adbe-fb320235054a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Leon]

Hi Evan, 

    Very appreciated for your help! My test_itr is set to 1000, which is essentially the same as what HDF5_classification sample gives me. When I supply it with the synthetic data (random data etc).. it works, but when I give it some useful data, it just terminates at iteration 0. Does it has anything to do with the distribution of my data ?

[Leon]

I am still not sure what's going on. I have double checked my solver, and the test_itr is at 1000, when I give it random data generated from python sklearn classification, it works great!. But when I use my process information the ones get from ps command, the training terminates at iteration 0. I have looked everywhere and it all seems right to me. 

my net is the HDF5 classification sample net. 

anyone has any pointers ? any help would be deeply appreciated!

Thanks,

Leon.