Check failed: data_ During training!! HEEEELP!!!

[el]

Hello,
I have to deal with the next message error:

I0527 13:39:46.255506 19627 solver.cpp:280] Learning Rate Policy: step
I0527 13:39:46.256721 19627 solver.cpp:337] Iteration 0, Testing net (#0)
F0527 13:42:42.221516 19627 blob.cpp:115] Check failed: data_
*** Check failure stack trace: ***
    @     0x7fc916752daa  (unknown)
    @     0x7fc916752ce4  (unknown)
    @     0x7fc9167526e6  (unknown)
    @     0x7fc916755687  (unknown)
    @     0x7fc916d3fb2e  caffe::Blob<>::mutable_cpu_data()
    @     0x7fc916d6f56d  caffe::hdf5_load_nd_dataset<>()
    @     0x7fc916e7522e  caffe::HDF5DataLayer<>::LoadHDF5FileData()
    @     0x7fc916eb3b45  caffe::HDF5DataLayer<>::Forward_gpu()
    @     0x7fc916e91b25  caffe::Net<>::ForwardFromTo()
    @     0x7fc916e91e97  caffe::Net<>::Forward()
    @     0x7fc916e8299c  caffe::Solver<>::Test()
    @     0x7fc916e832ce  caffe::Solver<>::TestAll()
    @     0x7fc916e833c0  caffe::Solver<>::Step()
    @     0x7fc916e83d89  caffe::Solver<>::Solve()
    @           0x40843e  train()
    @           0x405c8c  main
    @     0x7fc9152aff45  (unknown)
    @           0x40645d  (unknown)
    @              (nil)  (unknown)

I use hdf5 files, containing 1000 samples each, and my batch size is 256 or 512. This message also appears in another case, even in  iteration 100.
I cannot understand if this error comes from some failure of data or from the size of data.
If there was a destroyed image, would this affect hdf5 data layer which is formed earlier than the appearance of the message?
Any ideas?

[Kévin Serru]

Hello there,

I happen to have exactly the same ERROR message. I notice nobody answered your call but since it was a long time ago, if hopefully you managed to find the solution, would you care sharing it with me? Thank you!

Regards.

[HJ]

Hello,

I just got the same error  "blob.cpp:115] Check failed: data_ " . 

In my case, the error occurred when caffe was creating a fully connected layer and the reason was I missed "inner_product_param {

    num_output: 128 }" in that fc layer definition. 

I guess the error shows that some thing related to the blob is missing.

[Kajal Kansal]

I am also getting the same error. Anyone can suggest something to rid off this error?

[Przemek D]

In my case this was a bad mean.binaryproto file. But my failure stack trace looked a bit differently, the error in mutable_cpu_data() call occured during loading of the mean file. In your case it looks like your HDF5 database is in some way corrupted.

[stephen]

but how to deal with the bad mean,binaryproto file ,my  mean.binaryproto is only ten bytes,,and i met  blob.cpp:98] Check failed: data_.  and the error is in caffe::Blob<>::mutable_cpu_data(). my vodeo's framesize is bigger than the size in the C3Ddemo.

在 2016年11月30日星期三 UTC+8下午5:13:08，Przemek D写道：