Feed my own data into Caffe ( HDF5 )

[Martin Yu]

Hi people,

I have read the tutorial of Caffe and want to feed my own data into lenet. I converted MNIST data on Kaggle, which is of csv form, to HDF5. The resulting HDF5 dimension is:

train_data size:  (42000, 1, 28, 28)

train_labels size:  (42000, 1, 1, 1)

test_data size:  (28000, 1, 28, 28)

My HDF5 data layers are defined as:

layers {

  name: "mnist"

  type: HDF5_DATA

  top: "data"

  top: "label"

  hdf5_data_param {

  source: "sxyu/mnist/mnist_hdf5/data/mnist_train.txt"

        batch_size: 64

  }

  transform_param {

    scale: 0.00390625

  }

  include: { phase: TRAIN }

}

layers {

  name: "mnist"

  type: HDF5_DATA

  top: "data"

  top: "label"

  hdf5_data_param { 

       source: "sxyu/mnist/mnist_hdf5/data/test.txt" 

       batch_size: 100 }

  transform_param {

    scale: 0.00390625

  }

  include: { phase: TEST }

}

However, when the net starts initialising the check failed, and the error is:

F0929 14:42:15.652853 2076410640 io.cpp:133] Check failed: ndims <= max_dim (4 vs. 2)

Could anyone point me out what's wrong with my configuration? Should I feed data into caffe this way?

[Rayn Sakaguchi]

The label data dimensionality is the problem I think. It should be 2-D (or maybe 1-D). I think in my code, it's set as a row vector. 

[Martin Yu]

Hi Rayn,

I have read through the tutorial about blob, however, I think all blobs should be Number x Channels x Height x Width. How can I set the label data into 2-D ( or maybe 1-D ). I have stuck in this problem for a long time. Can you give me more hints? Or can you point me out some code/examples about personal data ( hdf5 ) input?

[Rayn Sakaguchi]

Hmm, the information I gave might've only been applicable to MATLAB HDF5 creation, but here is some example code that I use to put data into Caffe: trainData is a 4-D matrix where I take a 54x24 (#rows x cols) image patch, transpose it, and stack them into a 24x54x1xN matrix. my labels are 1xN row vectors in matlab. 

h5create(['train' num2str(cv) 'DT.h5'],'/data',[24 54 1 length(trainLabels)]);

h5create(['train' num2str(cv) 'DT.h5'],'/label',[1 length(trainLabels)]);

h5write(['train' num2str(cv) 'DT.h5'],'/data',trainData);

h5write(['train' num2str(cv) 'DT.h5'],'/label',trainLabels);

Regardless the error that you're getting implies that the dimensionality of something in your data is 4 where it should be 2. It's likely your label data. (Have you tried making it 42000x1?) Also your test data might need labels associated with it because in your TEST layer, you have a top:"label" specification. I'm not super familiar here with the nuts and bolts, but maybe give that a shot. (also to validate everything I did was correct I trained on the caffe mnist data, then tested on my created one to make sure I got everything oriented correctly)

[Martin Yu]

Hi Rayn,

THANKS VERY MUCH for you patient explanation!! First, according to your advice, I changed the dimensionality of train_label to be 42000x1, it worked; Then, I added test_labels as you suggested, which showed it was necessary for my experiment.

In addition to MATLAB HDF5 creation, I also tried Python HDF5 creation and it worked.

[Martin Yu]

Hi Rayn, 

Excuse me, one more question: 

I have trained my net with HDF5 training data and obtained caffemodel file. Now, I need to feed test data into the net and make predictions. As you suggested I associate test_labels which are all ones, however, the results are all accuracies instead of predictions. Do you have any idea about how to get these predictions? ( for example, I have 28000 rows of test data, therefore I should get 28000 predictions ). 

Thanks for your kind help.

[Rayn Sakaguchi]

You can create a deploy.prototxt and utilize the matlab/python wrapper. This is basically a modified version of the train prototxt. Particularly the data input and loss will need to be changed. There are imagenet examples of what they look like in both forms.

That's the way I have been doing it, there is probably a way to get caffe to output into a file natively, but I haven't tried looking for one.

[echoaimaomao]

Hi Martin Yu , I am sucking in the use of hdf5 data type these days. The results was satisfying when the image data was transformed into leveldb, but the hdf5 data did not work.
That is my train.h5 file , does something wrong with it ? Caffe can load the data and run normally ,but it did not converge at all. Thank you for your help.

 h5disp('train.h5')
HDF5 train.h5
Group '/'
    Dataset 'data'
        Size:  256x256x1x1715
        MaxSize:  256x256x1x1715
        Datatype:   H5T_IEEE_F64LE (double)
        ChunkSize:  []
        Filters:  none
        FillValue:  0.000000
    Dataset 'label'
        Size:  1x1715
        MaxSize:  1x1715
        Datatype:   H5T_IEEE_F64LE (double)
        ChunkSize:  []
        Filters:  none
        FillValue:  0.000000

[Dinesh]

I have confirmed that it is possible to export HDF5 files from Matlab (that look identical to your file above with h5disp), and it works. In fact, if you use Matlab's "low-level" HDF interface functions, you can get identical HDF5 files from Matlab/Python.

The one thing I had to be careful about when generating my HDF5 file was that HDF5_DATA layers do not seem to apply the transform_param to the data (even though they do not throw an error if passed transform_param in the definition). So you need to rescale data beforehand, and this makes a huge difference. I went from near-chance performance on MNIST to the usual >99% with this one change. 

[escorciav]

Hi Dinesh,

Which matlab functions are you using?

I remember that I got a transpose 2d-matrix in HDF5 files written by Python read by MATLAB

Thank you!!!

[Dinesh]

Hi Escorcia, 

Not sure if you're asking about exporting HDF5 from Matlab or importing. 

Exporting: Matlab is column-major and HDF row-major. This means if you use the "low-level" HDF interface (the H5S.create_simple > H5D.create > H5D.write pipeline), then you should use fliplr on your data size and label size when creating space for these. I got this pipeline working too, but the simpler method is to use h5create, then h5write. These "high-level" HDF interface functions automatically account for the row major-column major mismatch.  

To import HDF5 files into Matlab, use h5read(). 

[escorciav]

Thank you Dinesh. Maybe, I missed something in my experiment.

[Ding]

Hi, Rayn,

Thanks for your code for generating HDF5 files using Matlab. I have created HDF5 files (size: 1x512x1x9000)

But I get the following error when trying to open it with caffe.

Do you have the same experience?

Many thanks

I0307 17:59:21.575685  4338 hdf5_data_layer.cpp:66] Loading list of HDF5 filenames from: examples/FaceData.h5

I0307 18:01:10.133177  4338 hdf5_data_layer.cpp:80] Number of HDF5 files: 202358541

HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140690558573120:

  #000: ../../../src/H5F.c line 1586 in H5Fopen(): unable to open file

    major: File accessibilty

    minor: Unable to open file

在 2014年10月14日星期二 UTC+11上午2:41:24，Rayn Sakaguchi写道：

[Jimmy Ouyang]

This is problem of your file path.You need to change it to "./examples/FaceData.h5"

在 2015年3月7日星期六 UTC+8下午3:10:34，Ding写道：

[Jianyu Lin]

Hi Martin and Rayn,  

I am wondering how shall I set the chunck size when generating HDF5 file? Shall I set it according to the batch size or simply leave it blank?

Besides, In the /matla/example/demo.m file, the image's dimension is defined as "dim1xdim2xchannelxnum", but when being read by caffe, it follows this order: "numxchannelxdim2xdim1", is my observation correct

Could you please kindly help me with this questions? Many thanks.

[Xialei Liu]

Did you solve your problem with Hdf5, I faced the same problem, and I generate the hdf5 file using MATLAB like what you do. Thanks.

在 2015年1月7日星期三 UTC+1上午5:26:59，echoaimaomao写道：

[Xialei Liu]

I used the matlab to generate hdf5 as caffe's input, but the error is like this, Do you know what's the problem of this? Thinks.

Aborted at 1449492492 (unix time) try "date -d @1449492492" if you are using GNU date ***
PC: @     0x7f6597cdaee3 caffe::Blob<>::Reshape()
*** SIGFPE (@0x7f6597cdaee3) received by PID 19217 (TID 0x7f659862fa40) from PID 18446744071961423587; stack trace: ***
 @     0x7f6596270d40 (unknown)
    @     0x7f6597cdaee3 caffe::Blob<>::Reshape()
    @     0x7f6597cdb24c caffe::Blob<>::Blob()
    @     0x7f6597db1d07 caffe::InnerProductLayer<>::LayerSetUp()
    @     0x7f6597d22115 caffe::Net<>::Init()
    @     0x7f6597d231c5 caffe::Net<>::Net()
    @     0x7f6597cf3b3a caffe::Solver<>::InitTrainNet()
    @     0x7f6597cf4d7c caffe::Solver<>::Init()
    @     0x7f6597cf5089 caffe::Solver<>::Solver()
    @     0x7f6597d62413 caffe::Creator_AdaGradSolver<>()
    @           0x40eabe caffe::SolverRegistry<>::CreateSolver()
    @           0x407d0b train()
    @           0x405b61 main
    @     0x7f659625bec5 (unknown)
    @           0x40631d (unknown)
    @                0x0 (unknown)
Floating point exception (core dumped)

在 2015年1月13日星期二 UTC+1上午3:53:24，Dinesh写道：

[Jeremy Rutman]

I found that sending a 'bad' layer can do this, e.g. I sent an inner_product param {num_output:0}  which
crashed as below.
IMHO it would be useful to catch such attempts instead of allowing the mysterious crash, will attempt to code it if I find some nerves.

I0303 16:04:24.207450  4268 layer_factory.hpp:77] Creating layer data
I0303 16:04:24.208673  4268 net.cpp:106] Creating Layer data
I0303 16:04:24.208701  4268 net.cpp:411] data -> data
I0303 16:04:24.208735  4268 net.cpp:411] data -> label
I0303 16:04:24.208825  4269 db_lmdb.cpp:38] Opened lmdb plus_zero.train
I0303 16:04:24.208978  4268 data_layer.cpp:41] output data size: 32,3,100,100
I0303 16:04:24.214184  4268 net.cpp:150] Setting up data
I0303 16:04:24.214313  4268 net.cpp:157] Top shape: 32 3 100 100 (960000)
I0303 16:04:24.214332  4268 net.cpp:157] Top shape: 32 (32)
I0303 16:04:24.214342  4268 net.cpp:165] Memory required for data: 3840128
I0303 16:04:24.214360  4268 layer_factory.hpp:77] Creating layer conv1
I0303 16:04:24.214385  4268 net.cpp:106] Creating Layer conv1
I0303 16:04:24.214398  4268 net.cpp:454] conv1 <- data
I0303 16:04:24.214417  4268 net.cpp:411] conv1 -> conv1
I0303 16:04:24.215224  4268 net.cpp:150] Setting up conv1
I0303 16:04:24.215250  4268 net.cpp:157] Top shape: 32 50 96 96 (14745600)
I0303 16:04:24.215260  4268 net.cpp:165] Memory required for data: 62822528
I0303 16:04:24.215281  4268 layer_factory.hpp:77] Creating layer pool1
I0303 16:04:24.215299  4268 net.cpp:106] Creating Layer pool1
I0303 16:04:24.215311  4268 net.cpp:454] pool1 <- conv1
I0303 16:04:24.215323  4268 net.cpp:411] pool1 -> pool1
I0303 16:04:24.216086  4268 net.cpp:150] Setting up pool1
I0303 16:04:24.216109  4268 net.cpp:157] Top shape: 32 50 48 48 (3686400)
I0303 16:04:24.216120  4268 net.cpp:165] Memory required for data: 77568128
I0303 16:04:24.216130  4268 layer_factory.hpp:77] Creating layer conv2
I0303 16:04:24.216174  4268 net.cpp:106] Creating Layer conv2
I0303 16:04:24.216184  4268 net.cpp:454] conv2 <- pool1
I0303 16:04:24.216197  4268 net.cpp:411] conv2 -> conv2
I0303 16:04:24.216471  4268 net.cpp:150] Setting up conv2
I0303 16:04:24.216487  4268 net.cpp:157] Top shape: 32 20 44 44 (1239040)
I0303 16:04:24.216497  4268 net.cpp:165] Memory required for data: 82524288
I0303 16:04:24.216512  4268 layer_factory.hpp:77] Creating layer pool2
I0303 16:04:24.216526  4268 net.cpp:106] Creating Layer pool2
I0303 16:04:24.216536  4268 net.cpp:454] pool2 <- conv2
I0303 16:04:24.216548  4268 net.cpp:411] pool2 -> pool2
I0303 16:04:24.216564  4268 net.cpp:150] Setting up pool2
I0303 16:04:24.216577  4268 net.cpp:157] Top shape: 32 20 22 22 (309760)
I0303 16:04:24.216585  4268 net.cpp:165] Memory required for data: 83763328
I0303 16:04:24.216595  4268 layer_factory.hpp:77] Creating layer ip1
I0303 16:04:24.217203  4268 net.cpp:106] Creating Layer ip1
I0303 16:04:24.217218  4268 net.cpp:454] ip1 <- pool2
I0303 16:04:24.217231  4268 net.cpp:411] ip1 -> ip1
I0303 16:04:24.271767  4268 net.cpp:150] Setting up ip1
I0303 16:04:24.271828  4268 net.cpp:157] Top shape: 32 500 (16000)
I0303 16:04:24.271839  4268 net.cpp:165] Memory required for data: 83827328
I0303 16:04:24.271862  4268 layer_factory.hpp:77] Creating layer relu1
I0303 16:04:24.271879  4268 net.cpp:106] Creating Layer relu1
I0303 16:04:24.271889  4268 net.cpp:454] relu1 <- ip1
I0303 16:04:24.271904  4268 net.cpp:397] relu1 -> ip1 (in-place)
I0303 16:04:24.271934  4268 net.cpp:150] Setting up relu1
I0303 16:04:24.271949  4268 net.cpp:157] Top shape: 32 500 (16000)
I0303 16:04:24.271957  4268 net.cpp:165] Memory required for data: 83891328
I0303 16:04:24.271967  4268 layer_factory.hpp:77] Creating layer ip2
I0303 16:04:24.271982  4268 net.cpp:106] Creating Layer ip2
I0303 16:04:24.271993  4268 net.cpp:454] ip2 <- ip1
I0303 16:04:24.272006  4268 net.cpp:411] ip2 -> ip2
*** Aborted at 1457013864 (unix time) try "date -d @1457013864" if you are using GNU date ***
PC: @     0x7f6392bc2baa caffe::Blob<>::Reshape()
*** SIGFPE (@0x7f6392bc2baa) received by PID 4268 (TID 0x7f63931b4780) from PID 18446744071876389802; stack trace: ***
    @     0x7f6391d9ed40 (unknown)
    @     0x7f6392bc2baa caffe::Blob<>::Reshape()
    @     0x7f6392bc33ab caffe::Blob<>::Blob()
    @     0x7f6392cf6cf1 caffe::InnerProductLayer<>::LayerSetUp()
    @     0x7f6392be660c caffe::Net<>::Init()
    @     0x7f6392be7ef4 caffe::Net<>::Net()
    @     0x7f6392c1fce2 caffe::Solver<>::InitTrainNet()
    @     0x7f6392c20344 caffe::Solver<>::Init()
    @     0x7f6392c207d7 caffe::Solver<>::Solver()
    @     0x7f6392d19bc3 caffe::Creator_SGDSolver<>()
    @           0x40d6fe caffe::SolverRegistry<>::CreateSolver()
    @           0x409556 train()
    @           0x405bcb main
    @     0x7f6391d89ec5 (unknown)
    @           0x406195 (unknown)

[Jeremy Rutman]

I have spoken too soon it seems, after correcting my net generation I get the same error.

I generate a 'lenet' style architecture as below, which seems to work on other image dbs but not this one.  Incidentally will L.Accuracy be a valid layer?
I do not seem to find a python-specific layer creation spec.

def mynet(db, batch_size,n_classes=11  ):
    lr_mult1 = 1
    lr_mult2 = 2
    n=caffe.NetSpec()
    n.data,n.label=L.Data(batch_size=batch_size,backend=P.Data.LMDB,source=db,transform_param=dict(scale=1./255),ntop=2)
    n.conv1 = L.Convolution(n.data,param=[dict(lr_mult=lr_mult1),dict(lr_mult=lr_mult2)],
                            kernel_size=5,stride = 1, num_output=50,weight_filler=dict(type='xavier'))
   n.pool1 = L.Pooling(n.conv1, kernel_size=2, stride=2, pool=P.Pooling.MAX)
    n.conv2 = L.Convolution(n.pool1,param=[dict(lr_mult=lr_mult1),dict(lr_mult=lr_mult2)],
                            kernel_size=5,stride=1,num_output=20,weight_filler=dict(type='xavier'))
    n.pool2 = L.Pooling(n.conv2, kernel_size=2, stride=2, pool=P.Pooling.MAX)
    n.ip1 = L.InnerProduct(n.pool2,param=[dict(lr_mult=lr_mult1),dict(lr_mult=lr_mult2)],num_output=500,weight_filler=dict(type='xavier'))
    n.relu1 = L.ReLU(n.ip1, in_place=True)
    n.ip2 = L.InnerProduct(n.relu1,param=[dict(lr_mult=lr_mult1),dict(lr_mult=lr_mult2)],num_output=n_classes,weight_filler=dict(type='xavier'))
#    n.accuracy = L.Accuracy(n.ip2,n.label)
    n.loss = L.SoftmaxWithLoss(n.ip2,n.label)
    return n.to_proto()

[Jeremy Rutman]

ok i had introduced a problem into the lmdb. 
Still curious if I can use L.Accuracy(top,bottom) if anyone can weigh in , in meantime will try it and see what transpires.

[Jeremy Rutman]

 n.accuracy = L.Accuracy(n.ip2,n.label)

works as expected

On Thursday, March 3, 2016 at 5:16:25 PM UTC+2, Jeremy Rutman wrote:

ok i had introduced a problem into the lmdb. 
Still curious if I can use L.Accuracy(top,bottom) if anyone can weigh in , in meantime will try it and see what transpires.

On Thursday, March 3, 2016 at 4:59:45 PM UTC+2, Jeremy Rutman wrote:

I have spoken too soon it seems, after correcting my net generation I get the same error.

I generate a 'lenet' style architecture as below, which seems to work on other imag:e dbs but not this one.  Incidentally will L.Accuracy be a valid layer?

[Rakesh Kumar]

Hi JD,

 

Is it true for both training and Testing ?

I mean if I am providing my data in HDF5 format ,then caffe doesn't apply transfer parameter even if provided, both in training and testing case ?
Is it true if I generate my hdf5 file using matlab not using caffe ?