how to feed the image data to HDF5 on caffe?

[helxsz]

Hi ,everyone:

I had hard time working on caffe with HDF5 on the image classification and regression tasks, for some reason, the training on HDF5 will always fail at the first beginning that the test and train loss could very soon drop to close zero. after trying all the tricks such reducing the learning rate, adding RELU, dropout, nothing started to work, so I started to doubt that the HDF5 data I am feeding to caffe  is wrong.   

so currently I am working on the universal dataset (Oxford 102 category flower dataset and also it has public code ), firstly I started out by trying ImageData and LMDB layer for the classification, they all worked very well.  at last i used HDF5 data layer for the tuning, the training_prototxt doesn't change unless on the data layer which uses HDF5 instead. and again, at the start of the learning, the loss drops from 5 to 0.14 at iteration 60, 0.00146 at iteration 100, that seems to prove that HDF5 data is incorrect.

i have two image&label to HDF5 snippet on the github, all of them has been tested to be not working with caffe, I wonder anything wrong with this data, can anybody give me some advice? or if you have some HDF5 examples for classification or regression, which can be helpful to me a lot.

Thanks a lot

one snippet is shown as 

def generateHDF5FromText2(label_num):

print '\nplease wait...'

HDF5_FILE = ['hdf5_train.h5', 'hdf5_test1.h5']

        #store the training and testing data path and labels

  LIST_FILE = ['train.txt','test.txt']

for kk, list_file in enumerate(LIST_FILE):

#reading the training.txt or testing.txt to extract the all the image path and labels, store into the array

path_list = []

label_list = []

with open(list_file, buffering=1) as hosts_file:

for line in hosts_file:

line = line.rstrip()

array = line.split(' ')

lab = int(array[1])

label_list.append(lab)

path_list.append(array[0])

print len(path_list), len(label_list)

    

                # init the temp data and labels storage for HDF5

datas = np.zeros((len(path_list),3,227,227),dtype='f4') 

labels = np.zeros((len(path_list), 1),dtype="f4")

for ii, _file in enumerate(path_list):

                    # feed the image and label data to the TEMP data

   img = caffe.io.load_image( _file )

   img = caffe.io.resize( img, (227, 227, 3) ) # resize to fixed size

   img = np.transpose( img , (2,0,1))

   datas[ii] = img

   labels[ii] = int(label_list[ii])

    # store the temp data and label into the HDF5

with h5py.File("/data2/"+HDF5_FILE[kk], 'w') as f:

f['data'] = datas

f['label'] = labels

f.close()

[chidai...@gmail.com]

Hai Elxz,

Iam also facing this issue now after 1 year, kindly reply if you have any solution for this problem.

Reagrds
K S Kumar