How to create hdf5 dataset for multiple output labels

[Niko Gamulin]

Hi,

I have been looking around for tools to create a hdf5 dataset for multiple output labels but haven't found any example.

I have checked the post Feed my own data into Caffe ( HDF5 ) and generate_sample_data.py script example but haven't figured out yet how to define the dataset size and store images with belonging labels. Below is an example list of images in training_dataset folder:

0_164.jpeg 1 1 1 -1 -1
0_0.jpeg 1 1 1 -1 -1
1_18.jpeg 1 1 1 -1 -1
1_0.jpeg 1 1 1 -1 -1
2_354.jpeg 1 1 1 -1 -1
2_0.jpeg 1 1 1 -1 -1



Is there any simple tool (similar to convert_imageset which stores the images from the specified folder along with belonging labels listed in train.txt/val.txt to lmdg or leveldb file) to generate hdf5 database from the images and belonging labels in listed in specified .txt file?

If not, I would be thankful if anyone suggested any other appropriate way to prepare the hdf5 dataset for caffe hdf5 input data layer.

Thanks,

Niko

[Charles Qi]

I think this post will be helpful:

Create Multiple labels for regression and classification tasks with HDF5 dataset and slice layer

best of luck

[Niko Gamulin]

Thanks for the answer Charles!

If anyone might be interested in it, I have created a matlab procedure to prepare the hdf5 dataset for images with multiple labels

[el]

Hello Niko! Thank you for sharing your code..! Could you provide me some more specific information about hdf5 files? Do they store only square images? Those square images are identical to "Crop" parameter in Datalayer? Please help me, I'd really appreciate it!

[Jimmy]

@Niko Gamulin
I have a similar problem. My problem is not write a hdf5 file but is how to write the testing result from caffe (I use command line) to hdf5 file. Do you have any suggestions? Thanks.

[Jan C Peters]

HDF5 can store any kind of data that can be represented as multidimensional arrays of numbers. If you want to know more about HDF5, consult its site or wikipedia. This group is for questions about caffe.

Jan

[Antonio Paes]

Hey guys, I'm using a similar code of Niko sharing for create hdf5 from RGB images, but when I start my train on caffe i receive this error:

I0423 11:37:02.411355 12011 net.cpp:454] deconv <- data_data_0_split_1

I0423 11:37:02.411363 12011 net.cpp:411] deconv -> deconv

F0423 11:37:02.412328 12011 filler.hpp:249] Check failed: blob->num_axes() == 4 (1 vs. 4) Blob must be 4 dim.

*** Check failure stack trace: ***

Someone have any idea of how to solve that?

Thanks

[Viktor]

Hi I want to do regression with caffe and want to use @Niko Gaulin's matlab code for generating a hdf5 dataset.

Do I have to do preprcessing with my images like changing RGB to BGR and changing the image Dimension from [Width Height Channels Num] to [N C H W].

And what about scaling the pixels from [0 255] to [0 1] and subtract the mean is it necessary? 

Thank you very much for your help

[Bensvage]

Have you figured it out?~