Create Multiple labels for regression and classification tasks with HDF5 dataset and slice layer

[James Guo]

Since this question was asked a lot in this group and I think it's worthwhile to provide my personal experience to solve it. I'm currently using up to 20 different labels(including vectors and scalars) for the same data. The dataset format I used is HDF5. The advantage is in HDF5, you could use a vector as label, and the length of vector could be as long as you want. The idea here is you put all labels you want into this long vector, and slice it in .prototxt file. For example, you have an image data, and this data is corresponding to 3 different labels, say coordinates of left eye (x, y), age of people in the image (a) and gender of people in the image (g). The label in your HDF5 is simply (x, y, a, g)

Then in your .prototxt file, define slice layers as follows:

layers {

  name:   "image"

  type:     HDF5_DATA

  top:       "data"

  top:       "label"

  hdf5_data_param {

           source: path to your files

           batch_size: whatever you want

  }

  ....

  ....

}

layers {

  name: "slice0"

  type: SLICE

  bottom: "label"

  top: "left_eye_position"

  top: "label1"

  slice_param {

      slice_dim: 1

      slice_point: 2

  }

}

layers {

  name: "slice1"

  type: SLICE

  bottom: "label1"

  top: "age"

  top: "gender"

  slice_param {

      slice_dim: 1

      slice_point: 1

  }

}

Now you could do whatever you want in Caffe with these three labels

For detailed examples for HDF5 dataset and slice layer in Caffe, you could find them here

http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/hdf5_classification.ipynb

http://caffe.berkeleyvision.org/gathered/examples/siamese.html

Good luck!

[Pan.H.BestSonny]

What if I have one hundred label of one input.Slice would...   You have any idea?

在 2015年2月11日星期三 UTC+8下午7:15:20，James Guo写道：

[James Guo]

Yep. In the case of too many labels, the .prototxt file will become pretty ugly... No clue for the solution. But anyway, why do you need so many labels??

[Prabhu]

Slicing is a good idea for labels under 5. If the labels are like 100, it is not an ideal solution.

Caffe still i think can deal with 2D data in train phase. It can read vector label from HDF5 and calculate the euclidean loss. However i am not sure prediction of regression is possible with 2d labels.

i am struggling with predicting 30 labels for each input.

[James Guo]

If I remembered correctly, I've checked the source code of hdf5_data_layer, the layer initialisation restricts label dimension up to 2, which means we could only use 1D vector of labels since batch * label * 1 * 1. However, the good news is I just noticed that they eliminated such limitation in dev branch. So if you are using dev branch, it should be fine to use 2D labels. Then the problem goes to how to deal with 2D label using current loss layers. Feel free to provide any suggestions :)

[James Guo]

https://github.com/BVLC/caffe/blob/dev/src/caffe/layers/hdf5_data_layer.cpp#L35-L45

On Wednesday, 11 February 2015 16:53:18 UTC+1, Prabhu wrote:

[Karsten Flügge]

nice! thanks a lot for that ‘trick'!

it should become part of the official documentation!

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/fdb3332f-852c-4253-93fa-d373934b2dce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Beanfrog Green]

In caffe.proto, we can see the definition of slice_param:

message SliceParameter {
  optional uint32 slice_dim = 1 [default = 1];
  repeated uint32 slice_point = 2;
}

in common_layers.hpp

class SliceLayer : public Layer<Dtype> {
...
  virtual inline int ExactNumBottomBlobs() const { return 1; }
  virtual inline int MinTopBlobs() const { return 2; }
...

So we can use multiple slice_point to produce any number of top results, and the new layers can be defined as:

layers {

  name: "slice0"

  type: SLICE

  bottom: "label"

  top: "left_eye_position"

  top: "age"

  top: "gender"

  slice_param {

      slice_dim: 1

      slice_point: 1
      slice_point: 2

  }

}

在 2015年2月11日星期三 UTC+8下午7:15:20，James Guo写道：

Since this question was asked a lot in this group and I think it's worthwhile to provide my personal experience to solve it. I'm currently using up to 20 different labels(including vectors and scalars) for the same data. The dataset format I used is HDF5. The advantage is in HDF5, you could use a vector as label, and the length of vector could be as long as you want. The idea here is you put all labels you want into this long vector, and slice it in .prototxt file. For example, you have an image data, and this data is corresponding to 3 different labels, say coordinates of left eye (x, y), age of people in the image (a) and gender of people in the image (g). The label in your HDF5 is simply (x, y, a, g)

[James Guo]

Thanks, that's a good point and makes the solution more elegant. 

[Soda Heng]

James,

Have you figured out how to make predictions after training using HDF5 as input?

I have an HDF5 that has about 15,000 images that need to be predicted but can't get the predictor to predict it in batches due to memory constraints.

A tutorial on that would be awesome if you know how.

Thanks!

[James Guo]

Hello Soda,

Do you sure it's a memory problem? Because 15000 images seems not too many to me. For an alternative solution, you could separate the images into several HDF5 and put all these HDF5 path in the txt file, as Caffe's official instruction suggests. Good luck

[Wei Guo]

But how to convert imageset including labels to HDF5？
Thanks.

[Soda Heng]

Hi James,

It turned out that it was not a memory constraint. It had to do with my construction of the data. It was able to successfully predict now.

Another question i have is what is the max number of slices have you used? I'm trying to use 4 slice points to compare 5 outputs and getting Nan on the 5th loss when training after 3700 iterations. Have you been able to do the same or more slices?

Thanks,

Soda

[James Guo]

What I did is to convert raw images and labels into HDF5. There should be tons of materials you could find simply through Google, good luck :)

[James Guo]

I was using more than 20 slices. I think only the sky is the limit :-p

[Dean]

sorry ,i cannot understand what does it work.

  slice_param {

      slice_dim: 1

      slice_point: 1
      slice_point: 2

the label (x, y, a, g),coordinates of left eye (x, y), age of people in the image (a) and gender of people in the image (g).

how the   slice_point: 1  slice_point: 2 let the label to ->(x,y),(a),(g)?? how do the two slice_point slice the  label, slice the first part of the lable tp -> the first top: "left_eye_position".and the sec part of the label-> sec top: age .and the third part of the label -> the third top:gender??? 

在 2015年3月13日星期五 UTC+8下午5:12:57，Beanfrog Green写道：

[Dean]

sorry ,i cannot understand what does it work.

  slice_param {

      slice_dim: 1

      slice_point: 1
      slice_point: 2

the label (x, y, a, g),coordinates of left eye (x, y), age of people in the image (a) and gender of people in the image (g).

how the   slice_point: 1  slice_point: 2 let the label to ->(x,y),(a),(g)?? how do the two slice_point slice the  label, slice the first part of the lable tp -> the first top: "left_eye_position".and the sec part of the label-> sec top: age .and the third part of the label -> the third top:gender    ??? 

在 2015年2月11日星期三 UTC+8下午7:15:20，James Guo写道：

Since this question was asked a lot in this group and I think it's worthwhile to provide my personal experience to solve it. I'm currently using up to 20 different labels(including vectors and scalars) for the same data. The dataset format I used is HDF5. The advantage is in HDF5, you could use a vector as label, and the length of vector could be as long as you want. The idea here is you put all labels you want into this long vector, and slice it in .prototxt file. For example, you have an image data, and this data is corresponding to 3 different labels, say coordinates of left eye (x, y), age of people in the image (a) and gender of people in the image (g). The label in your HDF5 is simply (x, y, a, g)

[zzz]

Hi James,

If the data is too big to put into one hdf5 file. When I separate it into several files, should the number of image in one file be the same as batch size? Or caffe is aware of that?

Thanks in advance.

Zizhao

[Ronghang Hu]

>Should the number of image in one file be the same as batch size?

No, it doesn't matter.

[rsl]

Would you please be generous and share you experience with me? I have vector of labels for each image which represents the feature vector. I'd like to create a hdf5 file but I don't know how the table should look like. In lmdb files the first column is image name and the second in the single label. but what about multilabel hdf5 file which does not accept the image name as string? Thanks

[rsl]

I would like to know how do you save your 4D numpy array for images and 2D numpy array for labels. I know I have to create a 4d numpy array for data  like(N,C,H,W) a d 2d numpy array for labels like (N,L) which N is the image and L is the vector of labels. I wanna know how do you restore these data. Thanks a lot.

[rsl]

Thank you so much for your help. it works.

But the image names appears in the hdf5 file as 0.0 . I know hdf5 can not show string but how to specify the image there? is there any way to save string in hdf5?