Data layer with different types for data and label?

[Alex]

Hi,

I want to write a layer that serves as an input for the following message from an LMDB database:

message TKSUnaryDepthDatum {

  // Image datum.

  optional Datum image_datum = 1;

  // Image location.

  optional string image_source = 2;

  // Superpixel label datum.

  optional Datum superpixel_label_datum = 3;

  // Superpixel depth.

  optional BlobProto superpixel_depth = 4;

}

It's just a message containing two Datum files - an original image and a superpixel label image that simply has a superpixel ID written at each pixel - and a BlobProto containing the mean depth of each superpixel.

I'd like to make my layer using the given layers as much as possible (to avoid redundancy etc.). My approach was to make a layer as such:

template <typename Dtype, typename Dtype2>
class TKSDepthDataLayer : public DataLayer<Dtype>, public InputLayer<Dtype2> {
     //...
}

The issue I have here though is that the image Datum is an 8 bit RGB, and the superpixel label Datum is 16 bit grayscale 
- making it impossible for me to use the DataLayer itself since it will set the same shape and type to both images.
I'm trying to figure out if there is any compromise I can use without having to create my own layer. 
Have I got a chance here, and if so how would I best go at it?

[Shai Bagon]

Before you write any c++/proto code, why don't you try one of these two alternatives:

1. Store your training data in HDF5 binary files and use EXISTING "HDF5Data" layer. See this and this stack overflow threads for more details on HDF5 Data layer.

2. Write a "Python" layer: a python layer enables you to prep the data "on the fly" for caffe. See this stackoverflow thread for more details.

Feel free to ask more questions on stackoverflow. Make sure you tag your question with "caffe" tag.

[Alex]

The python layer did the trick nicely. Thanks!