HDF5Data Processing with Caffe's Transformer for training

11 views
Skip to first unread message

dusa

unread,
Dec 15, 2017, 9:22:39 AM12/15/17
to Caffe Users

I am trying to load data to the network, since I need a custom data input (3 tops: 1 for data image, 2 for different labels) I load the data with HD5F files. It looks similar to this:

layer {
  name: "data"
  type: "HDF5Data"
  top: "img"
  top: "alabels"
  top: "blabels"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "path_to_caffe/examples/hdf5_classification/data/train.txt"
    batch_size: 64
  }
}

I want to preprocess the images using Caffe's own Transformer (for standard), how can I do this when I have to initialize the Transformer with data blob of a network model:

transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})

All the examples I have found first loads a pre-trained net and sets a transformer with its data blob/shape, but I couldn't find any for training. Thanks.

dusa

unread,
Dec 15, 2017, 9:33:43 AM12/15/17
to Caffe Users
To elaborate, the Class Transformer requires the Net as parameter: Class definition: Transform input for feeding into a Net. Parameters: net : a Net for which the input should be prepared 
And for preprocess, the parameters are: in_ : name of input blob to preprocess for, data : (H' x W' x K) ndarray and it returns: caffe_in : (K x H x W) ndarray for input to a Net    
It is possible to write own functions I suppose but it seems Transformer could work in batches and I would prefer to follow Caffe standard for this experiment.
Reply all
Reply to author
Forward
0 new messages