creating HDF5 from RGB images using Matlab. Should I swap channels RGB -> BGR?

[Ehsan]

Hey guys,

Could you please tell me if the way I am generating HDF5 from images is correct or not? I have written the following script based on the demo.m from the hdf5creation directory. What I am not 100% sure about is the lines between asterisks. After reading an image I down scale the pixel values by 255 so that they fall between 0 and 1. Then, I change the order of channels from RGB to BGR and finally switch the order of rows and columns before passing to the "store2hdf5" function. I am not subtracting the mean like we do in LMDB format because the data has to fall between 0 and 1. Am I correct?

Thanks

%------------------------------------------------------

filename='trial.h5';

num_total_samples=3000;

chunksz=500;

created_flag=false;

totalct=0;

for batchno=1:ceil(num_total_samples/chunksz),

  fprintf('batch no. %d\n', batchno);

  last_read=(batchno-1)*chunksz;  

  left_num_samples = num_total_samples - last_read;    

  Adap_chunksz = min(left_num_samples,chunksz);

  batchdata = zeros(IMAGE_DIM,IMAGE_DIM,3,Adap_chunksz);

  batchlabs = zeros(1,Adap_chunksz);

  for t=1:Adap_chunksz,

      n = (batchno-1)*chunksz + t;

      full_image_address = [patches_root_address Filename_Stack_Cell{n}];

      im = imread(full_image_address);

      im = imresize(im, [IMAGE_DIM IMAGE_DIM], 'bilinear'); 

     %****************************************

      im = single(im./255);      

      im = im(:,:,[3 2 1]); % RGB -> BGR

      im = permute(im, [2 1 3]); % switch row and column to match with caffe

     %*****************************************

      batchdata(:,:,:,t) = im;

      batchlabs(t) = Label_Vec(n);

  end

  % store to hdf5

  startloc=struct('dat',[1,1,1,totalct+1], 'lab', [1,totalct+1]);

  curr_dat_sz=store2hdf5(filename, batchdata, batchlabs, ~created_flag, startloc, chunksz); 

  created_flag=true;% flag set so that file is created only once

  totalct=curr_dat_sz(end);% updated dataset size (#samples)  

  disp(['Number of Images Left to Store (' SetType ' Mode): ' num2str(left_num_samples)]);

end

[Dinesh]

Hi Ehsan, the preprocessing steps you use are entirely up to you, as long as you're not re-using some pretrained network. Caffenet uses just mean-subtraction to normalize, while some researchers often use mean-subtraction followed by stdev division etc. In general, mean subtraction is a good idea, for standard random weight initialization schemes.

If you *are* using a pretrained network, your preprocessing should match exactly with the preprocessing of the data used in pretraining. This could mean channel flipping, normalization (with the same mean and/or stdev as the pretraining data) etc. 

For instance, Caffenet's preprocessing stages are laid out in this example: http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/00-classification.ipynb (see "transformer" specifications). 

[Ehsan]

Thank you very much Dinesh for your explanation. According to what you said I think the following lines are correct if I would like to use say "VGG Net" as initial weights for fine-tuning:

     %******************************************

      im = single(im - MeanImage);      

      im = im(:,:,[3 2 1]); % RGB -> BGR

      im = permute(im, [2 1 3]); % switch row and column to match with caffe

     %******************************************

So, there will be no need to divide by 255 and mean-subtraction is what should be done instead.

Best,

Ehsan

[helxsz]

Hi, Ehsan, have you tested the difference of divide by 255 and mean-subtraction, does HDF5 layer support mean subtraction?   

[Antonio Paes]

Hi Ehsan, 

you have some similar problem like this:

I0420 09:23:54.503398 65817 net.cpp:454] deconv <- data_data_0_split_1

I0420 09:23:54.503407 65817 net.cpp:411] deconv -> deconv

F0420 09:23:54.504436 65817 filler.hpp:249] Check failed: blob->num_axes() == 4 (1 vs. 4) Blob must be 4 dim.

Thanks

[Jan]

For the record: HDF5Data layer loads the data from HDF5 into the blobs of the network as-is, no transformation such as mean subtraction is performed.

Jan