FCN training data augmentation

[Joe]

I am finetuning a FCN with my own dataset using LMDB as my backend and  my model works though I need to improve the performance. I would like experiment with data augmentation even though on the Fully Convolutional Networks for Semantic Segmentation paper it's noted it yielded no marked improvement.  In particular, I would like to try scaling, rotation, color, mirroring, cropping, vignetting, augmentations. My question is, do I have to perform the augmentations on both the data and the label layers?

[Przemek D]

When performing morphological augmentations like elastic deformations, rotation, scaling etc. (any that change the shape of the objects) you want your labels to match the transformed images. Be cautious however: labels are integer images with intensity of each pixel coding the class it belongs to - so if your augmentation scheme changes the intensity of the pixels (e.g. color augmentations or interpolation during shape transformations), you must take care not to accidentally alter the pixel labels.

[Joe]

Thank you for the reply. I have performed the geometric augmentations on both the images and labels. For the pixel value distortions, I have performed on the images only. All these were done prior to creating the LMDB databases. I was looking for a way to perform the augmentations on the fly by defining them in the prototxt files. For now, this works and I can see improvements but I would like to know whether it's possible to perform the augmentations while training.

[Przemek D]

DataLayer has a built in Transformation module which - to some extent - you can use to perform very primitive augmentations (crop and mirror). For more options you would have to some up with your own scheme. Perhaps through a Python layer?