How does crop layer works

[Ark]

Hello

Background : I am working on the FCN training.

I would like to understand how crop layer works.

Below is my understanding. Please correct me if I am wrong.

crop layer takes 2 bottom blobs, say A and B. And size of A > size of B.

then it crops A w.r.t B such that new size of A = size of B

If this is correct, my question is, how does crop layer understands where to crop. In all the cases, cropping won't be symmetric, right? There can be more pixels cropped at left than at right.

Also, I see two parameters, axis and offset. Axis says in what direction to crop, but what does offset means. I didn't understand the explanation in caffe.proto.

Questions:

1 - How does caffe understand where to crop?
2 - What does the offset parameter used for?

[Mohamed Ezz]

You're right. The cropping layer takes a crop bottom and a reference bottom.

To quickly answer your 2 questions :

1 - How does caffe understand where to crop? Apparently it does not by default. You have to tell it by specifying the offset
2 - What does the offset parameter used for? It is used to tell the cropping layer where exactly to crop.

Let's take an example:

Blobs are 4D : (Batch size , Number of channels/filter , Height, Width) = (N,C,H,W)

1-crop bottom A is (20, 50, 512, 512)

2-reference bottom B is (20, 10, 256, 256)

3-the top blob C (result blob) will be (20, 10, 256, 256) 

In this example we want to crop dimensions 1,2 & 3. But keep dimension 0 fixed.

So we set axis=1 (that will crop 1 and all the following axes)

The other parameter 'offset' specifies where exactly should the crop take place in A. Apparently this parameter has no defaults and have to be specified.

There are 2 modes :

1- Specify 3 offsets, one for each dimension : say offset = (25, 128, 128)

• So axis=1, offset=(25,128,128)
• The crop operation in numpy syntax will be : C = A[: , 25: 25+B.shape[1] , 128: 128+B.shape[2] , 128: 128+B.shape[3] ]
• In words : this will only take filters of A 25 to 35 and ignore the rest. And do a center crop for the spatial dimensions

2- Specify 1 offset that applies for all dimensions : offset = 25

• This will be exactly like in (1) with offset = (25, 25, 25)

The prototxt for mode (1) is the following :

layer {

  name: "crop_layer"

  type: "Crop"

  bottom: "A"

  bottom: "B"

  top: "C"

  crop_param {
    axis: 1

    offset: 25

    offset: 128
    offset: 128

  }

}

[Ark]

Hello Mohamed Ezz,

Thanks for your detailed answer. It is really helpful.

In earlier versions of caffe (especially a branch from longjon for FCN) didn't had the axis and offset parameter for crop layer. That is why I asked this question. Yes, the new caffe version has this feature.

I see recent caffe provides a function to calculate the offset. But for the sake understanding, could you explain how this crop offset can be manually calculated?

Best Regards,
Ark

[Mohamed Ezz]

Yea I've been there too...moving from longjon's branch to the recent caffe with a new crop layer.

Basically if you know you want to do center cropping (99% of the cases) it's easy to calculate the offsets :

To centrally crop a list of size 512 to only the middle 256 values, the offset would be 128.

offset = (Original_length - desired length ) / 2

do this for each dimension. To be able to do that you need to know the exact dimensions of the blobs coming in and out for your layers. Caffe prints these dimensions when the network is constructed.

[Evan Shelhamer]

This is not generally right and prone to error. Instead, refer to the coordinate mapping logic in https://github.com/BVLC/caffe/blob/master/python/caffe/coord_map.py and in particular the shift and scale calculations in `coord_map()`: https://github.com/BVLC/caffe/blob/master/python/caffe/coord_map.py#L57-L79​

Evan Shelhamer

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/530256dc-b0bb-49f0-acc4-64fa7b85b654%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ark]

Thank you @shelhamer @Mohamed Ezz

I will mark it as completed

[邰磊]

Thanks, it really helpful.

在 2016年5月24日星期二 UTC+8下午6:20:23，Mohamed Ezz写道：

[Yang Ming]

A great answer. Thank you. Your answer should be added to the official document.

在 2016年5月24日星期二 UTC+8下午6:20:23，Mohamed Ezz写道：

You're right. The cropping layer takes a crop bottom and a reference bottom.

[Tao Liu]

This answer is very clear. So does this means the size of B should be NO larger than the size of A regarding corresponding axis?

[rud...@gmail.com]

why the first offset is 25 rather than 20? i didn't understand.

在 2016年5月24日星期二 UTC+8下午6:20:23，Mohamed Ezz写道：

You're right. The cropping layer takes a crop bottom and a reference bottom.

[Vincent Zhang]

I think offset=25 is just an example. You can specify any offset you want. After all, this is an operation along the channel dimension.

在 2017年5月8日星期一 UTC+8下午3:49:28，rud...@gmail.com写道：

[gattupall...@gmail.com]

Could someone lease explain in similar way how the old crop function works?

template <typename Dtype>
void CropLayer<Dtype>::LayerSetUp(const vector<Blob<Dtype>*>& bottom,
    const vector<Blob<Dtype>*>& top) {
  const CropParameter& param = this->layer_param_.crop_param();
  CHECK_EQ(bottom.size(), 2) << "Wrong number of bottom blobs.";
  CHECK_EQ(bottom[0]->num_axes(), 4) << "Only works with 4D blobs.";
  CHECK_EQ(bottom[1]->num_axes(), 4) << "Only works with 4D blobs.";
  crop_h_ = param.offset_height();
  crop_w_ = param.offset_width();
}

template <typename Dtype>
void CropLayer<Dtype>::Reshape(const vector<Blob<Dtype>*>& bottom,
    const vector<Blob<Dtype>*>& top) {
  // Check that the image we are cropping minus the margin is bigger than the
  // destination image.
  CHECK_GT(bottom[0]->height()-crop_h_, bottom[1]->height())
      << "invalid offset";
  CHECK_GT(bottom[0]->width()-crop_w_, bottom[1]->width()) << "invalid offset";
  top[0]->Reshape(bottom[0]->num(), bottom[0]->channels(), bottom[1]->height(),
      bottom[1]->width());
}

template <typename Dtype>
void CropLayer<Dtype>::Forward_cpu(const vector<Blob<Dtype>*>& bottom,
    const vector<Blob<Dtype>*>& top) {
  const Dtype* bottom_data = bottom[0]->cpu_data();
  Dtype* top_data = top[0]->mutable_cpu_data();
  for (int n = 0; n < top[0]->num(); ++n) {
    for (int c = 0; c < top[0]->channels(); ++c) {
      for (int h = 0; h < top[0]->height(); ++h) {
        caffe_copy(top[0]->width(),
            bottom_data + bottom[0]->offset(n, c, crop_h_ + h, crop_w_),
            top_data + top[0]->offset(n, c, h));
      }
    }
  }
}

template <typename Dtype>
void CropLayer<Dtype>::Backward_cpu(const vector<Blob<Dtype>*>& top,
    const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
  const Dtype* top_diff = top[0]->cpu_diff();
  Dtype* bottom_diff = bottom[0]->mutable_cpu_diff();
  if (propagate_down[0]) {
    caffe_set(bottom[0]->count(), static_cast<Dtype>(0), bottom_diff);
    for (int n = 0; n < top[0]->num(); ++n) {
      for (int c = 0; c < top[0]->channels(); ++c) {
        for (int h = 0; h < top[0]->height(); ++h) {
          caffe_copy(top[0]->width(),
              top_diff + top[0]->offset(n, c, h),
              bottom_diff + bottom[0]->offset(n, c, crop_h_ + h, crop_w_));

[Dharamendra Prajapati]

Could you please tell which file of caffe repo need to changes for setting off the crop? 

Thanks

-D

[ngc...@gmail.com]

Hi,

After consulting this thread and some others, this is how I cropped the input (for H and W only)
So this example is training a LENET on MNIST with images cropped to 24x22 at the offset specified instead of the original 28x28.
To get it running, I just set the dummy layer's first two dimensions to 1 and the data values to 0.

Please correct me if my usage is errorneous. I don't decipher source codes well and the documentation is quite terse.

name: "LeNet"
layer {
  name: "mnist"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    scale: 0.00390625
  }
  data_param {
    source: "examples/mnist/mnist_train_lmdb"
    batch_size: 64
    backend: LMDB
  }
}
layer {
  name: "mnist"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  transform_param {
    scale: 0.00390625
  }
  data_param {
    source: "examples/mnist/mnist_test_lmdb"
    batch_size: 100
    backend: LMDB
  }
}
layer {
  type: "DummyData"
  name: "dummy_for_crop"
  top: "dummy_for_crop"
  dummy_data_param {
    shape { dim: 1 dim: 1 dim: 24 dim: 22 }
    data_filler { type: "constant" value: 0 }
  }
}

layer {
  name: "cropped_data"
  type: "Crop"
  bottom: "data"
  bottom: "dummy_for_crop"
  top: "cropped_data"
  crop_param {
    axis: 2
    offset: 2
    offset: 3
  }
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "cropped_data"   #
  top: "conv1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 20
    kernel_size: 5
    stride: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
...