FCN8s score_fr layer always zero

[Johannes Kolbe]

Hi,

I got the FCN AlexNet semantic segmentation working quite well and even trained it with the cityscapes dataset (cityscapes-dataset.com) with some good results. But now I wanted to go a step further and use the FCN-8s based on VGG-16 as given in this thread: https://groups.google.com/forum/#!topic/digits-users/0UF-xD-yGuY and using the pre-trained model from http://dl.caffe.berkeleyvision.org/fcn8s-heavy-pascal.caffemodel

The problem is, the score_fr layer always gives zero as an output and I really can't figure out where I went wrong. I attached my network and solver. Would be great if somebody can give me a hint for what might go wrong.

[Greg Heinrich]

Hello, the weights of your score_fr35 layer are all zeroes. You may want to initialize them with a "Xavier" weight filler. Also, the weights for your deconv layer are probably zeroes too, consider using a bilinear interpolation weight filler. More details on https://devblogs.nvidia.com/parallelforall/image-segmentation-using-digits-5/

--
You received this message because you are subscribed to the Google Groups "DIGITS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digits-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/digits-users/0c0fcfe9-6862-469b-83b1-9811beb7396b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Johannes Kolbe]

Hey Greg,

thanks for the quick answer. The weight initialization indeed solved the problem of the score layer being zero, but the curves for loss and accuracy still don't change a bit. It doesn't look quite as it should be, as I guess there should be at least some movement in them. But I am still quite new to all the deep learning stuff, so I might be wrong there and it might be something VGG-16 specific. Do you have any idea where this behaviour comes from?

Am Freitag, 6. Januar 2017 13:39:37 UTC+1 schrieb Greg Heinrich:

Hello, the weights of your score_fr35 layer are all zeroes. You may want to initialize them with a "Xavier" weight filler. Also, the weights for your deconv layer are probably zeroes too, consider using a bilinear interpolation weight filler. More details on https://devblogs.nvidia.com/parallelforall/image-segmentation-using-digits-5/

On Fri, Jan 6, 2017 at 12:56 PM, Johannes Kolbe <kolbe.j...@gmail.com> wrote:

Hi,

I got the FCN AlexNet semantic segmentation working quite well and even trained it with the cityscapes dataset (cityscapes-dataset.com) with some good results. But now I wanted to go a step further and use the FCN-8s based on VGG-16 as given in this thread: https://groups.google.com/forum/#!topic/digits-users/0UF-xD-yGuY and using the pre-trained model from http://dl.caffe.berkeleyvision.org/fcn8s-heavy-pascal.caffemodel

The problem is, the score_fr layer always gives zero as an output and I really can't figure out where I went wrong. I attached my network and solver. Would be great if somebody can give me a hint for what might go wrong.

--
You received this message because you are subscribed to the Google Groups "DIGITS Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to digits-users...@googlegroups.com.

[Greg Heinrich]

Did you also verify the weights of the deconv layers?

To unsubscribe from this group and stop receiving emails from it, send an email to digits-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/digits-users/596e1297-d630-4820-a6a3-1a96422caa0a%40googlegroups.com.

[Johannes Kolbe]

Well, I did initialize them with the weight_filler:{type:"bilinear"} So they are:
layer {
  name: "upscore_pool4_35"
  type: "Deconvolution"
  bottom: "fuse_pool4_35"
  top: "upscore_pool4_35"
  param {
    lr_mult: 0.0
  }
  convolution_param {
    num_output: 35
    bias_term: false
    kernel_size: 4
    stride: 2
    weight_filler {
      type: "bilinear"
    }
  }
}

and

layer {
  name: "upscore8_35"
  type: "Deconvolution"
  bottom: "fuse_pool3_35"
  top: "upscore8_35"
  param {
    lr_mult: 0.0
  }
  convolution_param {
    num_output: 35
    bias_term: false
    kernel_size: 16
    stride: 8
    weight_filler {
      type: "bilinear"
    }
  }
}

[Greg Heinrich]

I think you forgot the "group" parameter (see https://github.com/NVIDIA/DIGITS/blob/digits-5.0/examples/semantic-segmentation/fcn_alexnet.prototxt#L295-L302). Each deconv output feature map should only see the corresponding input feature map so you'll want to set the group field to 35. You can also check the visualization of the deconv filters, they should look similar to those of FCN-Alexnet (though they would be 1/2 and 1/4 the size).

[Johannes Kolbe]

Great, thanks. Seems like adding the group field did the trick. Thank you very much.