Example in Keras docs for the 2D convolution

[Lin Terry]

Hi all,

In the docs of convolution2D in https://keras.io/layers/convolutional/#convolution2d, shown below,

# apply a 3x3 convolution with 64 output filters on a 256x256 image: model = Sequential() model.add(Convolution2D(64, 3, 3, border_mode='same', input_shape=(3, 256, 256))) # now model.output_shape == (None, 64, 256, 256)

When my input data is (None, 3, 256, 256) and nb_filters, nb_row, nb_col = 64, 3, 3, why the output shape is (None, 64, 256, 256)? Can someone guide me and go to details of this convolution operation?

1.

By the comment  "# apply a 3x3 convolution with 64 output filters on a 256x256 image "

This function will apply 3x3 convolution to RGB 3 plans of 256x256 images, so why the shape is not (none, 64*3, 256, 256)?

2.

In this function, it seems we can assign any value of nb_filter. (Number of convolution kernels to use (dimensionality of the output)), in Keras, how can I get the values of these 64 kernels?

3. 

In this docs, actually it said the nb_filter will be the second value in output_shape. Can I know the details of this convolution? How the input data(None, 3, 256, 246) becomes output data (None, 64, 256, 256), what direction/axis is this convolution along? Is it as my guess that it's along the 256x256 plan?

Sincerely,

Terry 

[mhubri...@gmail.com]

Maybe I'm wrong, but it seems to me that you don't know a lot of basics yet (also because of your other post). That's totally fine! Everyone has to start at some point. But maybe it would be better if you try to read and work through some tutorials. For example here you can find a very good lecture script about CNNs: http://cs231n.github.io/convolutional-networks/. They also cover your questions about the dimensions etc.

[Lin Terry]

Hi Mhubri,

Cant't say you are wrong but I just want to double confirm the details of convolution2d in Keras and make sure my understanding is correct. It's very appreciated if you can provide the exact answer to me. Going back to read the lecture script didn't hep much since they are not the same example. Please note that my purpose is to get the exact parameters, coefficients of these 2D filters(maybe they are 3D?) but not just use it.

1. For current example, the input shape is (3, 256, 256) and the output shape is (64, 256, 256), it looks like the convolution filter actually is 3x3(spatially)x3(rgb) and there are 64 different convolution filters. Is this correct?

2. How does Keras define these 64 sets of 3x3 filters? Can I get the coefficients / weightings of these filters? (64 sets)

I am looking forward to your replying.

Sincerely,

Terry 

mhubri...@gmail.com於 2016年11月1日星期二 UTC+8上午1時33分33秒寫道：

[mhubri...@gmail.com]

Now you're on the right way!

As you said, we have 64 different filters which are 3x3(spatially)x3(rgb) each.

You can access the weights of these filters by calling model.get_weights() or model.layers[i].get_weights(), where i is the number of a specific layer.

[seggs...@gmail.com]

On Monday, October 31, 2016 at 5:30:20 PM UTC+1, Lin Terry wrote:

   1. When you use a 3 by 3 filter is considered only on the 2D spatial dimension of the input not the 3D input volume images(3 input channel is left aside). So the 3 X3 filter convolve through the 256 X256 input image. So 64 is number of filter used, the output is a stacked image of 64 X 256 X256.

   2. common settings of number of filters to be used are 32, 64, 128 under some calculations. This numbers are widely used and successful.

  3. I think this question is similar to no 1 question.

     regards,
     Segun

[Lin Terry]

Hi Sgan,

   1. When you use a 3 by 3 filter is considered only on the 2D spatial dimension of the input not the 3D input volume images(3 input channel is left aside). So the 3 X3 filter convolve through the 256 X256 input image. So 64 is number of filter used, the output is a stacked image of 64 X 256 X256.

>> Can we go details to this description? What do you mean 3 input channel is left aside? If the output shape is 64x256x265, then my thinking is, the input is 3x256x256, so the there will be only 64 filters of 3x3x3 and then apply to 256x256x3 volume. Is this correct? 

Although the input is 3x256x256 but they are the pixels in the same frame of on a time axis, so we don't call it 3D convolution but just call it 2D spatial convolution filtering. Is this what you mean?

Or actually there are only 64 sets of 3x3 filters?

Thanks for your kind guidance.

Sincerely,

Terry

seggs...@gmail.com於 2016年11月1日星期二 UTC+8下午8時11分07秒寫道：

[seggs...@gmail.com]

Terry,

By the way, My name is Segun. 3(RGB)X 256wide X 256high image is 2D, video is 3D. There are 64 filters of 3 X 3 sliding independently on the input image. The output will be after computation(dot product) 64 x (256) x (256), I bracket the 256 because it can be some other size maybe 64 X 224 X 224. Please read papers on Deep learning to understand more.

[Lin Terry]

Thanks Segun. And sorry for the typo of your name. I will check out the filter coefficients and then I can understand more.

Sincerely,

Terry

seggs...@gmail.com於 2016年11月2日星期三 UTC+8下午10時21分08秒寫道：

[Lin Terry]

Also thanks Mhubri.

Sincerely,

Terry

mhubri...@gmail.com於 2016年11月1日星期二 UTC+8下午4時15分31秒寫道：

[Lin Terry]

Hi both,

After check out the weight from model. For example, my model is
model = Sequential()
model.add(Convolution2D(1, 3, 3, border_mode='same', dim_ordering='tf', input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(Convolution2D(2, 3, 3, border_mode='same'))
model.add(Activation('relu'))

So doing a 3x3 convolution with 1 set of filter.
And then call to show the parameters,

input shape: (32, 32, 3)
output shape: (None, 32, 32, 1)

model layer 0: [array([[[[-0.44335088],
         [-0.20550063],
         [-0.54055387]],

        [[ 0.45638397],
         [ 0.18023646],
         [ 0.27544647]],

        [[-0.52478981],
         [-0.3563363 ],
         [-0.38794193]]],


       [[[-0.35504171],
         [ 0.22461028],
         [-0.05062532]],

        [[-0.19462331],
         [-0.0770151 ],
         [ 0.56060874]],

        [[-0.35007137],
         [-0.36605915],
         [-0.48056158]]],


       [[[ 0.16772677],
         [-0.12315023],
         [ 0.09024198]],

        [[ 0.47404319],
         [-0.33782277],
         [-0.56497747]],

        [[-0.0320316 ],
         [ 0.34991249],
         [ 0.22081527]]]], dtype=float32), array([ 0.], dtype=float32)]

So it shows a 3x3x3(rgb) filter and a constant offset 0. For this one, it's consistent to my understanding.

And then layer 1 is aviation.
Then layer 2, doing a 3x3 convolution with 2 set of filters, and it shows,

model layer 2: [array([[[[ 0.63717717,  0.62629467]],

        [[ 0.2503061 , -0.33820277]],

        [[-0.56018806, -0.26080328]]],


       [[[-0.24903327,  0.08353958]],

        [[ 0.00375767,  0.25935245]],

        [[ 0.34627977,  0.23111989]]],


       [[[-0.21744239,  0.59195977]],

        [[ 0.01376275, -0.70028448]],

        [[ 0.14046614,  0.1176499 ]]]], dtype=float32), array([ 0.,  0.], dtype=float32)]

Question 1:
it's a 3x3x2 filter, can you guide me how to translate this? They should be 2 separated 3x3x1 filters. (input shape is 32x32x1)

Question 2:
How does Keras decide these coefficient of convolution filters for each layer with different number of sets?

Sincerely,
Terry


Lin Terry於 2016年11月2日星期三 UTC+8下午10時46分25秒寫道：