Input size, kernel size, and number of filters (batch size?)

[bhen...@gmail.com]

Hey Keras enthusiasts,

    I'm using a conv1d neural net to classify time based signals into one of two categories, RTS signals, and non-RTS signals. An RTS signal looks like this

where as a non-RTS signal is just white noise. The good news is that is works really well (97%!). The bad is that I'm not quite sure how it's pulling it off. I've been digging around trying to understand how the hyper-parameters are applied to conv1d layers and just want to make sure I've got this straight. Here's my code:

# -*- coding: utf-8 -*-

"""

Created on Mon Sep  4 00:32:17 2017

@author: Ben WORK ONLY

"""

from keras.models import Sequential

from keras.layers import Dense, Dropout,Activation

from keras.layers import Embedding

from keras.layers import Conv1D, GlobalAveragePooling1D, MaxPooling1D, Flatten, LSTM

import numpy as np

x_test = np.load('C:/Users/Ben WORK ONLY/PiCam/x_test.npy')

x_train = np.load('C:/Users/Ben WORK ONLY/PiCam/x_train.npy')

y_test = np.load('C:/Users/Ben WORK ONLY/PiCam/y_test.npy')

y_train = np.load('C:/Users/Ben WORK ONLY/PiCam/y_train.npy')

X_train = np.expand_dims(x_train, axis=2) 

X_test = np.expand_dims(x_test, axis=2) 

model = Sequential()

model.add(Conv1D(32, 12, input_shape=(1500, 1)))

model.add(Activation('relu'))

model.add(MaxPooling1D(3))

model.add(Conv1D(64, 12, activation='relu'))

model.add(MaxPooling1D(3))

model.add(Conv1D(128, 12, activation='relu'))

model.add(GlobalAveragePooling1D())

model.add(Dropout(0.5))

model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',

              optimizer='rmsprop',

              metrics=['accuracy'])

model.fit(X_train, y_train, batch_size=16, epochs=1)

score = model.evaluate(X_test, y_test, batch_size=16)

#model.save('C:/Users/Ben WORK ONLY/PiCam/CNNlin_model.h5')

Note that my first layer has 32 filters, a kernel size of 12, and an input shape of 1500,1. If the kernel is indeed the 'width of the flashlight' as I've heard/read in a few examples, and I only have 32 filters, does that mean that only a part of my input data is being considered? 12 * 32 = 384 samples compared to the input of 1500. I'm certain I've misunderstood something here. Does it have to do with batch_size? Anyone care to illuminate?

[Matias Valdenegro]

No, its not 12 * 32 = 384 samples. The kernel size (12 in your case) means that 12 consecutive elements are being considered and they produce a single value, but this "window" (flashlight as you call it) slides over your signal, so you get 1500 - 6 elements that constitute another signal, Since there are 32 kernels, you get a 32-dimensional signals of length 1494. This has nothing to do with batch size.

You can use model.summary() to examine what the mode is doing and what are the input and output shapes of each layer, maybe this will help you further.

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/04ff3358-d9a6-4fe8-b269-883449b8f1c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[bhen...@gmail.com]

Thanks for the reply! That clears things up a bit for me. I now understand that each kernel produces a single value, what I'm still struggling with is how these kernels are differentiated from one another. Let's say a particular kernel is pointing at elements 13-24, it produces a scalar value that is stored as the first of my 32-dimensional signals for that window. Now, the next kernel is looking at the same window to produce the second value. What is the difference between those two? Am I being clear?

Here's my summary:

Layer (type)                 Output Shape              Param #   

=================================================================

conv1d_71 (Conv1D)           (None, 1489, 32)          416       

_________________________________________________________________

max_pooling1d_45 (MaxPooling (None, 496, 32)           0         

_________________________________________________________________

conv1d_72 (Conv1D)           (None, 485, 64)           24640     

_________________________________________________________________

max_pooling1d_46 (MaxPooling (None, 161, 64)           0         

_________________________________________________________________

conv1d_73 (Conv1D)           (None, 150, 128)          98432     

_________________________________________________________________

global_average_pooling1d_22  (None, 128)               0         

_________________________________________________________________

dropout_22 (Dropout)         (None, 128)               0         

_________________________________________________________________

dense_22 (Dense)             (None, 1)                 129       

=================================================================

Total params: 123,617

Trainable params: 123,617

Non-trainable params: 0

_________________________________________________________________

Thanks again for the clarity!

[bhen...@gmail.com]

Is each value in my 32 dimensional vector the result of a convolution between the 12 input values, and a 12x1 kernel matrix? Then, the 12x1 kernel values are changed to produce the second value in the 32 dim vector? Is the change based on back propagation?

[Matias Valdenegro]

What you call "change" is done by backpropagation, but it all happens
simultaneously during training. The actual filter values are learned so they
produce meaningful features for your problem.

Also, the whole process of sliding window of the filter over the signal is
convolution.

[bhen...@gmail.com]

Thanks Matias!