Conv2D Input shape Error

[Brent Lippert]

I'm trying to use Keras w/TensorFlow (Python3) backend to build a Convolutional NN for NLP classification. The issue I'm having is when in the defining the shape of my input when building my network. I'm correctly following their API (not my first time using Keras) yet I am getting a variety of inexplicable errors.

This is my Network configuration:

     convoNet = Sequential()

     convoNet.add(Conv2D(10, (29, 3), input_shape=(None, maxRow, 29, 1)))

     convoNet.add(MaxPooling2D(pool_size=(2, 2)))

     convoNet.add(Dropout(.5))

     convoNet.add(Dense(2, activation='softmax’))

     

     convoNet.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])

     convoNet.fit(trainingSet[0], trainLabels, nb_epoch=20, batch_size=10)

     scores = convoNet.evaluate(testingSet[0], testLabels)

     print("\n", scores)

And I get this Error:

     Traceback (most recent call last):

       File "/Users/bl755p/Documents/WRT_NLP.py", line 646, in <module>

         convoNet.add(Conv2D(10, (29, 3), input_shape=(None, maxRow, 29, 1)))

     TypeError: __init__() missing 1 required positional argument: ‘nb_col'

And I’ve also tried this (Only thing that changes is Conv2D so just gonna show you that):

     convoNet.add(Conv2D(10, (29, 3), nb_row=maxRow, nb_col=29))

And got this error:

     Traceback (most recent call last):

 

     File "/Users/bl755p/Documents/WRT_NLP.py", line 646, in <module>

  

       convoNet.add(Conv2D(10, (29, 3), nb_row=maxRow, nb_col=29))

     TypeError: __init__() got multiple values for argument 'nb_row'

And also tried:

     convoNet.add(Conv2D(10, (29, 3), 1, input_shape=(maxRow, 29, 1)))

With this error:

     Traceback (most recent call last):

       File "/Users/bl755p/Documents/WRT_NLP.py", line 645, in <module>

         convoNet.add(Conv2D(10, (29, 3), 1, input_shape=(maxRow, 29, 1)))

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/models.py", line 299, in add

       layer.create_input_layer(batch_input_shape, input_dtype)

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/engine/topology.py", line 401, in create_input_layer

    self(x)

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/engine/topology.py", line 546, in __call__

      self.build(input_shapes[0])

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/layers/convolutional.py", line 436, in build

 

       constraint=self.W_constraint)

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/engine/topology.py", line 418, in add_weight

         weight = initializer(shape, name=name)

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/initializations.py", line 65, in glorot_uniform

 

       s = np.sqrt(6. / (fan_in + fan_out))

     TypeError: unsupported operand type(s) for /: 'float' and ‘tuple'

And tried this:

     convoNet.add(Conv2D(10, (29, 3), (1,1), input_shape=(1, maxRow, 29)))

With this error:

     Traceback (most recent call last):

       File "/Users/bl755p/Documents/WRT_NLP.py", line 645, in <module>

 

       convoNet.add(Conv2D(10, (29, 3), (1,1), input_shape=(1, maxRow, 29)))

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/models.py", line 299, in add

 

       layer.create_input_layer(batch_input_shape, input_dtype)

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/engine/topology.py", line 401, in create_input_layer

 

       self(x)

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/engine/topology.py", line 546, in __call__

         self.build(input_shapes[0])

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/layers/convolutional.py", line 436, in build

    constraint=self.W_constraint)

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/engine/topology.py", line 418, in add_weight

 

       weight = initializer(shape, name=name)

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/initializations.py", line 66, in glorot_uniform

 

       return uniform(shape, s, name=name)

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/initializations.py", line 33, in uniform

 

       return K.random_uniform_variable(shape, -scale, scale, name=name)

 

     File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 616, in random_uniform_variable

 

       shape = tuple(map(int, shape))

     TypeError: int() argument must be a string, a bytes-like object or a number, not ‘tuple'

Tried this:

     convoNet = Sequential()

     convoNet.add(Input(shape=(None, maxRow, 29, 1)))

     convoNet.add(Conv2D(10, (29, 3)))

     convoNet.add(MaxPooling2D(pool_size=(2, 2)))

     convoNet.add(Dropout(.5))

     convoNet.add(Dense(2, activation='softmax'))

Error:

     Traceback (most recent call last):

       File "/Users/bl755p/Documents/WRT_NLP.py", line 645, in <module>

         convoNet.add(Input(shape=(None, maxRow, 29, 1)))

       File "/Users/bl755p/anaconda/envs/ATT_NLP-TensorFlow/lib/python3.5/site-packages/keras/models.py", line 284, in add

         'Found: ' + str(layer))

     TypeError: The added layer must be an instance of class Layer. Found: Tensor("input_1:0", shape=(?, ?, 23, 29, 1), dtype=float32)

I've also tried the other way you can configure your net:

     convoNet = Sequential([

    Conv2D(10, (29, 3), input_shape=(None, maxRow, 29, 1)),

    MaxPooling2D(pool_size=(2, 2)),

    Dropout(.5),

    Dense(2, activation='softmax')

])

With the same results.

These are among the ***MULTITUDE*** of different things I’ve tried to get this to compile/run correctly over the last 2-3 hours, and nothing in their API helps since I seem to be doing everything they say to do nor does anything I can find online so I am at a loss. What am I missing here?

[Daπid]

input_shape shouldn't include the batch dimension, so  for 2D inputs in channels_last mode, you should use input_shape=(maxRow, 29, 1). The layer Input is only for use in the functional API, not the Sequential.

On 31 March 2017 at 20:45, Brent Lippert <brent...@gmail.com> wrote:

     convoNet.add(Conv2D(10, (29, 3), input_shape=(None, maxRow, 29, 1)))

Are you sure you don't want to use a Conv1D there?

Conv1D(10, 3, input_shape=(maxRow, 29))

[Brent Lippert]

Ya the batch size I was pretty sure didn't need to be in there that's why in most of the examples I posted it isn't but thanks for confirming that. Then the Input layer I didn't think that was needed either but tried it for the sake of trying everything. 

As for using Convo1D, would that work? Each singular instance is a 2D matrix with maxRow rows (23 I think it was calculated to be) and 29 columns, wherein each row represents a single word with each column representing one of the words in my vocab so each row will be mostly 0s with a single 1 in the column of the word that row corresponds to. And there's only 1 channel being the word2vec embedding (though I may add other embeddings as other channels later). Conv1D would only be if each of my instances was a 1D vector right? Sorry I wasn't clear on what my instance dimensions were earlier, but I'm fairly certain I need Conv2D do I not?

[Robert McGwier]

You know things are bad when the examples in the github repository won't run on the latest version.

I declare this rock flew badly and went thud.

Bob

On Friday, March 31, 2017 at 2:45:40 PM UTC-4, Brent Lippert wrote:

I'm trying to use Keras w/TensorFlow (Python3) backend to build a Convolutional NN for NLP classification. The issue I'm having is when in the defining the shape of my input when building my network. I'm correctly following their API (not my first time using Keras) yet I am getting a variety of inexplicable errors.

----------  cut ---------

[Daπid]

On 31 March 2017 at 21:53, Brent Lippert <brent...@gmail.com> wrote:

As for using Convo1D, would that work? Each singular instance is a 2D matrix with maxRow rows (23 I think it was calculated to be) and 29 columns, wherein each row represents a single word with each column representing one of the words in my vocab so each row will be mostly 0s with a single 1 in the column of the word that row corresponds to. And there's only 1 channel being the word2vec embedding (though I may add other embeddings as other channels later). Conv1D would only be if each of my instances was a 1D vector right? Sorry I wasn't clear on what my instance dimensions were earlier, but I'm fairly certain I need Conv2D do I not?

Your sentence is a 2D vector, where the dimensions are (time, embedding). And this is what conv1D works with.

Note that Conv2D takes a 3D input (length, width, colour), but it is called 2D because the convolutions travel in length and width, taking all the colours at once. Similarly, Conv1D will travel along the time dimension, taking all the embedding dimensions at once.

Another important factor is to understand the convolution hypothesis. A convolution says that the same group of pixels is the same anywhere on an image, or that the same group of words have the same meaning regardless of if they are at the beginning or at the end, hence why you can share the weights. If you use 2D convolutions on your word embeddings you are saying that channel 1 and channel 2 have the same relation to each other than channels 23 and 24, but they aren't so related to each other.

On 31 March 2017 at 22:12, Robert McGwier <rwmc...@gmail.com> wrote:

You know things are bad when the examples in the github repository won't run on the latest version.

I declare this rock flew badly and went thud.

Help wanted! https://github.com/fchollet/keras/issues/5358

This is FOSS, and you still get a much better value than what you pay for!

[Benjamin Danek]

I had one of these errors pop up. It was this one: 

"The added layer must be an instance of class Layer. Found: (<keras.layers.convolutional.Conv2D object at 0x000002467C0C2EF0>,)"  

I was totally confused until I looked closer at my formatting. I had previously added all my layers directly into the model = sequential( [layer1], [layer2] .. [layerN]) but found I couldn't add all the layers in this way. I changed the format so that I made my model = sequential(), and then used the model.add([layer1]) function. My mistake was that I kept the comma behind each of these model.add() functions like "model.add(),". I removed them and everything was ok. 

Hope this helps