Gradient of model w.r.t. inputs

[Mohammad Amin Nabian]

Hello everyone, I need to take the derivative of a neural network w.r.t. its inputs. This is a very simple example:

X = np.array([8.])
B = np.array([2.])
X_train, X_eval, B_train, B_eval = cross_validation.train_test_split(X, B, test_size=0.0)
model = Sequential()
model.add(Dense(output_dim=1, input_shape=(1,)))
model.add(Activation(‘linear’))
model.compile(loss=‘mae’, optimizer=‘adam’)
model.fit(X_train, B_train, nb_epoch=100, batch_size=1)
outputTensor = model.output
listOfVariableTensors = model.inputs[0]
gradients = k.gradients(outputTensor, listOfVariableTensors)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
evaluated_gradients = sess.run(gradients,feed_dict={model.input:X_train[0].reshape(-1,1)})
evaluated_gradients

which builds a linear model (y=ax+b). The result for the gradient w.r.t. x should be a, but surprisingly, I get (a-b) from the code. (here is the results I get: weight=0.190165, bias=0.10000006,gradient=0.09016512). Could someone please explain this? What is going wrong here? Thanks! 

[Daπid]

I get the correct results to machine precision:

sess.run(model.layers[0].weights[0][:]) == sess.run(gradients,feed_dict={model.input:X.reshape(-1,1)})[0]

array([[ True]], dtype=bool)

Same result if I initialise the bias to 1.

    import numpy as np
    from keras.models import Sequential
    from keras.layers import Dense
    from keras import backend as k
    import tensorflow as tf

    X = np.array([8.])
    B = np.array([2.])

    model = Sequential()
    model.add(Dense(output_dim=1, input_shape=(1,), bias_initializer='ones'))

    model.compile(loss='mae', optimizer='adam')

    model.fit(X, B, nb_epoch=100, batch_size=1)

    outputTensor = model.output
    listOfVariableTensors = model.inputs[0]
    gradients = k.gradients(outputTensor, listOfVariableTensors)
    sess = tf.InteractiveSession()
    sess.run(tf.initialize_all_variables())

    evaluated_gradients = sess.run(gradients,feed_dict={model.input:X[0].reshape(-1,1)})

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/73551466-6316-4021-a5f7-725d6735f86f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.