Nan Loss for Autoencoder

[arriv...@gmail.com]

Dear group,

it is my aim to train a simple autoencoder, which samples down the dataset from 30 or so columns to 3 and then (middle layer) and then reconstructs it to 30 or so dimensions. The aim here is to measure the reconstruction error, giving me an idea of how good the network is at encoding the data and preserving its structure.When training, I get nans for the loss. The data looks totally normal and the inputs are nd numpy arrays. 

Anybody an Idea

The code is the following

I normalize the data. Outcome looks normal.

or i in range(0,len(data_prep.columns)):

    #print(i)

    try:

        data_prep.iloc[:,i] = (data_prep.iloc[:, i]).astype(float)

    except ValueError:

        print('float war nicht moeglich jetzt int')

        data_prep.iloc[:,i] = (data_prep.iloc[:, i]).astype(int)        

    maximum = np.nanmax(np.asarray(data_prep.iloc[:, i]))

    minimum = np.nanmin(np.asarray(data_prep.iloc[:, i]))

    rangevalues = maximum-minimum

    for x in range(0,400):

        data_prep.iloc[x, i] = (data_prep.iloc[x, i] - minimum)/rangevalues

        

#-------------Variable Declaration--------------------------------------

#Anzahl der Spalten von denen wir ausgeganen sind

input_data = Input(shape=(numberofdimensions,))

#---------------------Autoencoder Modell----------------------------------

model = Sequential()

model.add(keras.layers.Dense(3, input_shape = (numberofdimensions,),activation='relu'))

model.add(keras.layers.Dense(numberofdimensions,activation='relu'))

model.compile(optimizer='rmsprop', loss='mean_squared_error')

## ive tried different loss functions

#just splitting the data but not really using Y as validation yet.

X = data_prep.iloc[0: int(len(data_prep.index) * 0.7),0:numberofdimensions]

Y = data_prep.iloc[int(len(data_prep.index) * 0.7):399,0:numberofdimensions]

X = np.asarray(X)

Y = np.asarray(Y)

 model.fit(X, X, 

         batch_size=256, epochs=number_of_epochs, verbose=2)

[Daπid]

Do you have nans in your data?

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/e538d947-4924-4d23-9f81-7f6cb3cb706e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[arriv...@gmail.com]

Hallo David,  

thank you for your comment. It made me think about the nans issue. I have nans indeed, which in normal cases is alright. As long as you have the target y, a vecotr with nas could still be used for learning and weight updates. But since for the autoencoder the target is x, and the input data is x as well, the nas make it apparently impossible to calculate the cost. I guess you cannot have nas in your y s, which in our case is the x vecotr as is the input vector.

[Daπid]

No, nans are never ok. Any number multiplied by nan is a nan, so the output of a dense layer with nans in the inputs is all nans.

Clean your data before feeding it to your machine learning model. 

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users+unsubscribe@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/625809e7-f73e-4ea0-9d6b-b992d4e79dbc%40googlegroups.com.

[arriv...@gmail.com]

You're right... I didn't think it through... nan + x is always nan...I guess even if it would be factored it as 0, it would create some bumps in the loss function and maybe lead to some weights being trained the wrong way...even though thats kind of close to the idea of regularization

To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.