Zero validation accuracy when training with model.fit()

[Ahmed Baahmed]

Hey!

Hope you are doing well. Specifically, I am focusing on anomaly detection in time series data (multivariate). Although I am relatively new to Keras, I'm attempting to train a neural network to classify a dataset, categorizing time series as 1 if it is an anomaly and 0 otherwise. I've previously used Keras for other tasks, and everything appeared to function correctly. However, in this particular case, I am encountering a validation accuracy of zero after training. I have been unable to find any explanation for this. For instance, when attempting to train a classifier with "categorical_crossentropy" as the loss function and "adam" as the optimizer to classify into four classes, the output resembles the following:
# Importing the libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
from keras.models import Sequential
from keras.layers import Convolution1D, MaxPooling2D, Flatten, Dense

# Step 1:
#data = pd.read_csv("C:\\Users\\xps\\Desktop\\MLProjects\\Data\\Resting.csv")
data = pd.read_csv("/data/workingdir/abh/Math/IsolationForest/Resting.CSV")
data = data["Amplitude"]

# Define a threshold ..
# Compute mean and standard deviation of reconstruction errors
mean = np.mean(data)
std = np.std(data)

# Define threshold as multiple of standard deviation
#threshold = mean + 2 * std
threshold = 650
# Label the data based on the threshold
labels = np.where(data > threshold, 1, 0)

# Add the labels to the data
data = pd.DataFrame(data)
data['Labels'] = labels

# Create a subsequences (Tensor subsequence) from a 1d dataset

# Number of subsequences and Length of each subsequence
number_subsequences = 241
length_subsequences = 5000
# Channel dimension (set to 1 bcz its a time series 1d)
channel = 1
# Number of columns in the dataset
number_columns = data.shape[1]
# Calculate total number of data points
total_data_points = len(data)

# Calculate the stride to slide the window for creating subsequences
stride = (total_data_points - length_subsequences + 1) // number_subsequences

# Initialize the tensor to store subsequences
x_shape = (number_subsequences, length_subsequences, number_columns, channel)
x = np.zeros(x_shape)

# Iterate over each subsequence
for i in range(number_subsequences):
    start_index = i * stride
    end_index = start_index + length_subsequences
    subsequence = data.iloc[start_index:end_index, :].values
    #print(subsequence[:,0])
    # Reshape subsequence to fit into X tensor
    x[i, :, :, 0] = subsequence

# X tensor shape
print("Shape of x tensor:", x.shape)

from sklearn.model_selection import train_test_split

X = x[:, :, 0, :].squeeze(axis=-1)
y = x[:, :, 1, :].squeeze(axis=-1)

# Splitting the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv1D, MaxPooling1D, Flatten, Dense

cnn = Sequential([
    Conv1D(filters=128, kernel_size=3, activation='relu', input_shape=(length_subsequences, 1)),
    MaxPooling1D(pool_size=2),
   
    Conv1D(filters=256, kernel_size=3, activation='relu'),
    MaxPooling1D(pool_size=2),
    Flatten(),
    Dense(64, activation='relu'),
    Dense(5000, activation='softmax')  # Binary classification, so sigmoid activation
])

from tensorflow.keras.optimizers import Adam
# Part 1: Compiling the CNN
cnn.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

#learning_rate = 0.004
#optimizer = Adam(learning_rate=learning_rate)
#cnn.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy'])


# Reshape X_train to match the input shape expected by the model
X_train_reshaped = X_train.reshape((X_train.shape[0], X_train.shape[1], 1))
X_test_reshaped = X_test.reshape((X_test.shape[0], X_test.shape[1], 1))
# Train the model
history = cnn.fit(X_train_reshaped, y_train, epochs=100, batch_size=32, validation_split=0.2)

Epoch 1/100 5/5 [==============================] - 1s 104ms/step - loss: 74.3635 - accuracy: 0.0000e+00 - val_loss: 19.7074 - val_accuracy: 0.0000e+00 Epoch 2/100 5/5 [==============================] - 0s 47ms/step - loss: 9.2061 - accuracy: 0.0000e+00 - val_loss: 0.7590 - val_accuracy: 0.0000e+00 Epoch 3/100 5/5 [==============================] - 0s 48ms/step - loss: 0.6376 - accuracy: 0.0000e+00 - val_loss: 0.3085 - val_accuracy: 0.0000e+00 Epoch 4/100 5/5 [==============================] - 0s 46ms/step - loss: 0.2392 - accuracy: 0.0000e+00 - val_loss: 0.0916 - val_accuracy: 0.0000e+00 Epoch 5/100 5/5 [==============================] - 0s 46ms/step - loss: 0.0388 - accuracy: 0.0000e+00 - val_loss: 6.0347e-07 - val_accuracy: 0.0000e+00 Epoch 6/100 5/5 [==============================] - 0s 46ms/step - loss: 5.0125e-07 - accuracy: 0.0000e+00 - val_loss: 9.7785e-07 - val_accuracy: 0.0000e+00 Epoch 7/100 5/5 [==============================] - 0s 47ms/step - loss: 3.0494e-07 - accuracy: 0.0000e+00 - val_loss: 2.1861e-10 - val_accuracy: 0.0000e+00 Epoch 8/100 5/5 [==============================] - 0s 50ms/step - loss: 8.0590e-11 - accuracy: 0.0000e+00 - val_loss: 3.2112e-12 - val_accuracy: 0.0000e+00

[Matias Valdenegro]

Your model is configured for 5000 classes, while you seem to have 2 classes, so the last layer should have 1 neuron and sigmoid activation.