Thanks. I've been working a bit on the demo and I think I managed to add a simple replay buffer using the same logic as the
val trainingSamples: MutableList<TrainingSample> = mutableListOf().
I have 2 questions in case anyone could offer help:
1: Is there a way to debug the app besides using Logcat so I can see that everything works correctly? (Bottleneck, training cycles, batches, the replay buffer etc.)
Experimenting with the demo I found out that for example, on the first training I added 20 samples of 1 class and 20 samples of another class (2 different objects). I trained the model and performed inference fine. I then went to perform a 2nd training by adding 20 samples to a 3rd class (different object again). When training, the loss remained high at around 3-5 and the inference was completely wrong on the 3rd class. I want to understand why this occurs, I am guessing it is expected with transfer learning however I implemented my replay buffer so I want a way to debug to see what works incorrectly. With the replay buffer, the 3rd class should be able to be inferenced correctly as well.
From what I understood, every click on the training button to add new samples and perform on-device training again only uses the samples taken during that moment and not from previously correctly? So, a simple mutablelist that stores trainingsamples and acts as ar replay buffer should work in utilizing during the training those samples plus the samples taken during that training cycle right?
2: I also went through the model creation script in python that creates the training signatures to be used by the interpreter in android studio (generate_training_model.py). Am I able to add hidden layers in the training signature besides just a fully connected layer with softmax activation with the purpose of those hidden layers be trained on device? (The idea for this is to copy the last few MobileNetV2 layers and add them again in the training function so they can be trained on the device)
For reference, this is the function I am talking about: (I also guess __init__ should be changed too, right?)
def __init__(self, learning_rate=0.001):
"""Initializes a transfer learning model instance.
Args:
learning_rate: A learning rate for the optimzer.
"""
self.num_features = NUM_FEATURES
self.num_classes = NUM_CLASSES
# trainable weights and bias for softmax
self.ws = tf.Variable(
tf.zeros((self.num_features, self.num_classes)),
name='ws',
trainable=True)
self.bs = tf.Variable(
tf.zeros((1, self.num_classes)), name='bs', trainable=True)
# base model
self.base = tf.keras.applications.MobileNetV2(
input_shape=(IMG_SIZE, IMG_SIZE, 3),
alpha=1.0,
include_top=False,
weights='imagenet')
# loss function and optimizer
self.loss_fn = tf.keras.losses.CategoricalCrossentropy()
self.optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
@tf.function(input_signature=[
tf.TensorSpec([None, NUM_FEATURES], tf.float32),
tf.TensorSpec([None, NUM_CLASSES], tf.float32),
])
def train(self, bottleneck, label):
"""Runs one training step with the given bottleneck features and labels.
Args:
bottleneck: A tensor of bottleneck features generated from the base model.
label: A tensor of class labels for the given batch.
Returns:
Map of the training loss.
"""
with tf.GradientTape() as tape:
logits = tf.matmul(bottleneck, self.ws) + self.bs
prediction = tf.nn.softmax(logits)
loss = self.loss_fn(prediction, label)
gradients = tape.gradient(loss, [self.ws, self.bs])
self.optimizer.apply_gradients(zip(gradients, [self.ws, self.bs]))
result = {'loss': loss}
for grad in gradients:
result[grad.name] = grad
return result