How can I specify the validation set in caffe?

[Hossein Hasanpour]

Hello all,
usually we have some training data, a validation set and then a test set.
So far in all of caffe examples that I have seen, there was only a train and test option , no sign of any validation set related configuration .
So now, I want to use validation set in my training as well, How can I do this in caffe? is it even possible in caffe?

Thanks in advance

[Jan]

Theoretically yes, you can use several "stages" for each "phase", so you could make two stages for the TEST phase, with different datasets.

layer {
  type: "Data"
  ...
  include {
    phase: TRAIN
  }
}

layer {
  type: "Data"
  ...
  include {
    phase: TEST
    stage: "validation"
  }
}

layer {
  type: "Data"
  ...
  include {
    phase: TEST
    stage: "test"
  }
}


Then you need to add a test_iter and test_stage for every stage in your solver.prototxt:

test_iter: 200
test_state {
  stage: "validation"
}
test_iter: 100
test_state {
  stage: "test"
}

Or: you can simply identify the TEST phase with the validation dataset completely. Then, when the training is done, use the trained net ("deploy" version) on your test set and compute a final accuracy value. That is my understanding of an actual test set (if you have separate validation and test sets). I guess the BVLC guys just called the phase TEST since you often do not have three datasets (train, validate, test), but only two, and then the name of the second set is not clear, sometimes its calles test, sometimes validate. But I guess the usual workflow is to do the test evaluation after the training.

Jan