I want to implement multi task networks using Caffe. From what I have understood, if I have 3 tasks, I need to split the final shared layer into 3 layers, and send each resulting layer to its own softmax layer and subsequent loss layer. I can give weights to the different loss layers using the loss_weight parameter to get a weighted sum of the loss. However, I am unable to understand what should be the input to the network. Do I simply have a data layer with multiple tops, one for the data and the others for each label corresponding to each task?
If I have 3 tasks, then each data sample has 3 labels associated with it. In that case, will using something like this suffice:
source: "path/to/leveldb"
As far as I realised, leveldb can take only one label associated with an image. In that case, how do I send data to a multi task network, with labels for each task? Do I define multiple data layers? Each data layer will have the same data, but different labels for different tasks?