How to define and train from multiple labels in .prototxt?

[Diego Durante]

Hello!

I have problems to define input sources with multiple labels in order to train a network in caffe, I cannot understand how to define the input sources.

I'm modifing caffenet in order to get multiple labels and multiple loss and accuracy optimizations.


In details:

I have one label file, that has some labels_weather.txt, such as:

 1. 0.jpg 0
 2. 1.jpg 3
 3. 2.jpg 1
 4. 3.jpg 0
 5. 4.jpg 2
 6. ...

Also, I have another file (labels_day_night.txt), that have information such as

 1. 0.jpg 0
 2. 1.jpg 1
 3. 2.jpg 0
 4. 3.jpg 0
 5. 4.jpg 1
 6. ...


My problems are:

 1. How can I create an LMDB database that take inputs from both data labels
 2. How can I add multiple data input sources
 3. There is a better way to solve this problem?


Here the first part of my .prototxt file (there are errors on this part):


```

# My problem is: I don't know how to create LMDB library, and how to create input data layers on prototxt file
name: "Caffenet"

layers {
  name: "data"
  type: DATA
  top: "data"
  top: "label-weather"
  data_param {
    source: "/home/diego/Code/dayweatherDeepLearning/Folds/lmdb/Test_fold_is_0/weather_train_lmdb/"
    backend: LMDB
    batch_size: 50
  }
  transform_param {
    crop_size: 227
    mean_file: "/home/diego/Code/dayweatherDeepLearning/Folds/mean/Test_fold_is_0/mean.binaryproto"
    mirror: true
  }
  include: { phase: TRAIN }
}
layers {
  name: "data"
  type: DATA
  top: "data"
  top: "label-weather"
  data_param {
    source:  "/home/diego/Code/dayweatherDeepLearning/Folds/lmdb/Test_fold_is_0/weather_val_lmdb"
    backend: LMDB
    batch_size: 50
  }
  transform_param {
    crop_size: 227
    mean_file: "/home/diego/Code/dayweatherDeepLearning/Folds/mean/Test_fold_is_0/mean.binaryproto"
    mirror: false
  }
  include: { phase: TEST }
}

layers {
  name: "data"
  type: DATA
  top: "data"
  top: "label-day"
  data_param {
    source: "/home/diego/Code/dayweatherDeepLearning/Folds/lmdb/Test_fold_is_0/day_train_lmdb/"
    backend: LMDB
    batch_size: 50
  }
  transform_param {
    crop_size: 227
    mean_file: "/home/diego/Code/dayweatherDeepLearning/Folds/mean/Test_fold_is_0/mean.binaryproto"
    mirror: true
  }
  include: { phase: TRAIN }
}
layers {
  name: "data"
  type: DATA
  top: "data"
  top: "label-day"
  data_param {
    source:  "/home/diego/Code/dayweatherDeepLearning/Folds/lmdb/Test_fold_is_0/day_val_lmdb"
    backend: LMDB
    batch_size: 50
  }
  transform_param {
    crop_size: 227
    mean_file: "/home/diego/Code/dayweatherDeepLearning/Folds/mean/Test_fold_is_0/mean.binaryproto"
    mirror: false
  }
  include: { phase: TEST }
}

```

Next, I have multiple loss on train_val_test.prototxt, here is the end of the file (accuracy and loss definitions):

```

...
#First layers here
# accuracy and loss layers:
layers {
  name: "accuracy-weather"
  type: ACCURACY
  bottom: "fc8-weather"
  bottom: "label-weather"
  top: "accuracy-weather"
  include: { phase: TEST }
}
layers {
  name: "loss-weather"
  type: SOFTMAX_LOSS
  bottom: "fc8-weather"
  bottom: "label-weather"
  top: "loss-weather"
}

layers {
  name: "accuracy-day"
  type: ACCURACY
  bottom: "fc8-day"
  bottom: "label-day"
  top: "accuracy-day"
  include: { phase: TEST }
}
layers {
  name: "loss-day"
  type: SOFTMAX_LOSS
  bottom: "fc8-day"
  bottom: "label-day"
  top: "loss-day"
}

```
Thanks for all!