Caffe Regression with HDF5Data. Loss doesn't decrease. Same Output for any input.

190 views
Skip to first unread message

Arjun Varshney

unread,
Jul 25, 2016, 7:29:13 PM7/25/16
to Caffe Users
Hello Caffe'ers, 

I am trying to train my data (regression) which is of the input with dimensions( n* 3* 30 * 5) contains continuous frequency values. The label is of the form (n * 1 * 1 * 1 ) whic is also continuous variable ( Frequency Value).
For this I have used HDF5Data as input layer to Caffe and I have started with very basic Logistic Regression example (One data Layer, One FC Layer and one Euclidean Loss Layer) in my network architecture. 
I see that It doesn't get trained. The networks keeps guessing after few iterations. 
I've tried to flatten the data ( n * 1 * 1 * x) and then train, but the result is same. 

In the logs, I see high loss but after few iterations it stabilizes and never decreases ( and then starts oscillating).
I get same output values for any input. I have read few posts in this forum in the past for this but it didn't help. 

To me, it looks more of the HDF5Data problem rather than network architecture. I have tried with varying learning rate and changing type of Solver Algos. 

Here is my solver and train_test.prototxt files. 

Training Size : 5400
Testing Size: 600

Solver :

test_iter: 30
test_interval: 100
base_lr: 1e-6
momentum: 0.001
weight_decay: 0.0005
lr_policy: "inv"
gamma: 0.001
power: 0.75
display: 200
max_iter: 10000

Train_Test prototxt: 

layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "/home/augmented_Fdata/tr_checkdata.txt"
    batch_size: 20
    shuffle : true
  }
}

layer {
  name: "data"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  hdf5_data_param {
    source: "/home/augmented_Fdata/te_checkdata.txt"
    batch_size: 20
    shuffle : true
  }
}

layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "data"
  top: "ip1"
  inner_product_param {
    num_output: 1
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "loss"
  type: "EuclideanLoss"
  bottom: "ip1"
  bottom: "label"
  top: "loss"
}

Note: I have even tried adding few more layers ( two fully connected layers with/without ReLU Layer as another simplistic approach but I get the same result)

Any thoughts/ hints/ solutions?


kishen suraj P

unread,
Feb 22, 2017, 11:24:10 PM2/22/17
to Caffe Users
Constant prediction is the bias value of the last layer. All your weight would mostly be zero.

jiljji

unread,
Feb 23, 2017, 12:53:34 AM2/23/17
to Caffe Users
could you write the whole train_val net?

在 2016年7月26日星期二 UTC+8上午7:29:13，Arjun Varshney写道：

Ecaterina Damian

unread,
Dec 14, 2017, 4:52:42 AM12/14/17
to Caffe Users
Hello Kishen, would you mind expanding on this? I've been struggling with this problem for a while now and your answer is the first one that promises an actual solution.

Many thanks,
Cat

Nicola Fiorato

unread,
Dec 17, 2017, 3:24:31 PM12/17/17
to Caffe Users
I think it's a problem of weights initialization, to verify try to print the weights using the net->params() instructions to  retrieve the vector of weights.
Try to: specify bias_filler as constant in every layer, gaussian weight_filler I think it's better for regression, std = 0.1 could be good.
Reply all
Reply to author
Forward
0 new messages