Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Using Caffe to create an MLP (Innerproduct) with MemoryData input

381 views
Skip to first unread message

Klaas Dijkstra

unread,
Aug 11, 2016, 2:38:26 PM8/11/16
to Caffe Users
Hello,

I am trying to use Caffe for training from memory data. I cannot use one of the database backends. I read here: http://caffe.berkeleyvision.org/tutorial/layers.html that I should use the MemoryData layer.
To try some easy things first, I want to train a 3 layer Multi-Layer Perceptron with 2 inputs and 2 outputs (2 classes). The hidden layer has 10 units. I use C++ for interfacing with Caffe.

I use the following prototxt:

layer {
  name: "data_layer"
  type: "MemoryData"
  top: "data_layer"
  top: "target_layer"
  memory_data_param {
    batch_size: 1
    channels: 1
    height: 1
    width: 2
  }
}
layer {
  name: "input_layer"
  type: "InnerProduct"
  bottom: "data_layer"
  top: "input_layer"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 2
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "input_layer_transfer"
  type: "Sigmoid"
  bottom: "input_layer"
  top: "input_layer"
}
layer {
  name: "hidden_layer"
  type: "InnerProduct"
  bottom: "input_layer"
  top: "hidden_layer"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 10
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "hidden_layer_transfer"
  type: "Sigmoid"
  bottom: "hidden_layer"
  top: "hidden_layer"
}
layer {
  name: "output_layer"
  type: "InnerProduct"
  bottom: "hidden_layer"
  top: "output_layer"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 2
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "output_layer_transfer"
  type: "SoftmaxWithLoss"
  bottom: "output_layer"
  bottom: "target_layer"
  top: "output_layer"
}


The creation fails on the final SoftmaxWithLoss. See below for the log file:

I0811 20:14:34.592681  9256 layer_factory.hpp:77] Creating layer data_layer
I0811 20:14:34.593682  9256 net.cpp:91] Creating Layer data_layer
I0811 20:14:34.593682  9256 net.cpp:399] data_layer -> data_layer
I0811 20:14:34.594669  9256 net.cpp:399] data_layer -> target_layer
I0811 20:14:34.595681  9256 net.cpp:141] Setting up data_layer
I0811 20:14:34.595681  9256 net.cpp:148] Top shape: 1 1 1 2 (2)
I0811 20:14:34.596681  9256 net.cpp:148] Top shape: 1 (1)
I0811 20:14:34.596681  9256 net.cpp:156] Memory required for data: 24
I0811 20:14:34.596681  9256 layer_factory.hpp:77] Creating layer input_layer
I0811 20:14:34.597681  9256 net.cpp:91] Creating Layer input_layer
I0811 20:14:34.597681  9256 net.cpp:425] input_layer <- data_layer
I0811 20:14:34.598670  9256 net.cpp:399] input_layer -> input_layer
I0811 20:14:34.599671  9256 common.cpp:36] System entropy source not available, using fallback algorithm to generate seed instead.
I0811 20:14:34.600682  9256 net.cpp:141] Setting up input_layer
I0811 20:14:34.600682  9256 net.cpp:148] Top shape: 1 2 (2)
I0811 20:14:34.600682  9256 net.cpp:156] Memory required for data: 40
I0811 20:14:34.601681  9256 layer_factory.hpp:77] Creating layer input_layer_transfer
I0811 20:14:34.601681  9256 net.cpp:91] Creating Layer input_layer_transfer
I0811 20:14:34.601681  9256 net.cpp:425] input_layer_transfer <- input_layer
I0811 20:14:34.602681  9256 net.cpp:386] input_layer_transfer -> input_layer (in-place)
I0811 20:14:34.602681  9256 net.cpp:141] Setting up input_layer_transfer
I0811 20:14:34.602681  9256 net.cpp:148] Top shape: 1 2 (2)
I0811 20:14:34.602681  9256 net.cpp:156] Memory required for data: 56
I0811 20:14:34.603682  9256 layer_factory.hpp:77] Creating layer hidden_layer
I0811 20:14:34.603682  9256 net.cpp:91] Creating Layer hidden_layer
I0811 20:14:34.603682  9256 net.cpp:425] hidden_layer <- input_layer
I0811 20:14:34.604681  9256 net.cpp:399] hidden_layer -> hidden_layer
I0811 20:14:34.605670  9256 net.cpp:141] Setting up hidden_layer
I0811 20:14:34.605670  9256 net.cpp:148] Top shape: 1 10 (10)
I0811 20:14:34.605670  9256 net.cpp:156] Memory required for data: 136
I0811 20:14:34.606681  9256 layer_factory.hpp:77] Creating layer hidden_layer_transfer
I0811 20:14:34.606681  9256 net.cpp:91] Creating Layer hidden_layer_transfer
I0811 20:14:34.606681  9256 net.cpp:425] hidden_layer_transfer <- hidden_layer
I0811 20:14:34.606681  9256 net.cpp:386] hidden_layer_transfer -> hidden_layer (in-place)
I0811 20:14:34.607682  9256 net.cpp:141] Setting up hidden_layer_transfer
I0811 20:14:34.607682  9256 net.cpp:148] Top shape: 1 10 (10)
I0811 20:14:34.607682  9256 net.cpp:156] Memory required for data: 216
I0811 20:14:34.607682  9256 layer_factory.hpp:77] Creating layer output_layer
I0811 20:14:34.607682  9256 net.cpp:91] Creating Layer output_layer
I0811 20:14:34.608682  9256 net.cpp:425] output_layer <- hidden_layer
I0811 20:14:34.608682  9256 net.cpp:399] output_layer -> output_layer
I0811 20:14:34.609683  9256 net.cpp:141] Setting up output_layer
I0811 20:14:34.610682  9256 net.cpp:148] Top shape: 1 2 (2)
I0811 20:14:34.610682  9256 net.cpp:156] Memory required for data: 232
I0811 20:14:34.610682  9256 layer_factory.hpp:77] Creating layer output_layer_transfer
I0811 20:14:34.611670  9256 net.cpp:91] Creating Layer output_layer_transfer
I0811 20:14:34.611670  9256 net.cpp:425] output_layer_transfer <- output_layer
I0811 20:14:34.611670  9256 net.cpp:425] output_layer_transfer <- target_layer
I0811 20:14:34.611670  9256 net.cpp:386] output_layer_transfer -> output_layer (in-place)
I0811 20:14:34.611670  9256 layer_factory.hpp:77] Creating layer output_layer_transfer
F0811 20:14:34.612669  9256 blob.hpp:122] Check failed: axis_index < num_axes() (1 vs. 0) axis 1 out of range for 0-D Blob with shape (1)

I suspect it has something to do with the "target_layer". Only I don't quite understand what Caffe tries to tell me. I've tried to use an "Input" layer instead of an "MemoryData" layer, but this gives me the same error. I cannot find anything about 0-D blob on internet or this google group. Although I understand the idea, there are not many examples of using MemoryData on internet.

What am I doing wrong? Help is appreciated.

Klaas Dijkstra

unread,
Aug 12, 2016, 2:22:59 AM8/12/16
to Caffe Users
I actually found a good MLP example here: https://github.com/beniz/deepdetect/blob/master/templates/caffe/mlp/mlp.prototxt
This helped me to figure out that the SoftmaxWithLoss layer cannot be inplace. So I replaced the loss layer with the following:

layer {
  name: "output_layer_transfer"
  type: "SoftmaxWithLoss"
  bottom: "output_layer"
  bottom: "target_layer"

  top: "output_layer_transfer"
}

Now the network initialized fine.
Reply all
Reply to author
Forward
0 new messages