Using Caffe to create an MLP (Innerproduct) with MemoryData input

381 views
Skip to first unread message

Klaas Dijkstra

unread,
Aug 11, 2016, 2:38:26 PM8/11/16
to Caffe Users
Hello,

I am trying to use Caffe for training from memory data. I cannot use one of the database backends. I read here: http://caffe.berkeleyvision.org/tutorial/layers.html that I should use the MemoryData layer.
To try some easy things first, I want to train a 3 layer Multi-Layer Perceptron with 2 inputs and 2 outputs (2 classes). The hidden layer has 10 units. I use C++ for interfacing with Caffe.

I use the following prototxt:

layer {
  name: "data_layer"
  type: "MemoryData"
  top: "data_layer"
  top: "target_layer"
  memory_data_param {
    batch_size: 1
    channels: 1
    height: 1
    width: 2
  }
}
layer {
  name: "input_layer"
  type: "InnerProduct"
  bottom: "data_layer"
  top: "input_layer"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 2
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "input_layer_transfer"
  type: "Sigmoid"
  bottom: "input_layer"
  top: "input_layer"
}
layer {
  name: "hidden_layer"
  type: "InnerProduct"
  bottom: "input_layer"
  top: "hidden_layer"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 10
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "hidden_layer_transfer"
  type: "Sigmoid"
  bottom: "hidden_layer"
  top: "hidden_layer"
}
layer {
  name: "output_layer"
  type: "InnerProduct"
  bottom: "hidden_layer"
  top: "output_layer"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 2
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "output_layer_transfer"
  type: "SoftmaxWithLoss"
  bottom: "output_layer"
  bottom: "target_layer"
  top: "output_layer"
}


The creation fails on the final SoftmaxWithLoss. See below for the log file:

I0811 20:14:34.592681  9256 layer_factory.hpp:77] Creating layer data_layer
I0811 20:14:34.593682  9256 net.cpp:91] Creating Layer data_layer
I0811 20:14:34.593682  9256 net.cpp:399] data_layer -> data_layer
I0811 20:14:34.594669  9256 net.cpp:399] data_layer -> target_layer
I0811 20:14:34.595681  9256 net.cpp:141] Setting up data_layer
I0811 20:14:34.595681  9256 net.cpp:148] Top shape: 1 1 1 2 (2)
I0811 20:14:34.596681  9256 net.cpp:148] Top shape: 1 (1)
I0811 20:14:34.596681  9256 net.cpp:156] Memory required for data: 24
I0811 20:14:34.596681  9256 layer_factory.hpp:77] Creating layer input_layer
I0811 20:14:34.597681  9256 net.cpp:91] Creating Layer input_layer
I0811 20:14:34.597681  9256 net.cpp:425] input_layer <- data_layer
I0811 20:14:34.598670  9256 net.cpp:399] input_layer -> input_layer
I0811 20:14:34.599671  9256 common.cpp:36] System entropy source not available, using fallback algorithm to generate seed instead.
I0811 20:14:34.600682  9256 net.cpp:141] Setting up input_layer
I0811 20:14:34.600682  9256 net.cpp:148] Top shape: 1 2 (2)
I0811 20:14:34.600682  9256 net.cpp:156] Memory required for data: 40
I0811 20:14:34.601681  9256 layer_factory.hpp:77] Creating layer input_layer_transfer
I0811 20:14:34.601681  9256 net.cpp:91] Creating Layer input_layer_transfer
I0811 20:14:34.601681  9256 net.cpp:425] input_layer_transfer <- input_layer
I0811 20:14:34.602681  9256 net.cpp:386] input_layer_transfer -> input_layer (in-place)
I0811 20:14:34.602681  9256 net.cpp:141] Setting up input_layer_transfer
I0811 20:14:34.602681  9256 net.cpp:148] Top shape: 1 2 (2)
I0811 20:14:34.602681  9256 net.cpp:156] Memory required for data: 56
I0811 20:14:34.603682  9256 layer_factory.hpp:77] Creating layer hidden_layer
I0811 20:14:34.603682  9256 net.cpp:91] Creating Layer hidden_layer
I0811 20:14:34.603682  9256 net.cpp:425] hidden_layer <- input_layer
I0811 20:14:34.604681  9256 net.cpp:399] hidden_layer -> hidden_layer
I0811 20:14:34.605670  9256 net.cpp:141] Setting up hidden_layer
I0811 20:14:34.605670  9256 net.cpp:148] Top shape: 1 10 (10)
I0811 20:14:34.605670  9256 net.cpp:156] Memory required for data: 136
I0811 20:14:34.606681  9256 layer_factory.hpp:77] Creating layer hidden_layer_transfer
I0811 20:14:34.606681  9256 net.cpp:91] Creating Layer hidden_layer_transfer
I0811 20:14:34.606681  9256 net.cpp:425] hidden_layer_transfer <- hidden_layer
I0811 20:14:34.606681  9256 net.cpp:386] hidden_layer_transfer -> hidden_layer (in-place)
I0811 20:14:34.607682  9256 net.cpp:141] Setting up hidden_layer_transfer
I0811 20:14:34.607682  9256 net.cpp:148] Top shape: 1 10 (10)
I0811 20:14:34.607682  9256 net.cpp:156] Memory required for data: 216
I0811 20:14:34.607682  9256 layer_factory.hpp:77] Creating layer output_layer
I0811 20:14:34.607682  9256 net.cpp:91] Creating Layer output_layer
I0811 20:14:34.608682  9256 net.cpp:425] output_layer <- hidden_layer
I0811 20:14:34.608682  9256 net.cpp:399] output_layer -> output_layer
I0811 20:14:34.609683  9256 net.cpp:141] Setting up output_layer
I0811 20:14:34.610682  9256 net.cpp:148] Top shape: 1 2 (2)
I0811 20:14:34.610682  9256 net.cpp:156] Memory required for data: 232
I0811 20:14:34.610682  9256 layer_factory.hpp:77] Creating layer output_layer_transfer
I0811 20:14:34.611670  9256 net.cpp:91] Creating Layer output_layer_transfer
I0811 20:14:34.611670  9256 net.cpp:425] output_layer_transfer <- output_layer
I0811 20:14:34.611670  9256 net.cpp:425] output_layer_transfer <- target_layer
I0811 20:14:34.611670  9256 net.cpp:386] output_layer_transfer -> output_layer (in-place)
I0811 20:14:34.611670  9256 layer_factory.hpp:77] Creating layer output_layer_transfer
F0811 20:14:34.612669  9256 blob.hpp:122] Check failed: axis_index < num_axes() (1 vs. 0) axis 1 out of range for 0-D Blob with shape (1)

I suspect it has something to do with the "target_layer". Only I don't quite understand what Caffe tries to tell me. I've tried to use an "Input" layer instead of an "MemoryData" layer, but this gives me the same error. I cannot find anything about 0-D blob on internet or this google group. Although I understand the idea, there are not many examples of using MemoryData on internet.

What am I doing wrong? Help is appreciated.

Klaas Dijkstra

unread,
Aug 12, 2016, 2:22:59 AM8/12/16
to Caffe Users
I actually found a good MLP example here: https://github.com/beniz/deepdetect/blob/master/templates/caffe/mlp/mlp.prototxt
This helped me to figure out that the SoftmaxWithLoss layer cannot be inplace. So I replaced the loss layer with the following:

layer {
  name: "output_layer_transfer"
  type: "SoftmaxWithLoss"
  bottom: "output_layer"
  bottom: "target_layer"

  top: "output_layer_transfer"
}

Now the network initialized fine.
Reply all
Reply to author
Forward
0 new messages