Dueling Network Architectures in Caffe

[Patrick Jörger]

Hi,

I am rather new to caffe and wanted to build a dueling network according to the paper "Dueling Network Architectures for Deep Reinforcement Learning". I am stuck at the point where I have to subtract a scalar (the mean of the advantage values) computed by a reduction layer from a vector (5x1) computed by a InnerProduct layer.

Eltwise layer did not work out since both blobs have different dimensions.

My next attempt was using the concat layer to concat the scalar step-by-step into a 5x1 vector and then use Eltwise layer to subtract one from the other. This did not work out and I got the Error: "Check failed: axis_index < num_axes() (0 vs. 0) axis 0 out of range for 0-D Blob with shape (1)"

The concat layer I use looks like this:

layer{
  name: "advantage_mean_concat_1"
  bottom: "advantage_mean"
  bottom: "advantage_mean"
  top: "advantage_mean_broadcast_1"
  type: "Concat"
  concat_param {
    axis: 0
  }
}

Does anyone has an idea what my problem is and how I can make such a dueling architecture work? I have not found any tutorials or posts regarding dueling architectures in caffe so it would also be appreciated if anyone knows about such tutorials or posts.

Thanks.