Using the Python Layer

[Duncan Goudie]

Dear Caffe Users, this is my first post on here so please bear with me.

I've been trying to train a simple DNN with a Python Layer using the latest master version of Caffe, but have been encountering errors. The Caffe trainer returns:

[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 27:9: Unknown enumeration value of "PYTHON" for field "type".

This is the prototext I use:

layers {
    name: "python_multiplier"
    type: PYTHON
    bottom: "data"
    top: "python_multiplier"
    python_param {
        module: "python/caffe/test/test_python_layer.py"
        layer: "SimpleLayer"
    }
}

It thinks the problem is with the "type" field. I have also tried: Python, "Python" and PYTHON_LAYER, but to no avail.

I have been looking in the source code. I looked at the layer factory, uncommented the

LOG(INFO) << "Registering layer type: " << type;

line so that I could see if it was being registered, and it was:

I0818 11:23:56.855293  3744 layer_factory.hpp:107] Registering layer type: Python

I have also uncommented the line:

WITH_PYTHON_LAYER := 1

within makefile.config and re-compiled the whole package.

Does anyone know how to get the Python Layer to work in Caffe?

What are we supposed to put within the "module" field within python_param? Is this the path to the .py file containing the python layer?

[zzz]

Hi,

Have you compared with you configuration with the python layer example in examples/pycaffe.

The example use "Python" as type name.

[Duncan Goudie]

Thank you very much, I didn't know there was a Python Layer example in that folder. It accepts it and starts training now. Not much seems to be different so it is strange that it accepted a copy and pasting of

layer {
  type: "Python"
  name: 'loss'
  top: 'loss'
  bottom: 'ip4'
  bottom: 'label'
  python_param {
    # the module name -- usually the filename -- that needs to be in $PYTHONPATH
    module: 'pyloss'
    # the layer name -- the class name in the module
    layer: 'EuclideanLossLayer'
  }
  # set loss weight so Caffe knows this is a loss layer.
  # since PythonLayer inherits directly from Layer, this isn't automatically
  # known to Caffe
  loss_weight: 1
}

and not my original syntax.

[zzz]

:)

[Peerajak Witoonchart]

Hi I just make an ipynb documentation on this issue. I would like to share it here. Please help check if my code is correct.
เมื่อ วันอังคารที่ 18 สิงหาคม ค.ศ. 2015 18 นาฬิกา 00 นาที 26 วินาที UTC+7, Duncan Goudie เขียนว่า:

[Duncan Goudie]

I just realised why my old prototext did not work. I was using the old style prototext (I started using Caffe in Jan 2015). So anyone who got used to the old style and is dipping their fingers into developing new layers with the most up to date Caffe, make sure that your prototext for layers are: "layer" and not "layers". See the examples in caffe_root/examples/.

[Wong Fungtion]

@Duncan Goudie, Is there any way to convert the caffemodel pre-trained using "layers" to the one using "layer"? For example VGG16, I want to add a new python loss layer to it, but I want to using the pre-trained caffemodel in model zoo using "layers".  

在 2015年8月28日星期五 UTC+8上午1:01:42，Duncan Goudie写道：

[Jeremy Rutman]

my one attempt at a pr got negged since there is already code for this I didn't see - 

https://github.com/BVLC/caffe/blob/master/tools/upgrade_net_proto_text.cpp

[Wong Fungtion]

Yes, this script works perfect and the .caffemodal file works on both "layer" and "layers".

在 2017年2月28日星期二 UTC+8下午9:34:46，Jeremy Rutman写道：