LayerParameter has no field named "layer".

[Jeremy Rutman]

I have used python datalayers in a voc-fcnn model (from the shelhamer goldmine) and it runs fine - 

I am using the latest main branch caffe.

However when I try this same datalayer with a different network (alexnet also from shelamer's fcn) I hit an error 

$ /root/caffe/build/tools/caffe train  -solver solver.prototxt -weights fcn-alexnet-pascal.caffemodel   -gpu 0

...
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 375:7: Message type "caffe.LayerParameter" has no field named "layer".

my pythonlayer (which again works with the voc net) looks like 

layer {
  name: "data"
  type: "Python"
  top: "data"
  top: "label"
  python_param {
    module: "jrlayers"
    layer: "JrDataLayer"
    param_str: "{\'images_dir\': \'/root/imgdbs/train/\', \'labels_dir\': \'/root/imgdbs/labels/',\'seed\': 1337, \'split\': \'train\', \'imagesfile\':\'trainimages.txt\', \'mean\': (120, 120, 120)}"
  }

I took out the phase-train and phase-test directives and split the original train_val.prototxt into train.proto and val.proto in an attempt to make this all look as much like the working net  as possible, to no avail.

If I try to avoid the python layer using lmdb I hit a different issue involving the size of the labels vs. the output

Anyway since the only thing I am changing is the weights file , is there info in the weightfile concerning what type of layers it wants? seems unlikely but I cant think of why else this wouldnt work 

A somewhat related question, is there any difference between 'finetuning' ie starting from a weightsfile a-la

caffe train  -solver solver.prototxt -weights fcn-alexnet-pascal.caffemodel

and 'resuming training' as in 

caffe train --solver=solver.prototxt --snapshot=train_iter_10000.solverstate

[Jeremy Rutman]

whoops it looks like i need to be using the branch at https://github.com/longjon/caffe/tree/future

my bad

[Jeremy Rutman]

still no dice, I am hitting 

F0502 10:55:49.009650 15512 softmax_loss_layer.cpp:42] Check failed: outer_num_ * inner_num_ == bottom[1]->count() (240000 vs. 720000) Number of labels must match number of predictions; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be N*H*W, with integer values in {0, 1, ..., C-1}.

the 720,000 is my 3x600x400 input image while the 240000 is 600x400 ; i actually copied my label image (using label=np.array([label,label,label]))  but wierdly still get the 24000.
anyone have luck doing alexnet fcnn?

[Jan]

Have you set softmax_axis: 1 in your SoftmaxWithLoss layer? This is what you normally want with FCNs.

Jan

[Jeremy Rutman]

till now i hadn't heard of the softmax_axis parameter - I tried it and hit

[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 428:17: Message type "caffe.LossParameter" has no field named "softmax_axis".

on both master and fcn branches.

I did see the 'axis' param in the crop layer coming before softmax so I tried setting that to axis: 1  without the softmax_axis,  but it also does not seem to help matters; in fcn branch it is not recognized, while in master branch I still hit the 'number of labels must match number of predictions' as above. 
Incidentally I am a bit puzzled by the number of predictions being reported as 3*600*400 as opposed to n_classes*600*400 (in my case n_classes being 57)

In any case where could I find out what the axis params are doing ? i guess choosing along which dimension to compare label and prediction?

[Jan]

I meant this: https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto#L1101-L1104

But I realized that it's set to 1 by default anyway, so that shouldn't be an issue. This axis is the axis that is used in the intenal softmax and argmax computation, and it is the axis that 'vanishes' in the result, as the above message explains.

What are the sizes/shapes of the two bottom blobs that are the loss layer's bottoms? They should be printed out at scaffolding time.

Jan

[Jeremy Rutman]

ok your suggestion found my error, in a previous fix-attempt I had recast the labels to 3xWxH   , and I hadn't changed it back.

I think you have put me back on track, thanks.

I incidentally now hit 'Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0)  CUBLAS_STATUS_MAPPING_ERROR' but I think I have gotten past that one before.

[Jeremy Rutman]

the already-well-written error message would be easier to understand if it was written:

Check failed: outer_num_ * inner_num_ == bottom[1]->count() (240000 vs. 720000) Number of predictions must match number of labels; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be N*H*W, with integer values in {0, 1, ..., C-1}.

(I switched the positions of 'predictions' and 'labels' to match the numeric values being shown ; the current miscongruence caused me a bit of grief.)

[Jeremy Rutman]

i now see (after changing the softmax_loss_layer.cpp with a mind to a PR) that the order of predictions/labels may be ambiguous :

  outer_num_ = bottom[0]->count(0, softmax_axis_);
  inner_num_ = bottom[0]->count(softmax_axis_ + 1);
  CHECK_EQ(outer_num_ * inner_num_, bottom[1]->count())

i imagine that if the prototxt lists the incoming bottoms in reverse order, then the bottom[0] and bottom[1] will be reversed, rendering my attempt at clarification  moot.

[Jeremy Rutman]

anyway does anyone have some clew as to why I would be hitting 

 failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0)  CUBLAS_STATUS_MAPPING_ERROR

the label dimensions

I0502 15:16:22.787999 25431 net.cpp:156] Setting up label
I0502 15:16:22.788020 25431 net.cpp:164] Top shape: 1 1 600 400 (240000)

the prediction ('score') dimensions

I0502 15:16:28.884013 25431 net.cpp:434] crop -> score
I0502 15:16:28.884189 25431 net.cpp:156] Setting up crop
I0502 15:16:28.884212 25431 net.cpp:164] Top shape: 1 57 600 400 (13680000)

I0502 15:16:28.884258 25431 net.cpp:111] Creating Layer prob

I0502 15:16:28.884273 25431 net.cpp:478] prob <- score

I0502 15:16:28.884291 25431 net.cpp:478] prob <- label

I0502 15:16:28.884311 25431 net.cpp:434] prob -> loss

[Jeremy Rutman]

Ok the problem there was a little harder to crack.
As it was a cublas error I tried on CPU instead of GPU. Doing so led to 

F0502 15:33:40.381618 25470 softmax_loss_layer.cpp:70] Check failed: label_value < prob_.shape(softmax_axis_) (57 vs. 57)

which made it clear that my 57 categories were 1-indexed (as they came from matlab) instead of 0-indexed.
I heap further curses upon whatever matlab developer came with the 1-index.

[yuca...@gmail.com]

hello, I also meet the error "Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0)  CUBLAS_STATUS_MAPPING_ERROR "  , how do you solve it ?

在 2016年5月2日星期一 UTC+8下午9:45:55，Jeremy Rutman写道：

[Jeremy Rutman]

layer parameter has no field named layer - this is often caused by not closing all your parentheses in the previous layer definition.

cublas_status_success 11 vs 0 - i can't recall the solution to that one, i think it may have been a cuda install problem.