How to use the 'sigmoid cross entropy loss' in caffe?

Suo Qiu

unread,

May 4, 2015, 8:09:16 AM5/4/15

to caffe...@googlegroups.com

I want to train a convolutional network with three convolutional layers on the CIFAR-10 data set.

I defined the data layer,

layers {
name: "cifar10"
type: DATA
top: "data"
top: "label"
data_param {
source: "cifar10_train_lmdb"
batch_size: 100
backend: LMDB
}
include {
phase: TRAIN
}
transform_param {
mean_file: "mean.binaryproto"
}
}

and the loss layer after the full connected layer,

layers {
name: "ip1"
type: INNER_PRODUCT
bottom: "pool3"
top: "ip1"
blobs_lr: 1.
blobs_lr: 2.
inner_product_param {
num_output: 10 #ten classes
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "softmax"
type: SOFTMAX
bottom: "ip1"
top: "soft1"
}
layer {
name: "loss"
type: SIGMOID_CROSS_ENTROPY_LOSS
bottom: "soft1"
bottom: "label"
top: "loss"
}

This definition triggered an error that, the count of the bottoms should be same. The shape of "soft1" was N * c * 1 * 1. The shape of "label" was N * 1 * 1 * 1.

How to fix this problem? Any suggestions?

James Guo

unread,

May 4, 2015, 8:39:50 AM5/4/15

to caffe...@googlegroups.com

I think this is as the error suggested, the softmax will give you the probability of each class, thus it's N * c * 1 * 1, N is the batch size and c is the number of classes, if I understand correctly. But your label only have N * 1 * 1* 1, maybe there is some problem with your label??

Swami

unread,

Jun 29, 2015, 1:03:05 AM6/29/15

to caffe...@googlegroups.com

@Suo: Were you able to solve this problem ?

@James Guo: label is supposed to be N*1*1*1 ie one label per sample which indicates which class it belongs to.

Any update on this will be highly appreciated.

Bob Zigon

unread,

Aug 12, 2015, 3:25:16 PM8/12/15

to Caffe Users

@Suo : Were you able to solve this problem? I have the same problem.

--Bob

Shravan t r

unread,

Oct 29, 2015, 5:23:55 AM10/29/15

to Caffe Users

The problem is with the Label file. With Sigmoid Cross entropy, the number of labels per image should be same as the number of outputs pf the Inner product. Thus labels should have N x c x 1 x 1.

On Monday, 4 May 2015 14:09:16 UTC+2, Suo Qiu wrote:

Zhenjie Zhang

unread,

Nov 13, 2015, 4:36:03 AM11/13/15

to Caffe Users

guess you have only one number for each label, that might be the problem. For 10 categories, you need to use a tuple of size 10, for each label, instead of using a numerical number. Example, 2 should be translated to 0010000000, 4 should be 0000100000. et al... This way, you will have Nc11 dimension for your label blob.

Zhenjie Zhang

unread,

Nov 13, 2015, 4:44:47 AM11/13/15

to Caffe Users

there are other problems too. the input to your sigmoid_cross_entropy should be ip1, not soft1. And, you should not be using sigmoid_cross_entropy in your case, anyways, since you are dealing with a multi-class problem, and should not be using binary cross entropy implemented in sigmoid cross entropy. A SoftmaxWithLoss is more appropriate.

Afromeck Afromeck

unread,

Jan 6, 2016, 4:36:58 PM1/6/16

to Caffe Users

If I set those transformation you mentioned, how can I handle the fact that convert_imageset.cpp is waiting for an integer in label position and 00010000 for example is not?

Reply all

Reply to author

Forward