Generalized dice loss for multi-class segmentation: Caffe implementation

[Shima Masoumi]

Hi I have written a python layer for Generalized dice. 

I modified the code here and add some codes from the link 

Could experts please have a look? I am getting an error that it seems it is related to Python layer. 

import numpy as np
import caffe
"""Usage
# Dice coefficient
layer {
    type: 'Python'
    name: 'dice'
    bottom: 'score_nclasses'
    bottom: 'label'
    top: 'dice'
    python_param {
      module: 'generalized_dice'
      layer: 'GeneralizedDiceLossLayer'
    }
}
"""


class GeneralizedDiceLossLayer(caffe.Layer):
    """
    A Python Layer that calculate the generalized Dice loss based on the number of class
    """
    def setup(self,bottom,top):
        if len(bottom) != 2:
            raise Exception("Need two inputs to compute Dice coefficient. ")
        self.dice=0
        self.sum=0
       
    def reshape(self,bottom,top):
        #check input dimension match
        if bottom[0].count != 5*bottom[1].count:
            raise Exception("Prediction must have 5 elements of the prediction")
        #loss output is scalar
        self.diff=np.zeros_like(bottom[0].data,dtype=np.float32)
        top[0].reshape(1)
       
    def forward(self, bottom, top):
            self.diff[...] = bottom[1].data
            top[0].data[...] = 1 - self.dice_coef_multi_class(bottom[0], bottom[1])


    def backward(self, top, propagate_down,  bottom):
            if propagate_down[1]:
                    raise Exception("label not diff")
            elif propagate_down[0]:
                    a=(-2. * self.diff + self.dice) / self.sum
                    bottom[0].diff[...] = a
            else:
                    raise Exception("no diff")
            # =============================


    def dice_coef_multi_class(self, y_pred, y_true):
            n_classes = 5
            smooth=np.float32(1e-7)
            y_true=y_true.data
            y_pred=y_pred.data
            #y_pred = np.argmax(y_pred, 1)
            #y_pred = np.expand_dims(y_pred,1)
            listLabels=range(n_classes)
            y_pred=self.imageSoftmax(y_pred)
            #print np.shape(y_pred)
            y_pred=np.squeeze(y_pred, axis=0)
            y_pred=self.hotEncodeSegmentation(y_pred, listLabels)


            #y_pred=np.ndarray.flatten(y_pred)
            y_true = np.ndarray.flatten(y_true)


            dice = np.zeros(n_classes)
            self.sum = np.zeros([n_classes])
            for i in range(n_classes):
                    gt=y_pred[i,:,:]
                    gt[gt==1]=i
                    gt_i=np.ndarray.flatten(gt)
                    y_true_i = np.equal(y_true, i)
                    y_pred_i = np.equal(gt_i, i)
                    self.sum[i] = np.sum(y_true_i) + np.sum(y_pred_i) + smooth
                    dice[i] = (2. * np.sum(y_true_i * y_pred_i) + smooth) / self.sum[i]
            self.sum=np.sum(self.sum)
            self.dice=np.sum(dice)
            return self.dice
    def hotEncodeSegmentation(self,segmentation_array, listLabels=[1]):
        """
        Link: https://github.com/gattia/neural_network_medical_image_segmentation/blob/master/hotEncode.py
        """
        #print np.shape(segmentation_array),listLabels
        f=np.shape(segmentation_array)
        outputImageShape = (len(listLabels), f[1], f[2])
        #print f
        segmentation_hotEncoded = np.zeros(outputImageShape)
        index =0
        for label in listLabels:
            locationLabel = np.where(segmentation_array==label)
            segmentation_hotEncoded[index, locationLabel[0], locationLabel[1]]=1
            index +=1
        #background =np.where(segmentation_array==0)
        #segmentation_hotEncoded[background[0], index, background[1], background[2], background[3]]
        return segmentation_hotEncoded
    def imageSoftmax(self,outputSegmentation):
        '''
        This assumes that the outputSegmentation has 5 dimensions:
        dimensions= [diff images, different labels, x-dimension, y-dimension]
        '''
        sigmoid = lambda x: 1/(1+np.exp(-x))
        sigmoid_matrix = sigmoid(outputSegmentation)
        softmax_matrix = sigmoid_matrix / np.sum(sigmoid_matrix, axis=1)
        return(softmax_matrix)
           

    The code is modified for 2D images. I am getting the following error:

I0611 14:44:58.253374 32567 net.cpp:685] Ignoring source layer dropout_d4c
*** Aborted at 1528699498 (unix time) try "date -d @1528699498" if you are using GNU date ***
PC: @     0x7fc5dd7ea73c (unknown)
*** SIGSEGV (@0x38) received by PID 32567 (TID 0x7fc5dd977b00) from PID 56; stack trace: ***
    @     0x7fc5db0004b0 (unknown)
    @     0x7fc5dd7ea73c (unknown)
    @     0x7fc5dd7f3851 (unknown)
    @     0x7fc5dd7ee564 (unknown)
    @     0x7fc5dd7f2da9 (unknown)
    @     0x7fc5db10e5ad (unknown)
    @     0x7fc5dd7ee564 (unknown)
    @     0x7fc5db10e664 __libc_dlopen_mode
    @     0x7fc5db0e0a85 (unknown)
    @     0x7fc5d1d68a99 __pthread_once_slow
    @     0x7fc5db0e0ba4 backtrace
    @     0x7dc57be69643 (unknown)
    @     0x7dc57be69b40 (unknown)
    @     0x7dc57be50c4b (unknown)
    @     0x7fc5db6c26c8 PyNumber_Add
    @     0x7fc5db65650c PyEval_EvalFrameEx
    @     0x7fc5db79305c PyEval_EvalCodeEx
    @     0x7fc5db65bf1d PyEval_EvalFrameEx
    @     0x7fc5db65c044 PyEval_EvalFrameEx
    @     0x7fc5db65c044 PyEval_EvalFrameEx
    @     0x7fc5db79305c PyEval_EvalCodeEx
    @     0x7fc5db6e9370 (unknown)
    @     0x7fc5db6bc273 PyObject_Call
    @     0x7fc5db7303ac (unknown)
    @     0x7fc5db6bc273 PyObject_Call
    @     0x7fc5db792487 PyEval_CallObjectWithKeywords
    @     0x7fc5db6f0fa7 PyEval_CallFunction
    @     0x7dc549dd06d9 boost::python::call<>()
    @     0x7dc549dd0d90 caffe::PythonLayer<>::Forward_cpu()
    @     0x7fc5dcbd3ba7 caffe::Net<>::ForwardFromTo()
    @     0x7fc5dcbd3f17 caffe::Net<>::Forward()
    @     0x7fc5dcb49032 caffe::Solver<>::Test()

[Torsten Straßer]

Hi Shima,

did you have success with this? I could run the layer without the errors you describe. However,

the loss value did not much change from 1.

Thanks

Torsten

[Shima Masoumi]

Did you run my code without any changes?

I could not run it. Could I ask your pycaffe has been built in anaconda environment?

I have build caffe in an anaconda environment and uncomment the following lines in `Makefile.config` :

ANACONDA_HOME := $(HOME)/anaconda2

PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

$(ANACONDA_HOME)/include/python2.7 \

$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

PYTHON_LIB := $(ANACONDA_HOME)/lib

I really could not find any reason.  I would really appreciate if experts help.

Thanks

[torsten....@biomed-engineering.de]

Yes, I could run your code without changes.

However, I don't use anaconda but have caffe and python (2.7.12)  installed in ubuntu.

Unfortunately, I'm not an expert in this.

[wqmgoog]

but the default build python version is 3.6 even in the 2.7 conda env

发自我的小米手机

在 torsten....@biomed-engineering.de，2018年6月30日 00:03写道：

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/c8854589-5522-4fec-8bf0-fe483b90cd65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ali Akbari]

Hi Shima,

Have you find any solution for your problem. I have the same problem. I think the issue comes from incompatibility of a compiler. But I don't have any idea to solve that. I would be so thankful to let me know if you have found any solution for this issue,

[Przemek D]

It's pretty difficult to troubleshoot on a piece of code this complex. Right now it's impossible to say whether the problem is with Python layers in general or just the one you wrote. Could you try to produce a simpler example that would still cause the error? Maybe remove the forward and backward code temporarily, this might help to isolate the issue.

[Shima Masoumi]

Hi Ali,

Sorry for late reply,

This code had problem, I fixed the parts, however, I get loss=1, which I do not know where I am doing mistake. Could you solve your issue? If yes, could you please let me know as well.

Regards

Shima