When testing Fully convolutional networks on pascal voc 2011, I cann't get the reported result.

[BenG]

There's an error with the last post, so I post it again. In urgent need for help. 

I'm testing the caffemodel of FCN32s on pascal voc 2011 data, and the accuracy I got is not the reported mIU 64.0 as the author reported, much higher, 

I must have done something wrong, here's my code for testing, can any who has successfully tested it help me with it ? 

I got mIU about 72. I don't know why, there must be something wrong in my code. 

The val set is in "val.txt"

Below is the code for testing:

net = caffe.Net("deploy.prototxt", "fcn-32s-pascal.caffemodel", caffe.TEST)

files = []

with open("val.txt","r") as f:

    for line in f.readlines():

        line = line.strip("\n")

        files.append(line)

f.close()

colors = []

with open("color.txt", "r") as cf:

    for line in cf.readlines():

        line = line.strip("\n")

        tmp = line.split()

        colors.append((int(tmp[3]), int(tmp[2]), int(tmp[1])) )

cf.close()

bound = (192,224,224)

sum = 0.

ct=0 # counter 

for l in files:

    ct += 1

    print ct, l

    img = cv2.imread("data/images/%s.jpg"%(str(l)))

    in_ = np.array(img, dtype=np.float32)

    in_ -= np.array((104.00698793,116.66876762,122.67891434))

    in_ = in_.transpose((2,0,1))

    net.blobs['data'].reshape(1, *in_.shape)

    net.blobs['data'].data[...] = in_

    net.forward()

    out = net.blobs['upscore'].data[0].argmax(axis=0)

    

    mask = cv2.imread("data/mask/%s.png"%(l))

    label = np.zeros(mask[:,:,0].shape)

    obj = []

    value = 0.

    num = 0

    for i in range(len(colors)):

        eq = np.equal(mask, colors[i])

        eq = np.prod(eq, 2).astype(int)

        if (eq > 0).any():

            num += 1

            label += ( i+1 ) * eq

            obj.append(i+1)

    if num == 0:

        raise

    eq = np.equal(mask, bound)

    eq = np.prod(eq, 2).astype(int)

    label[np.nonzero(eq > 0)] = 255

    out[np.nonzero(eq > 0)] = 255

    

    # including background 

    if(label == 0).any():

        obj.append(0)

        num += 1

    for idx in obj:

        count1 = np.count_nonzero( (out == idx) & (label == idx))

        count2 = np.count_nonzero( (out == idx) | (label==idx))

        print count1, count2

        value += float(count1)/count2 

    value = value/num

    sum += value

        

print 1111, sum/1111

[Evan Shelhamer]

See reply at https://groups.google.com/d/msg/caffe-users/9qNggEa8EaQ/HsDEEx-gCgAJ

Evan Shelhamer

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/efb6d8c4-05f2-42d8-b76a-62bd16aa2185%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yang mk]

do you have save this question? I have the same question. Thank you.

在 2015年8月4日星期二 UTC+8上午4:42:57，Evan Shelhamer写道：

[Steve Bengi]

what question? Can you present your problem in details ? 

[Yang mk]

I am so sorry. Here is my questiones in details: question. thank you

在 2016年1月25日星期一 UTC+8下午7:25:58，Ben写道：

[Steve Bengi]

You don't have to resize images and groundtruths to 500 by 500, it's ok if you just input it as it is. For example, if you input an image of size 300 by 400, you will get the prediction with same size 300 by 400. Then you directly evaluate this prediction. No resizing is needed. 

--
You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/T_U11C5joIQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.

To post to this group, send email to caffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/64e65c37-37c7-471f-9b53-48ea1331efd3%40googlegroups.com.

[Yang mk]

yes, I tried it. As you can see, I tried it in my third way. But I got 64.9 mean I/U. I don't konw why. And I test it on seg11valid. 

can you tell me your email? Thank you very much.

在 2016年1月28日星期四 UTC+8下午4:44:09，Ben写道：

[Ben]

sorry, I didn't notice the third scheme, probably you should carefully check your evaluation code.

[Yang mk]

do you get the gaven mean I/U? and as you mentioned above, you got 72 mean I/U, can you tell me how to solve this question, please?

在 2016年1月28日星期四 UTC+8下午8:01:24，Ben写道：