Skip to first unread message
Joaquim Fèlix Martínez Artigot
Apr 19, 2018, 2:30:57 PM4/19/18
to Caffe Users
I'm trying to finetune Girshick's Faster RCNN to detect apples (only 2 classes, apple and background, althought I don't have negative examples) in canopies. When training my accuracy at Iteration=0 is 0.279412 but after that it's always 1.
I attached my train, test and solver prototxt.
I've read other cases of this Accuracy = 1 error but the error in those cases was that the annotations were all negative examples so it wasn't learning anything. I'm not sure the error may come from loading my annotations. I printed the output of load annotations to see the values and check if they were right, but maybe I'm missing something
def _load_annotation(self, index):
print "I reached load_annotation"
filename = os.path.join(self._data_path, 'Annotations', index + '.txt')
print "Loading file"
print filename
with open(filename) as f:
data = f.readlines()
num_objs = len(data)
print "num_objs is"
print num_objs
boxes = np.zeros((num_objs, 4), dtype=np.uint16)
gt_classes = np.zeros((num_objs), dtype=np.int32)
overlaps = np.zeros((num_objs, self.num_classes), dtype=np.float32)
# "Seg" area here is just the box area
#seg_areas = np.zeros((num_objs), dtype=np.float32)
# Load object bounding boxes into a data frame.
#for ix, obj in enumerate(objs):
for ix, aline in enumerate(data):
tokens = aline.strip().split()
print "len(tokens) is:"
print len(tokens)
if len(tokens) != 4:
continue
cls = 1 ## this file uses 0 as the background
x1 = float(tokens[0])
y1 = float(tokens[1])
x2 = float(tokens[2])
y2 = float(tokens[3])
gt_classes[ix] = cls
boxes[ix, :] = [x1, y1, x2, y2]
print "ix and cls are"
print ix
print cls
overlaps[ix, cls] = 1.0 #just one class, apple
overlaps = scipy.sparse.csr_matrix(overlaps)
print "boxes"
print "gt_classes"
print "overlaps"
print boxes
print gt_classes
print overlaps
return {'boxes' : boxes,
'gt_classes': gt_classes,
'gt_overlaps' : overlaps,
'flipped' : False}
#'seg_areas' : seg_areas}
And this is one example of loading the annotations
boxes
gt_classes
overlaps
[[1727 716 1825 814]
[1410 784 1508 883]
[1643 895 1715 967]
[1700 790 1758 848]
[1554 847 1607 900]
[1190 983 1259 1052]
[1505 977 1574 1047]
[1463 933 1532 1003]
[1437 866 1520 950]
[1720 1148 1804 1232]
[1800 1197 1892 1289]
[1392 1397 1466 1471]
[1220 1125 1309 1214]
[1109 1084 1199 1173]
[1298 1131 1387 1220]
[1420 1215 1485 1280]
[1110 1228 1206 1323]
[1243 1435 1327 1518]
[ 77 1392 145 1459]
[ 131 1418 192 1479]
[ 74 1207 155 1287]
[ 302 1082 391 1171]
[ 333 995 413 1075]
[ 142 1114 222 1194]
[ 152 1086 232 1166]
[ 100 740 180 820]
[ 546 689 618 761]
[ 557 732 629 804]
[ 639 914 718 993]
[ 804 775 875 846]
[1704 1107 1775 1179]
[1001 773 1065 837]
[1114 804 1185 874]
[ 856 988 926 1058]
[ 871 901 934 964]
[ 950 1061 1007 1118]
[ 793 1211 862 1280]
[ 796 1364 865 1433]
[ 487 1317 563 1393]
[ 630 1439 706 1515]
[ 719 1454 795 1530]
[ 987 1280 1055 1349]
[ 957 1424 1019 1486]
[ 565 1263 639 1338]
[1545 1132 1612 1199]
[1589 891 1656 958]
[1498 746 1571 820]]
[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1]
(0, 1) 1.0
(1, 1) 1.0
(2, 1) 1.0
(3, 1) 1.0
(4, 1) 1.0
(5, 1) 1.0
(6, 1) 1.0
(7, 1) 1.0
(8, 1) 1.0
(9, 1) 1.0
(10, 1) 1.0
(11, 1) 1.0
(12, 1) 1.0
(13, 1) 1.0
(14, 1) 1.0
(15, 1) 1.0
(16, 1) 1.0
(17, 1) 1.0
(18, 1) 1.0
(19, 1) 1.0
(20, 1) 1.0
(21, 1) 1.0
(22, 1) 1.0
(23, 1) 1.0
(24, 1) 1.0
(25, 1) 1.0
(26, 1) 1.0
(27, 1) 1.0
(28, 1) 1.0
(29, 1) 1.0
(30, 1) 1.0
(31, 1) 1.0
(32, 1) 1.0
(33, 1) 1.0
(34, 1) 1.0
(35, 1) 1.0
(36, 1) 1.0
(37, 1) 1.0
(38, 1) 1.0
(39, 1) 1.0
(40, 1) 1.0
(41, 1) 1.0
(42, 1) 1.0
(43, 1) 1.0
(44, 1) 1.0
(45, 1) 1.0
(46, 1) 1.0
I0419 19:46:15.321749 5623 solver.cpp:406] Test net output #0: bbox_pred = 4.47311e-05
I0419 19:46:15.321836 5623 solver.cpp:406] Test net output #1: bbox_pred = -2.32641e-05
I0419 19:46:15.321872 5623 solver.cpp:406] Test net output #2: bbox_pred = -2.33807e-05
I0419 19:46:15.321902 5623 solver.cpp:406] Test net output #3: bbox_pred = -8.21502e-05
I0419 19:46:15.321933 5623 solver.cpp:406] Test net output #4: bbox_pred = -3.70282e-05
I0419 19:46:15.321960 5623 solver.cpp:406] Test net output #5: bbox_pred = 0.000118738
I0419 19:46:15.321990 5623 solver.cpp:406] Test net output #6: bbox_pred = -1.77449e-05
I0419 19:46:15.322018 5623 solver.cpp:406] Test net output #7: bbox_pred = -4.10721e-05
I0419 19:46:15.322046 5623 solver.cpp:406] Test net output #8: cls_prob = 0.500026
I0419 19:46:15.322074 5623 solver.cpp:406] Test net output #9: cls_prob = 0.499974
I0419 19:46:16.097544 5623 solver.cpp:229] Iteration 0, loss = 4.55499
I0419 19:46:16.097707 5623 solver.cpp:245] Train net output #0: accuracy = 0.279412
I0419 19:46:16.097754 5623 solver.cpp:245] Train net output #1: bbox_loss = 0.00223378 (* 1 = 0.00223378 loss)
I0419 19:46:16.097791 5623 solver.cpp:245] Train net output #2: cls_loss = 0.778582 (* 1 = 0.778582 loss)
I0419 19:46:16.097826 5623 solver.cpp:245] Train net output #3: rpn_cls_loss = 0.715126 (* 1 = 0.715126 loss)
I0419 19:46:16.097858 5623 solver.cpp:245] Train net output #4: rpn_loss_bbox = 1.59839 (* 1 = 1.59839 loss)
I0419 19:46:16.097892 5623 sgd_solver.cpp:106] Iteration 0, lr = 0.001
I0419 19:46:25.594306 5623 solver.cpp:338] Iteration 20, Testing net (#0)
I0419 19:46:25.594491 5623 net.cpp:748] Ignoring source layer input-data
I0419 19:46:25.594498 5623 net.cpp:748] Ignoring source layer data_input-data_0_split
I0419 19:46:25.594502 5623 net.cpp:748] Ignoring source layer im_info_input-data_1_split
I0419 19:46:25.594506 5623 net.cpp:748] Ignoring source layer gt_boxes_input-data_2_split
I0419 19:46:25.594516 5623 net.cpp:748] Ignoring source layer rpn_cls_score_rpn_cls_score_0_split
I0419 19:46:25.594521 5623 net.cpp:748] Ignoring source layer rpn_bbox_pred_rpn_bbox_pred_0_split
I0419 19:46:25.594524 5623 net.cpp:748] Ignoring source layer rpn_cls_score_reshape_rpn_cls_score_reshape_0_split
I0419 19:46:25.594527 5623 net.cpp:748] Ignoring source layer rpn-data
I0419 19:46:25.594532 5623 net.cpp:748] Ignoring source layer rpn_loss_cls
I0419 19:46:25.594534 5623 net.cpp:748] Ignoring source layer rpn_loss_bbox
I0419 19:46:25.594538 5623 net.cpp:748] Ignoring source layer roi-data
I0419 19:46:25.594542 5623 net.cpp:748] Ignoring source layer labels_roi-data_1_split
I0419 19:46:25.594547 5623 net.cpp:748] Ignoring source layer cls_score_cls_score_0_split
I0419 19:46:25.594552 5623 net.cpp:748] Ignoring source layer loss_cls
I0419 19:46:25.594554 5623 net.cpp:748] Ignoring source layer loss_bbox
I0419 19:46:25.594558 5623 net.cpp:748] Ignoring source layer accuracy
I0419 19:46:25.707377 5623 solver.cpp:406] Test net output #0: bbox_pred = 4.34616e-05
I0419 19:46:25.707445 5623 solver.cpp:406] Test net output #1: bbox_pred = -3.22815e-05
I0419 19:46:25.707482 5623 solver.cpp:406] Test net output #2: bbox_pred = -4.15669e-05
I0419 19:46:25.707525 5623 solver.cpp:406] Test net output #3: bbox_pred = -8.5121e-05
I0419 19:46:25.707556 5623 solver.cpp:406] Test net output #4: bbox_pred = 0.0004886
I0419 19:46:25.707587 5623 solver.cpp:406] Test net output #5: bbox_pred = -0.00157662
I0419 19:46:25.707617 5623 solver.cpp:406] Test net output #6: bbox_pred = 0.000241718
I0419 19:46:25.707646 5623 solver.cpp:406] Test net output #7: bbox_pred = -0.0011017
I0419 19:46:25.707676 5623 solver.cpp:406] Test net output #8: cls_prob = 0.467187
I0419 19:46:25.707705 5623 solver.cpp:406] Test net output #9: cls_prob = 0.532813
I0419 19:46:26.272559 5623 solver.cpp:229] Iteration 20, loss = 4.68815
I0419 19:46:26.272698 5623 solver.cpp:245] Train net output #0: accuracy = 1
I0419 19:46:26.272747 5623 solver.cpp:245] Train net output #1: bbox_loss = 0.555879 (* 1 = 0.555879 loss)
I0419 19:46:26.272784 5623 solver.cpp:245] Train net output #2: cls_loss = 0.217105 (* 1 = 0.217105 loss)
I0419 19:46:26.272819 5623 solver.cpp:245] Train net output #3: rpn_cls_loss = 0.0757147 (* 1 = 0.0757147 loss)
I0419 19:46:26.272855 5623 solver.cpp:245] Train net output #4: rpn_loss_bbox = 3.27694 (* 1 = 3.27694 loss)
I0419 19:46:26.272886 5623 sgd_solver.cpp:106] Iteration 20, lr = 0.001
/imatge/jfmartinez/py-faster-rcnn-2/py-faster-rcnn/tools/../lib/fast_rcnn/bbox_transform.py:24: RuntimeWarning: invalid value encountered in log
targets_dh = np.log(gt_heights / ex_heights)
I0419 19:46:35.626662 5623 solver.cpp:338] Iteration 40, Testing net (#0)
I0419 19:46:35.626799 5623 net.cpp:748] Ignoring source layer input-data
I0419 19:46:35.626832 5623 net.cpp:748] Ignoring source layer data_input-data_0_split
I0419 19:46:35.626858 5623 net.cpp:748] Ignoring source layer im_info_input-data_1_split
I0419 19:46:35.626883 5623 net.cpp:748] Ignoring source layer gt_boxes_input-data_2_split
I0419 19:46:35.626914 5623 net.cpp:748] Ignoring source layer rpn_cls_score_rpn_cls_score_0_split
I0419 19:46:35.626945 5623 net.cpp:748] Ignoring source layer rpn_bbox_pred_rpn_bbox_pred_0_split
I0419 19:46:35.626969 5623 net.cpp:748] Ignoring source layer rpn_cls_score_reshape_rpn_cls_score_reshape_0_split
I0419 19:46:35.626993 5623 net.cpp:748] Ignoring source layer rpn-data
I0419 19:46:35.627015 5623 net.cpp:748] Ignoring source layer rpn_loss_cls
I0419 19:46:35.627040 5623 net.cpp:748] Ignoring source layer rpn_loss_bbox
I0419 19:46:35.627064 5623 net.cpp:748] Ignoring source layer roi-data
I0419 19:46:35.627087 5623 net.cpp:748] Ignoring source layer labels_roi-data_1_split
I0419 19:46:35.627116 5623 net.cpp:748] Ignoring source layer cls_score_cls_score_0_split
I0419 19:46:35.627141 5623 net.cpp:748] Ignoring source layer loss_cls
I0419 19:46:35.627167 5623 net.cpp:748] Ignoring source layer loss_bbox
I0419 19:46:35.627192 5623 net.cpp:748] Ignoring source layer accuracy
I0419 19:46:35.740416 5623 solver.cpp:406] Test net output #0: bbox_pred = 3.82229e-05
I0419 19:46:35.740546 5623 solver.cpp:406] Test net output #1: bbox_pred = -5.48789e-05
I0419 19:46:35.740584 5623 solver.cpp:406] Test net output #2: bbox_pred = -5.60168e-05
I0419 19:46:35.740614 5623 solver.cpp:406] Test net output #3: bbox_pred = -6.57868e-05
I0419 19:46:35.740643 5623 solver.cpp:406] Test net output #4: bbox_pred = 0.00189189
I0419 19:46:35.740672 5623 solver.cpp:406] Test net output #5: bbox_pred = -0.00213907
I0419 19:46:35.740702 5623 solver.cpp:406] Test net output #6: bbox_pred = 0.00390347
I0419 19:46:35.740731 5623 solver.cpp:406] Test net output #7: bbox_pred = -0.0030234
I0419 19:46:35.740761 5623 solver.cpp:406] Test net output #8: cls_prob = 0.445186
I0419 19:46:35.740798 5623 solver.cpp:406] Test net output #9: cls_prob = 0.554814
I0419 19:46:36.252399 5623 solver.cpp:229] Iteration 40, loss = 3.28001
I0419 19:46:36.252523 5623 solver.cpp:245] Train net output #0: accuracy = 1
I0419 19:46:36.252569 5623 solver.cpp:245] Train net output #1: bbox_loss = 0.702367 (* 1 = 0.702367 loss)
I0419 19:46:36.252605 5623 solver.cpp:245] Train net output #2: cls_loss = 0.152012 (* 1 = 0.152012 loss)
I0419 19:46:36.252640 5623 solver.cpp:245] Train net output #3: rpn_cls_loss = 0.0351711 (* 1 = 0.0351711 loss)
I0419 19:46:36.252673 5623 solver.cpp:245] Train net output #4: rpn_loss_bbox = 2.84947 (* 1 = 2.84947 loss)
I0419 19:46:36.252704 5623 sgd_solver.cpp:106] Iteration 40, lr = 0.001
If there's anything else you need to see let me know and I'll share with you as soon as possible, thanks
Xun Victor
Apr 20, 2018, 5:17:26 AM4/20/18
to Caffe Users
Hi Joaquim,
Looking at your gt_classes, I see that you only have "1" (apple) in your training data. Because of that, the network learns to output "1" every time as it never sees examples from the background class.
Then, accuracy is computed on test data which is presumably also only constituted of apple classes. So the network is always correct.
You should add background training examples on both your training and validation/test set.
Also, accuracy is not 1 after 20 iterations, but rather straight away after the first forward at iteration=1. The reason you only see it at iteration 20 is that you specified display: 20 in solver.prototxt
Joaquim Fèlix Martínez Artigot
Apr 20, 2018, 6:03:27 PM4/20/18
to Caffe Users
Hi Xun Victor
first of all thanks for your answer.
I added images without apples to the training and test sets. But I'm encountering an error when loading the annotations for these images. What I did was add 4 annotations per image, each being one quarter of the picture, to have 4 annotations of negative examples per image to reduce number of images. When the annotations are loaded, as I set cls to 0 for negative examples and 1 for apples, at roidb.py this line creates an assertion error since it finds cls = 0
assert all(max_classes[nonzero_inds] != 0)
What I tried to do then was train the net with the images and keep the annotation files empty, but that kept the accuracy = 1 issue
Reply all
Reply to author
Forward
0 new messages