siamese and 2-channel network for image similarity

70 views
Skip to first unread message

Shawn Lee

unread,
Nov 2, 2017, 4:42:17 AM11/2/17
to Caffe Users
Hi everyone,

sorry to bother you...


I tried the default siamese network on Caffe to get two images similarity.
I got nice performance. Accurancy about 0.95.

I tried to train 2channel network for same dataset.
It seems not work.

This is my prototext(modify from cvpr2015\2ch\yosemite_deploy.txt, and i use caffe's hinge loss layer instead of paper's hinge loss function)
name: "2channel"
layer {
  name: "pair_data"
  type: "Data"
  top: "pair_data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    scale: 0.00390625
  }
  data_param {
    source: "examples/images/outputDir_20171025/train/train_leveldb"
    batch_size: 64
  }
}
layer {
  name: "pair_data"
  type: "Data"
  top: "pair_data"
  top: "label"
  include {
    phase: TEST
  }
  transform_param {
    scale: 0.00390625
  }
  data_param {
    source: "examples/images/outputDir_20171025/test/train_leveldb"
    batch_size: 96
  }
}
layer {
  name: "caffe_SpatialConvolution_0"
  type: "Convolution"
  bottom: "pair_data"
  top: "caffe_SpatialConvolution_0"
  convolution_param {
    num_output: 48
    pad_h: 0
    pad_w: 0
    kernel_h: 5
    kernel_w: 5
    stride_h: 2
    stride_w: 2
  }
}
layer {
  name: "caffe_ReLU_1"
  type: "ReLU"
  bottom: "caffe_SpatialConvolution_0"
  top: "caffe_SpatialConvolution_0"
}
layer {
  name: "caffe_Pooling_2"
  type: "Pooling"
  bottom: "caffe_SpatialConvolution_0"
  top: "caffe_Pooling_2"
  pooling_param {
    pool: MAX
    kernel_h: 2
    kernel_w: 2
    stride_h: 2
    stride_w: 2
    pad_h: 0
    pad_w: 0
  }
}
layer {
  name: "caffe_SpatialConvolution_3"
  type: "Convolution"
  bottom: "caffe_Pooling_2"
  top: "caffe_SpatialConvolution_3"
  convolution_param {
    num_output: 86
    pad_h: 0
    pad_w: 0
    kernel_h: 3
    kernel_w: 3
    stride_h: 1
    stride_w: 1
  }
}
layer {
  name: "caffe_ReLU_4"
  type: "ReLU"
  bottom: "caffe_SpatialConvolution_3"
  top: "caffe_SpatialConvolution_3"
}
layer {
  name: "caffe_Pooling_5"
  type: "Pooling"
  bottom: "caffe_SpatialConvolution_3"
  top: "caffe_Pooling_5"
  pooling_param {
    pool: MAX
    kernel_h: 2
    kernel_w: 2
    stride_h: 1
    stride_w: 1
    pad_h: 0
    pad_w: 0
  }
}
layer {
  name: "caffe_SpatialConvolution_6"
  type: "Convolution"
  bottom: "caffe_Pooling_5"
  top: "caffe_SpatialConvolution_6"
  convolution_param {
    num_output: 128
    pad_h: 0
    pad_w: 0
    kernel_h: 5
    kernel_w: 5
    stride_h: 1
    stride_w: 1
  }
}
layer {
  name: "caffe_ReLU_7"
  type: "ReLU"
  bottom: "caffe_SpatialConvolution_6"
  top: "caffe_SpatialConvolution_6"
}
layer {
  name: "caffe_Flatten_8"
  type: "Flatten"
  bottom: "caffe_SpatialConvolution_6"
  top: "caffe_Flatten_8"
}
layer {
  name: "caffe_InnerProduct_9"
  type: "InnerProduct"
  bottom: "caffe_Flatten_8"
  top: "caffe_InnerProduct_9"
  inner_product_param {
    num_output: 2
    axis: -1
  }
}
layer {
  name: "accuracy"
  type: "Accuracy"
  bottom: "caffe_InnerProduct_9"
  bottom: "label"
  top: "accuracy"
  include {
    phase: TEST
  }
}
layer {
  name: "loss"
  type: "HingeLoss"
  bottom: "caffe_InnerProduct_9"
  bottom: "label"
  top: "loss"
  hinge_loss_param {
    norm: L2
  }
}

below show the loss and accuracy 


















what did i do wrong?
shoud i modify the solver parameter or...?
{
base_lr: 0.1
momentum: 0.9
weight_decay: 0.0005
# The learning rate policy
lr_policy: "inv"
gamma: 0.0001
power: 0.75
}

by the way,
I use opencv 3.3 dnn to test the trained model
below is my code
{
int sz[] = { 1, 2, 40, 40 };
cv::Mat blob = Mat(4, sz, CV_32F);

img1.convertTo(img1, CV_32F);
img1 *= 0.00390625f;

img2.convertTo(img2, CV_32F);
img2 *= 0.00390625f;
img1.convertTo(Mat(img1.rows, img1.cols, CV_32F, blob.ptr(0, 0)), CV_32F);
img2.convertTo(Mat(img2.rows, img2.cols, CV_32F, blob.ptr(0, 1)), CV_32F);

//Mat inputBlob = blobFromImage(blob, 0.00390625f, Size(40, 40), Scalar(), false);   //Convert Mat to batch of images
Mat prob;
CV_TRACE_REGION("forward");
net.setInput(blob, "data");
prob = net.forward("caffe_InnerProduct_9");
PrintfMat(prob, "prob");//my function to output mat element
}
however i change any different images to test, the prob always get same values.
prob.at<float>(0,0) always is 1.01
prob.at<float>(0,1) always is 1.01

Did I make the wrong image to test?(about the blob data structure)


Thanks for anyone help.






Reply all
Reply to author
Forward
0 new messages