Aborted at 1475952725 (unix time) try "date -d @1475952725" if you are using GNU date

[Sharp Weapon]

I am training a simple network. Having trouble to get caffe run, I decided to test run on 20 images only. But I can't get past this following error message. I have rebuilt caffe as suggested by other posts but didn't solve the issue.

13:52:00.583408 45606 sgd_solver.cpp:106] Iteration 7, lr = 0.01
I1008 13:52:00.905189 45606 solver.cpp:228] Iteration 8, loss = 87.3365
I1008 13:52:00.905206 45606 solver.cpp:244]     Train net output #0: loss = 87.3365 (* 1 = 87.3365 loss)
I1008 13:52:00.905238 45606 sgd_solver.cpp:106] Iteration 8, lr = 0.01
I1008 13:52:01.227761 45606 solver.cpp:228] Iteration 9, loss = 87.3365
I1008 13:52:01.227777 45606 solver.cpp:244]     Train net output #0: loss = 87.3365 (* 1 = 87.3365 loss)
I1008 13:52:01.227784 45606 sgd_solver.cpp:106] Iteration 9, lr = 0.01
I1008 13:52:01.227901 45606 solver.cpp:454] Snapshotting to binary proto file _iter_10.caffemodel
*** Aborted at 1475952725 (unix time) try "date -d @1475952725" if you are using GNU date ***
PC: @     0x7f5e0130768c caffe::BlobProto::SerializeWithCachedSizesToArray()
*** SIGSEGV (@0xd70e000) received by PID 45606 (TID 0x7f5e01e0ea00) from PID 225501184; stack trace: ***
    @     0x7f5df32c98d0 (unknown)
    @     0x7f5e0130768c caffe::BlobProto::SerializeWithCachedSizesToArray()
    @     0x7f5e0130d13f caffe::LayerParameter::SerializeWithCachedSizesToArray()
    @     0x7f5e0130f8d7 caffe::NetParameter::SerializeWithCachedSizesToArray()
    @     0x7f5dfb6fd58a (unknown)
    @     0x7f5dfb6fd655 (unknown)
    @     0x7f5dfb6fd7bf (unknown)
    @     0x7f5dfb76815b (unknown)
    @     0x7f5e01389803 caffe::WriteProtoToBinaryFile()
    @     0x7f5e013a1a82 caffe::Solver<>::SnapshotToBinaryProto()
    @     0x7f5e013a1b6f caffe::Solver<>::Snapshot()
    @     0x7f5e013a3219 caffe::Solver<>::Step()
    @     0x7f5e013a34a9 caffe::Solver<>::Solve()
    @           0x409426 train()
    @           0x405c83 main
    @     0x7f5df2f30b45 (unknown)
    @           0x406565 (unknown)
    @                0x0 (unknown)
*** Error in `caffe': malloc(): memory corruption: 0x000000000d4ceac0 ***

I have a feeling that it is caused by my solver file. Here is the my solver.


net: "/X/train.prototxt"
test_iter: 5
test_interval: 5
base_lr: 0.01
momentum: 0.9
weight_decay: 0.0005
lr_policy: "step"
stepsize: 5
gamma: 0.1
power: 0.75
display: 5
max_iter: 20
snapshot: 10
snapshot_prefix: "/X/A"
solver_mode: CPU

Do you see any issue on my solver?

Cheers,