Compiling a custom layer in latest Caffe

[Ankit Dhall]

Hello,

I am trying to port a custom layer to a newer version (the one with header files for each layer) of Caffe from an older one (the one with common_layer.hpp). The custom layer is only for CPU and doesn't have a .cu implementation. I have created a new .hpp file for the custom layer. However, I get an error while running the tests and also while running a network with the custom layer in it.

Here's the output of the terminal,

.build_debug/test/test_all.testbin --gtest_filter=*Gating*

Cuda number of devices: 4

Current device id: 0

Current device name: GeForce GTX TITAN X

Note: Google Test filter = *Gating*

[==========] Running 12 tests from 4 test cases.

[----------] Global test environment set-up.

[----------] 3 tests from GatingInnerProductLayerTest/0, where TypeParam = caffe::CPUDevice<float>

[ RUN      ] GatingInnerProductLayerTest/0.TestSetUp

[       OK ] GatingInnerProductLayerTest/0.TestSetUp (438 ms)

[ RUN      ] GatingInnerProductLayerTest/0.TestForward

[       OK ] GatingInnerProductLayerTest/0.TestForward (0 ms)

[ RUN      ] GatingInnerProductLayerTest/0.TestGradient

GatingInnerProductLayer Backward bottom_backward diff i0 : 0.59375

GatingInnerProductLayer Backward bottom_backward diff i1 : 0.4625

GatingInnerProductLayer Backward bottom_backward diff i2 : 0.3875

GatingInnerProductLayer Backward bottom_backward diff i3 : 0.616

GatingInnerProductLayer Backward bottom_backward diff i4 : 0.268

GatingInnerProductLayer Backward bottom_backward diff i5 : 0.471

Passed backward test

[       OK ] GatingInnerProductLayerTest/0.TestGradient (1 ms)

[----------] 3 tests from GatingInnerProductLayerTest/0 (439 ms total)

[----------] 3 tests from GatingInnerProductLayerTest/1, where TypeParam = caffe::CPUDevice<double>

[ RUN      ] GatingInnerProductLayerTest/1.TestSetUp

[       OK ] GatingInnerProductLayerTest/1.TestSetUp (0 ms)

[ RUN      ] GatingInnerProductLayerTest/1.TestForward

[       OK ] GatingInnerProductLayerTest/1.TestForward (0 ms)

[ RUN      ] GatingInnerProductLayerTest/1.TestGradient

GatingInnerProductLayer Backward bottom_backward diff i0 : 0.59375

GatingInnerProductLayer Backward bottom_backward diff i1 : 0.4625

GatingInnerProductLayer Backward bottom_backward diff i2 : 0.3875

GatingInnerProductLayer Backward bottom_backward diff i3 : 0.616

GatingInnerProductLayer Backward bottom_backward diff i4 : 0.268

GatingInnerProductLayer Backward bottom_backward diff i5 : 0.471

Passed backward test

[       OK ] GatingInnerProductLayerTest/1.TestGradient (1 ms)

[----------] 3 tests from GatingInnerProductLayerTest/1 (1 ms total)

[----------] 3 tests from GatingInnerProductLayerTest/2, where TypeParam = caffe::GPUDevice<float>

[ RUN      ] GatingInnerProductLayerTest/2.TestSetUp

[       OK ] GatingInnerProductLayerTest/2.TestSetUp (80 ms)

[ RUN      ] GatingInnerProductLayerTest/2.TestForward

*** Error in `.build_debug/test/test_all.testbin': free(): invalid pointer: 0x0000000204502600 ***

*** Aborted at 1468510396 (unix time) try "date -d @1468510396" if you are using GNU date ***

PC: @     0x7f16fdca9c37 (unknown)

*** SIGABRT (@0x35460000236e) received by PID 9070 (TID 0x7f1705005a40) from PID 9070; stack trace: ***

    @     0x7f16fe048330 (unknown)

    @     0x7f16fdca9c37 (unknown)

    @     0x7f16fdcad028 (unknown)

    @     0x7f16fdce62a4 (unknown)

    @     0x7f16fdcf255e (unknown)

    @     0x7f16feb8129a caffe::GatingInnerProductLayer<>::Forward_cpu()

    @           0x4c4751 caffe::Layer<>::Forward_gpu()

    @           0x4c2646 caffe::Layer<>::Forward()

    @           0x5191da caffe::GatingInnerProductLayerTest_TestForward_Test<>::TestBody()

    @           0x964c8b testing::internal::HandleSehExceptionsInMethodIfSupported<>()

    @           0x9603d4 testing::internal::HandleExceptionsInMethodIfSupported<>()

    @           0x94d7eb testing::Test::Run()

    @           0x94df84 testing::TestInfo::Run()

    @           0x94e572 testing::TestCase::Run()

    @           0x9533fc testing::internal::UnitTestImpl::RunAllTests()

    @           0x965cbe testing::internal::HandleSehExceptionsInMethodIfSupported<>()

    @           0x960fff testing::internal::HandleExceptionsInMethodIfSupported<>()

    @           0x952190 testing::UnitTest::Run()

    @           0x4b455d main

    @     0x7f16fdc94f45 (unknown)

    @           0x4b4329 (unknown)

    @                0x0 (unknown)

Aborted (core dumped)

 While running any file with the custom layer, I get a similar error that says

'free(): invalid pointer'

When I use the same custom layer on the older version of Caffe, it runs fine, the tests as well as a network that has the layer.

Since the GPU test fails (to fall back to the CPU version), I think there's something I'm missing in terms of specifying to Caffe about the layer having only a .cpp implementation and not a .cu version for the layer.

Any suggestions would be helpful.

Regards,

Ankit