cgo1.7.1 arm: irregular execution time

[tpoisonooo]

My colleague gives me a `face_recognization.so`, `face_recognization.so` use `libcaffe.so`.

I compiled it with cgo based on arm.

cgo code:

    package algo

    /*

    #include "load_so.h"

    #cgo CFLAGS: -std=gnu99

    #cgo LDFLAGS: -ldl

    #cgo LDFLAGS: -L./ -L/usr/lib -L/usr/local/lib -L/usr/lib -L/usr/local/cuda/lib -L/home/ubuntu/caffe_gx/caffe/.build_release/lib  -lcaffe  -lcudart -lcublas -lcurand -lglog -lgflags -lprotobuf -lboost_system -lboost_filesystem -lm -lhdf5_hl -lhdf5 -lleveldb -lsnappy -llmdb -lopencv_core -lopencv_highgui -lopencv_imgproc -lboost_thread -lstdc++ -lcudnn -lcblas -latlas -lopencv_ml -lopencv_objdetect -lopencv_imgproc -lboost_thread -lstdc++ -lcblas -latlas -lopencv_core -lopencv_highgui -lopencv_imgproc -lboost_thread -lstdc++ -lboost_python  -lboost_filesystem -lboost_system -lboost_thread -lboost_serialization -lopencv_ml  -L./  -lface_recognization

    */

    import "C"

    import "unsafe"

    func ExtractFeature(bytes []byte) ([]float32, error) {

    ...

    errCode := C.extractFeature(pInput, inSize, pOuput, (*C.int)(unsafe.Pointer(&outSize)), (C.int)(mode))

    ...

    }

In load_so.c, I use `dl_open` to load relative function.

    void init(char* pDir){

    soHandle = dlopen("./libface_recognization.so", RTLD_LAZY);

    if (!soHandle) {

    fprintf(stderr, "open so fail: %s\n", dlerror());

    return;

    }

    // feature extraction

        ...

    extract = (fptr_model_extract)dlsym(soHandle, "Face_Feature_Extract");

    if (!extract){

    fprintf(stderr, "dlsym Face_Feature_Extract fail: %s\n", dlerror());

    return;

    }

        ...

    }

        

    int extractFeature(char* input, int inLen, float* output, int* outLen, int mode){

        (*extract)(featureHandle, (const char*)input, inLen, output, outLen, mode);;

    }

Calling relationship:

` func ExtractFeature` in golang --> `extractFeature` in load_so.c --> `Face_Feature_Extract` in face_rec.so --> libcaffe.so

1. In his C++ demo, `Face_Feature_Extract` function always cost 0.1 second. 

2. In my golang code, the `extractFeature`  in load_so.c costs 1.8 or 0.1 second, sometimes the caffe function in libcaffe.so spent 1.8 second, sometimes it is normal.

3. If I loop `extractFeature` function 1000 times in `load_so.c`, average execution time is 0.1 second.

PS:

go env:

    GOARCH="arm"

    GOBIN=""

    GOEXE=""

    GOHOSTARCH="arm"

    GOHOSTOS="linux"

    GOOS="linux"

    GOPATH="/home/ubuntu/GoProjects"

    GORACE=""

    GOROOT="/home/ubuntu/golang/go17armv6l"

    GOTOOLDIR="/home/ubuntu/golang/go17armv6l/pkg/tool/linux_arm"

    CC="gcc"

    GOGCCFLAGS="-fPIC -marm -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build518500973=/tmp/go-build -gno-record-gcc-switches"

    CXX="g++"

    CGO_ENABLED="1"

hardware: 

    nvidia jetson tx1

uname -a: 

    Linux tegra-ubuntu 3.10.96-tegra #1 SMP PREEMPT Tue May 17 16:31:40 PDT 2016 aarch64 aarch64 aarch64 GNU/Linux

I doubt that cgo1.7 is not stable, anybody can help me?

[Konstantin Khomoutov]

On Tue, 18 Oct 2016 22:12:44 -0700 (PDT)
tpoisonooo <tpois...@gmail.com> wrote:

> My colleague gives me a `face_recognization.so`,
> `face_recognization.so` use `libcaffe.so`.
> I compiled it with cgo based on arm.

[...]

> 1. In his C++ demo, `Face_Feature_Extract` function always cost 0.1
> second.
>
> 2. In my golang code, the `extractFeature` in load_so.c costs 1.8 or
> 0.1 second, sometimes the caffe function in libcaffe.so spent 1.8
> second, sometimes it is normal.
>
> 3. If I loop `extractFeature` function 1000 times in `load_so.c`,
> average execution time is 0.1 second.

[...]

> I doubt that cgo1.7 is not stable, anybody can help me?

Does the situation change if, in the goroutine you're measuring your
calls, you do

runtime.LockOSThread()
// cgo calling and measuring code here as it exists now.
defer runtime.UnlockOSThread()

?

The reason I'm asking is that calling C code via cgo from a Go
program works way differently than calling C code from a C program.
In the latter case you just perform "normal" call of a function.
In the former case it looks more like a syscall because the "C side" is
quite alien to the "Go side" (to begin with, they use different stacks).

So the first thing I'd try is to acquire a dedicated OS thread for the
goroutine calling the C side and see if that helps.

[tpoisonooo]

在 2016年10月19日星期三 UTC+8下午9:19:01，Konstantin Khomoutov写道：

 Thanks for your help. It seems that libcaffe.so has thread-restriction. I removed `dlopen`, 

call `C. Face_Feature_Extract ` in main thread, execution time is stable.

I have also checked https://groups.google.com/forum/#!msg/golang-nuts/IiWZ2hUuLDA/SNKYYZBelsYJ.