How to use jl_call1 with a struct argument?

[K leo]

I tried the following, it compiles OK and runs OK, but it appears the julia function is not called (because there is no output from the println statement).  What is wrong?

#include <julia.h>
#include <iostream>
using namespace std;
struct TestType {
    double a;
    double b;
};
int main(int argc, char *argv[])
{
    TestType A, *ret;
    A.a=2.;
    A.b=3.;
    jl_init(NULL);
    jl_load("TestArg.jl");
    jl_value_t * mod = (jl_value_t*)jl_eval_string("TestArg");
    jl_function_t * func = jl_get_function((jl_module_t*)mod,"TestFunc");
    jl_call1(func, (jl_value_t *)&A);
    jl_atexit_hook(0);
    return 0;
}
# TestArg.jl
module TestArg
type TestType
    a::Float64
    b::Float64
end
function TestFunc(data::TestType)
    println("in TestArg.TestFunc ")
end
end

[Isaiah Norton]

In c, 'struct TestType' is plain old data, not a Julia type (jl_value_t). Either change to an immutable on julia side, and change function signature to  'Ptr{TestType}'; or allocate a 'jl_value_t*' with the correct type tag, and set the fields.

[K leo]

Isaiah,

Thanks for the reply.

I tried your advice A, i.e. "change to an immutable on julia side, and change function signature to  'Ptr{TestType}'", and the code behaves the same as before, i.e. it compiles OK and runs OK but does not show the output of the println in the function.

I guess at the moment what puzzles me is not whether the data get passed correctly (because I am not trying to access the data yet), but why the one single statement in the Julia function is not executed.  That function appears to be taken correctly because if I change the function name in Julia then there is a core dump.  Any thoughts?

I have not yet tried your advice B as I am not too clear about it.

[Isaiah Norton]

(Disregard my suggestion about `Ptr{TestType}`, `jl_call1` only takes boxed arguments so that won't work)

The problem is that the passed argument type does not match any available method (because it isn't actually passed a julia type at all) so `jl_call1` silently gives up. The `jl_call*` methods are in TRY/CATCH blocks to prevent spilling errors across the API, and they do not print error messages.

The following example works, it's not really efficient but it does demonstrate type construction from C:

#include <julia.h>

#include <iostream>

using namespace std;

struct TestType {

    double a;

    double b;

};

int main(int argc, char *argv[])

{

    TestType A, *ret;

    A.a=2.;

    A.b=3.;

    jl_init("/Users/inorton/git/jl71/usr/lib/");

    jl_load("TestArg.jl");

    jl_value_t * mod = (jl_value_t*)jl_eval_string("TestArg");

   // construct a TestType instance

    jl_value_t* jl_A = jl_new_struct((jl_datatype_t*)jl_get_function((jl_module_t*)mod, "TestType"),

                                     jl_box_float64(A.a), jl_box_float64(A.b));

    jl_function_t * func = jl_get_function((jl_module_t*)mod,"TestFunc");

    jl_call1(func, jl_A);

    jl_atexit_hook(0);

    return 0;

}

Using the embedding API requires a substantial depth of understanding of , including stack vs heap and argument passing, as well as dynamic language concepts such as boxing, and rooting. The embedding section of the manual is designed to provide an overview, but ultimately you are going to have to read and grok the Julia source to get every little detail.

[Bart Janssens]

On Fri, Sep 9, 2016 at 6:44 PM Isaiah Norton <isaiah...@gmail.com> wrote:

   // construct a TestType instance

    jl_value_t* jl_A = jl_new_struct((jl_datatype_t*)jl_get_function((jl_module_t*)mod, "TestType"),

                                     jl_box_float64(A.a), jl_box_float64(A.b));

The arguments to jl_new_struct need to be rooted, so you should do something like:

jl_value_t* a = NULL;
jl_value_t* b = NULL;
JL_GC_PUSH2(a,b);

a = jl_box_float64(A.a);
b = jl_box_float64(A.b);

// use a and b in the new struct call

JL_GC_POP();

What is the objective here? If this is still to be used from C++, there may be easier ways using Cxx.jl or CxxWrap.jl.

Cheers,

Bart

[K leo]

Thanks again.

What is  jl_init("/Users/inorton/git/jl71/usr/lib/")?

[K leo]

Bart,

Can you explain what you mean by "need to be rooted"?  The jl_new_struct statement as Isaiah suggested works, why do we need the additional statements as you suggested?

[Yichao Yu]

On Fri, Sep 9, 2016 at 8:36 PM, K leo <cnbi...@gmail.com> wrote:

Bart,

Can you explain what you mean by "need to be rooted"?  The jl_new_struct statement as Isaiah suggested works, why do we need the additional statements as you suggested?

Because it doesn't, it'll segfault at any time if the GC feels like giving you a bad day.

http://julia.readthedocs.io/en/latest/manual/embedding/#memory-management

[Yichao Yu]

On Fri, Sep 9, 2016 at 9:03 PM, Yichao Yu <yyc...@gmail.com> wrote:

On Fri, Sep 9, 2016 at 8:36 PM, K leo <cnbi...@gmail.com> wrote:

Bart,

Can you explain what you mean by "need to be rooted"?  The jl_new_struct statement as Isaiah suggested works, why do we need the additional statements as you suggested?

Because it doesn't, it'll segfault at any time if the GC feels like giving you a bad day.

http://julia.readthedocs.io/en/latest/manual/embedding/#memory-management

Also, having a C compatible object model, there's no need to use the boxed version. You can simply let julia allocate the struct for you using `jl_new_struct_uninit` and then initialize it as what you would do in C (the pointer returns points to the object with the layout you declare in julia, which should map directly to a c struct).

For types with only `isbits` fields, you can simply copy the memoy into newly allocated memory. For non-isbits fields, you can do that for initialization too, but not setting the field afterward (no matter how the object was allocated), requires a few more code, see the same section of manual linked above.

[Bart Janssens]

I just realise I made a typo, the rooting should be done using:  (note the &)

JL_GC_PUSH2(&a,&b);

Op za 10 sep. 2016 02:36 schreef K leo <cnbi...@gmail.com>:

[K leo]

I tried to compile the code with the GC statements, but got a bunch of error regarding jl_tls_states:

test6.cpp:(.text+0xd5): undefined reference to `jl_tls_states'

The compile script is as follows:

g++ -o test -fPIC -I$JULIA_DIR/include/julia test6.cpp -L$JULIA_DIR/lib/ -L$JULIA_DIR/lib/julia -lLLVM-3.7.1 -ljulia $JULIA_DIR/lib/julia/libstdc++.so.6

Where is `jl_tls_states' defined?

[Yichao Yu]

On Sat, Sep 10, 2016 at 6:56 AM, K leo <cnbi...@gmail.com> wrote:

I tried to compile the code with the GC statements, but got a bunch of error regarding jl_tls_states:

Use julia-config.jl to determine your compile flags.

[Kristoffer Carlsson]

Be aware of https://github.com/JuliaLang/julia/issues/14577#issuecomment-228568059 though

[K leo]

 julia-config.jl is hardly useful at all.  I unpacked the Linux binary of Julia (0.5) in ~/Software/julia-9c76c3e89a, and I set JULIA_DIR accordingly. When I execute julia-config in the following way, I got all kinds of "undefined reference to" error messages.

> $JULIA_DIR/share/julia/julia-config.jl --cflags --ldflags --ldlibs | xargs g++ test6.cpp

/usr/bin/ld: warning: libLLVM-3.7.1.so, needed by /home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so, not found (try using -rpath or -rpath-link)

/home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so: undefined reference to `llvm::AttributeSet::hasAttribute(unsigned int, llvm::Attribute::AttrKind) const'

/home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so: undefined reference to `llvm::LoopBase<llvm::BasicBlock, llvm::Loop>::getLoopLatch() const'

.........

If I only execute julia-config for flags, and I use these flags to cook up g++ script, and the g++ script got me the same problems.

> $JULIA_DIR/share/julia/julia-config.jl --cflags --ldflags --ldlibs 

-DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/xxx/Software/julia-9c76c3e89a/lib\" -I/home/xxx/Software/julia-9c76c3e89a/include/julia

-L/home/xxx/Software/julia-9c76c3e89a/lib

-Wl,-rpath,/home/xxx/Software/julia-9c76c3e89a/lib -ljulia

The output for --ldflags is clearly incorrect or incomplete for my installation on Ubuntu.  The script that generally works for me is the following, where some extra things have to be set.

> g++ -o test -fPIC -I$JULIA_DIR/include/julia test6.cpp -L$JULIA_DIR/lib/ -L$JULIA_DIR/lib/julia -lLLVM-3.7.1 -ljulia $JULIA_DIR/lib/julia/libstdc++.so.6

[Bart Janssens]

The concrete error you got is probably due to the missing -DJULIA_ENABLE_THREADING.

Op za 10 sep. 2016 14:53 schreef K leo <cnbi...@gmail.com>:

[Yichao Yu]

On Sat, Sep 10, 2016 at 8:53 AM, K leo <cnbi...@gmail.com> wrote:

 julia-config.jl is hardly useful at all.  I unpacked the Linux binary of Julia (0.5) in ~/Software/julia-9c76c3e89a, and I set JULIA_DIR accordingly. When I execute julia-config in the following way, I got all kinds of "undefined reference to" error messages.

> $JULIA_DIR/share/julia/julia-config.jl --cflags --ldflags --ldlibs | xargs g++ test6.cpp

/usr/bin/ld: warning: libLLVM-3.7.1.so, needed by /home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so, not found (try using -rpath or -rpath-link)

/home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so: undefined reference to `llvm::AttributeSet::hasAttribute(unsigned int, llvm::Attribute::AttrKind) const'

/home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so: undefined reference to `llvm::LoopBase<llvm::BasicBlock, llvm::Loop>::getLoopLatch() const'

.........

If I only execute julia-config for flags, and I use these flags to cook up g++ script, and the g++ script got me the same problems.

> $JULIA_DIR/share/julia/julia-config.jl --cflags --ldflags --ldlibs 

-DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/xxx/Software/julia-9c76c3e89a/lib\" -I/home/xxx/Software/julia-9c76c3e89a/include/julia

-L/home/xxx/Software/julia-9c76c3e89a/lib

-Wl,-rpath,/home/xxx/Software/julia-9c76c3e89a/lib -ljulia

The output for --ldflags is clearly incorrect or incomplete for my installation on Ubuntu.  The script that generally works for me is the following, where some extra things have to be set.

> g++ -o test -fPIC -I$JULIA_DIR/include/julia test6.cpp -L$JULIA_DIR/lib/ -L$JULIA_DIR/lib/julia -lLLVM-3.7.1 -ljulia $JULIA_DIR/lib/julia/libstdc++.so.6

The two compilation appears to be running against two independent installation and they are not comparable. Assuming the private libraries are found in lib/julia the julia-config.jl script should probably be updated to include it though.
 

[Yichao Yu]

On Sat, Sep 10, 2016 at 9:06 AM, Bart Janssens <ba...@bartjanssens.org> wrote:

The concrete error you got is probably due to the missing -DJULIA_ENABLE_THREADING.

Correct. In general though, you shouldn't try to figure out the correct flags manually.

[K leo]

I need to add "-L/home/djia/Software/julia-9c76c3e89a/lib/julia -lLLVM-3.7.1" into the script cooked up based on julia-config output, and then it works.

Should I file an issue on julia-config?

[K leo]

On Saturday, September 10, 2016 at 9:12:10 PM UTC+8, Yichao Yu wrote:

On Sat, Sep 10, 2016 at 8:53 AM, K leo <cnbi...@gmail.com> wrote:

 julia-config.jl is hardly useful at all.  I unpacked the Linux binary of Julia (0.5) in ~/Software/julia-9c76c3e89a, and I set JULIA_DIR accordingly. When I execute julia-config in the following way, I got all kinds of "undefined reference to" error messages.

> $JULIA_DIR/share/julia/julia-config.jl --cflags --ldflags --ldlibs | xargs g++ test6.cpp

/usr/bin/ld: warning: libLLVM-3.7.1.so, needed by /home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so, not found (try using -rpath or -rpath-link)

/home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so: undefined reference to `llvm::AttributeSet::hasAttribute(unsigned int, llvm::Attribute::AttrKind) const'

/home/xxx/Software/julia-9c76c3e89a/lib/libjulia.so: undefined reference to `llvm::LoopBase<llvm::BasicBlock, llvm::Loop>::getLoopLatch() const'

.........

If I only execute julia-config for flags, and I use these flags to cook up g++ script, and the g++ script got me the same problems.

> $JULIA_DIR/share/julia/julia-config.jl --cflags --ldflags --ldlibs 

-DJULIA_ENABLE_THREADING=1 -fPIC -DJULIA_INIT_DIR=\"/home/xxx/Software/julia-9c76c3e89a/lib\" -I/home/xxx/Software/julia-9c76c3e89a/include/julia

-L/home/xxx/Software/julia-9c76c3e89a/lib

-Wl,-rpath,/home/xxx/Software/julia-9c76c3e89a/lib -ljulia

The output for --ldflags is clearly incorrect or incomplete for my installation on Ubuntu.  The script that generally works for me is the following, where some extra things have to be set.

> g++ -o test -fPIC -I$JULIA_DIR/include/julia test6.cpp -L$JULIA_DIR/lib/ -L$JULIA_DIR/lib/julia -lLLVM-3.7.1 -ljulia $JULIA_DIR/lib/julia/libstdc++.so.6

The two compilation appears to be running against two independent installation and they are not comparable. Assuming the private libraries are found in lib/julia the julia-config.jl script should probably be updated to include it though.
 

I do have two installation of Julia (0.4.6 and 0.5), but as I set JULIA_DIR to the 0.5 dir, the two compilations apply to the same installation.

[K leo]

Hi Bart,

Do you think "jl_A" should also be rooted as it is later used as the argument?

On Saturday, September 10, 2016 at 3:38:27 AM UTC+8, Bart Janssens wrote:

[K leo]

Hi Bart,

These are meant to call Julia code from C++.  You mentioned "there may be easier ways using Cxx.jl or CxxWrap.jl".  Are the two packages only for calling C/C++ from Julia, and not the otherway around?  Am I missing something?

On Saturday, September 10, 2016 at 3:38:27 AM UTC+8, Bart Janssens wrote:

[Yichao Yu]

On Sat, Sep 10, 2016 at 8:41 PM, K leo <cnbi...@gmail.com> wrote:

Hi Bart,

Do you think "jl_A" should also be rooted as it is later used as the argument?

Yes. When you use it.

[Bart Janssens]

On Sun, Sep 11, 2016 at 2:47 AM K leo <cnbi...@gmail.com> wrote:

Hi Bart,

These are meant to call Julia code from C++.  You mentioned "there may be easier ways using Cxx.jl or CxxWrap.jl".  Are the two packages only for calling C/C++ from Julia, and not the otherway around?  Am I missing something?

Yes, both are initialized from Julia. So you would use them to start your C++ framework, but then you can call back into Julia from there (you don't need the jl_init stuff then, a nice bonus). I've added a test replicating your example to CxxWrap.jl:
https://github.com/barche/CxxWrap.jl/commit/9711b4d6baca99c2c30280d197bba3168167ed59

In Cxx.jl, as Keno said you can implement a C++ (member) function using Julia code, like this for example:

using Cxx
using Base.Test

type JuliaTestType

  a::Float64
  b::Float64
end

function julia_test_func(data)
  println("a: ", data.a, ", b: ", data.b)
  @test data.a == 2.
  @test data.b == 3.
  nothing
end

# Declare a C++ class
cxx"""
class CppClass {
public:
  void call_julia();
};
"""

# Implement member function in Julia
@cxxm "void CppClass::call_julia()" begin
  A = JuliaTestType(2.,3.)
  julia_test_func(A)
end

# Call C++ function
icxx"""
CppClass a;
a.call_julia();
"""

[K leo]

Thanks so much for the guide, Bart.  Will study.