cgo: support for exporting structs?

[Andy Maloney]

Given this:

type Bar struct {
    Flurb int
    Blarg string
}

func Foo() Bar {
    return Bar{42, "blarg"}
}

go build -buildmode=c-archive produces this error:

# foo
./foo.go:6:11: Go type not supported in export: struct {
    Flurb    int
    Blarg    string
}

Are there any plans to support exporting of structs? (Or am I doing this incorrectly?)

It would be useful if it could produce this:

struct Bar {
    GoInt Flurb;
    GoString Blarg;
};

extern struct Bar Foo();

[Ian Lance Taylor]

On Thu, Aug 6, 2015 at 3:26 PM, Andy Maloney <asma...@gmail.com> wrote:
>
> Given this:
>
> type Bar struct {
> Flurb int
> Blarg string
> }
>
> func Foo() Bar {
> return Bar{42, "blarg"}
> }
>
> go build -buildmode=c-archive produces this error:
>
> # foo
> ./foo.go:6:11: Go type not supported in export: struct {
> Flurb int
> Blarg string
> }
>
>
>
> Are there any plans to support exporting of structs? (Or am I doing this
> incorrectly?)

We can support structs in some cases. It's a TODO in the source code
right now (see Package.cgoType in cmd/cgo/out.go).

Ian

[Andy Maloney]

So I spent some time picking apart and experimenting with out.go and have "something that does something".

The approach I took is this:

• after all the Decls are added to the Package in Record() (in main.go), call a function which looks at each of the exported functions and uses the Decls to create a map to store some information about struct parameters & results
• in cgoType(), use the map that was just created to lookup the name of the struct and its size (which we calculated and stored in the map) and, if we find it, return it
• at the end of writeExportHeader(), write out typedefs for each of the structs in the map

This results in the following:

type Bar struct {
    Flurb int
    Blarg string
}

/* Foo does something */
//export Foo
func Foo() Bar {
    return Bar{42, "foo"}
}

being exported in the headers as follows:

typedef struct {
    GoInt    Flurb;
    GoString    Blarg;
} Bar;


/* Foo does something */

extern Bar Foo();

and _cgo_export.c looks like this:

extern void _cgoexp_187552b8a432_Foo(void *, int);

Bar Foo()
{
    _cgo_wait_runtime_init_done();
    struct {
        Bar r0;
    } __attribute__((__packed__)) a;
    crosscall2(_cgoexp_187552b8a432_Foo, &a, 24);
    return a.r0;
}

and it links and executes properly.


Questions:

1. Given the structure and intent of the code in out.go, is this a reasonable approach?
2. Right now I just look at structs, but it would make sense to me that if you declare
   type Foo int
   and you use it as a result to a function we should see
   typedef GoInt Foo;
   in the header file.  Would that make sense to add too?
3. Is there a way given an ast.Field.Type to get it as a string?  I found a couple of things online, but no solution which fits since we don't have the text of the source file to use the Pos with.  I ended up using
   fmt.Sprintf("%v", param.Type)
   which works but feels wrong...
4. I created the type map like this on Package:
   map[ast.Expr]ExpType
   (where ExpType is where I'm storing info about the struct).  I don't think this works across packages.  Is there a way to uniquely identify a type in the compilation unit what I can use as a key?  Keeping in mind that, the way I've implemented it, I need access to that key in cgoType()...

I have other questions of course, but figured I would start there in case I'm way off base.

[Ian Lance Taylor]

On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney <asma...@gmail.com> wrote:
>
> after all the Decls are added to the Package in Record() (in main.go), call
> a function which looks at each of the exported functions and uses the Decls
> to create a map to store some information about struct parameters & results
> in cgoType(), use the map that was just created to lookup the name of the
> struct and its size (which we calculated and stored in the map) and, if we
> find it, return it
> at the end of writeExportHeader(), write out typedefs for each of the
> structs in the map

I'm not quite sure what you mean here. It seems to me that you can
treat any struct used by an exported function as though the Go code
said "C.struct_foo". That should define in Go like other cgo
structs. Perhaps this is what you are doing, or perhaps I am missing
something.

> Right now I just look at structs, but it would make sense to me that if you
> declare
> type Foo int
> and you use it as a result to a function we should see
> typedef GoInt Foo;
> in the header file. Would that make sense to add too?

I doubt it. C typedefs are not Go typedefs. Go's types are much
stricter. I expect that changing this would break currently working
code.

> Is there a way given an ast.Field.Type to get it as a string? I found a
> couple of things online, but no solution which fits since we don't have the
> text of the source file to use the Pos with. I ended up using
> fmt.Sprintf("%v", param.Type)
> which works but feels wrong...

That eems fine to me.

> I created the type map like this on Package:
> map[ast.Expr]ExpType
> (where ExpType is where I'm storing info about the struct). I don't think
> this works across packages. Is there a way to uniquely identify a type in
> the compilation unit what I can use as a key? Keeping in mind that, the way
> I've implemented it, I need access to that key in cgoType()...

The cgo interface by definition lives within a single Go package. You
can't pass C types between Go packages.

Ian

[Andy Maloney]

On Sunday, August 9, 2015 at 4:05:34 PM UTC-4, Ian Lance Taylor wrote:

On Sun, Aug 9, 2015 at 11:47 AM, Andy Maloney <asma...@gmail.com> wrote:
>
> after all the Decls are added to the Package in Record() (in main.go), call
> a function which looks at each of the exported functions and uses the Decls
> to create a map to store some information about struct parameters & results
> in cgoType(), use the map that was just created to lookup the name of the
> struct and its size (which we calculated and stored in the map) and, if we
> find it, return it
> at the end of writeExportHeader(), write out typedefs for each of the
> structs in the map

I'm not quite sure what you mean here.  It seems to me that you can
treat any struct used by an exported function as though the Go code
said "C.struct_foo".  That should define in Go like other cgo
structs.  

I'm not seeing how to do this, but does that mean from my example above I'd end up with this in the header:

typedef struct {
    GoInt    r0;
    GoString    r1;
} struct_Bar;


extern struct_Bar Foo();

(And I'd lose the names of the members like I currently do in the return structs?)
 

Perhaps this is what you are doing, or perhaps I am missing
something.

Oh no - it's much more likely I'm missing something.  :-)

So when I'm in cgoType() and looking at an *ast.StructType - how do I get the type name?  That's where I'd need to call it "struct_Bar", right?
 

> Right now I just look at structs, but it would make sense to me that if you
> declare
> type Foo int
> and you use it as a result to a function we should see
> typedef GoInt Foo;
> in the header file.  Would that make sense to add too?

I doubt it.  C typedefs are not Go typedefs.  Go's types are much
stricter.  I expect that changing this would break currently working
code.

Wouldn't this just be preserving the name of the type in the C code?  Whether you call it a GoInt or a Foo in this case should be the same - it's just an alias in C.  Not sure I see how that would break anything.

> Is there a way given an ast.Field.Type to get it as a string?  I found a
> couple of things online, but no solution which fits since we don't have the
> text of the source file to use the Pos with.  I ended up using
> fmt.Sprintf("%v", param.Type)
> which works but feels wrong...

That eems fine to me.


> I created the type map like this on Package:
> map[ast.Expr]ExpType
> (where ExpType is where I'm storing info about the struct).  I don't think
> this works across packages.  Is there a way to uniquely identify a type in
> the compilation unit what I can use as a key?  Keeping in mind that, the way
> I've implemented it, I need access to that key in cgoType()...

The cgo interface by definition lives within a single Go package.  You
can't pass C types between Go packages.

Ah - I thought I could split my code into several "internal" packages that may reference each other and create one .a archive out of it.

(FYI - I am looking at using this to move some or all of a data model out of a medium-size C++ app to Go and then link it in as a lib.  Maybe it isn't quite a fit.)

Thanks for your time Ian.

[Ian Lance Taylor]

Sorry, I was confused. You have structs defined in Go, and you need
them to be defined in C. For some reason I was thinking about this
the other way around.

Sure, something like what you described sounds reasonable.

>> > Right now I just look at structs, but it would make sense to me that if
>> > you
>> > declare
>> > type Foo int
>> > and you use it as a result to a function we should see
>> > typedef GoInt Foo;
>> > in the header file. Would that make sense to add too?
>>
>> I doubt it. C typedefs are not Go typedefs. Go's types are much
>> stricter. I expect that changing this would break currently working
>> code.
>
>
> Wouldn't this just be preserving the name of the type in the C code?
> Whether you call it a GoInt or a Foo in this case should be the same - it's
> just an alias in C. Not sure I see how that would break anything.

Same confusion on my part. I'm not sure the typedef names help much
but I suppose it is fine.

Ian

[Andy Maloney]

I imagine it will take some back-and-forth with you to get it into shape - make sure it's idiomatic and fits with the current structure - so I'll wait until after 1.5 is out.

It requires changes I put in another CL (https://go-review.googlesource.com/#/c/13061/ ).  How do I handle that when I want to create a new CL?  Just branch off that one?  Or do just wait until the other one is accepted and merged?

[Ian Lance Taylor]

On Sun, Aug 9, 2015 at 4:24 PM, Andy Maloney <asma...@gmail.com> wrote:
>
> I imagine it will take some back-and-forth with you to get it into shape -
> make sure it's idiomatic and fits with the current structure - so I'll wait
> until after 1.5 is out.
>
> It requires changes I put in another CL
> (https://go-review.googlesource.com/#/c/13061/ ). How do I handle that when
> I want to create a new CL? Just branch off that one? Or do just wait until
> the other one is accepted and merged?

My understanding is that you can create a new CL based on an existing
one by running "git codereview change" and deleting the Change-ID
line. You will get a new Change-ID, and a different CL. Gerrit will
show the CLs as related. But I haven't really tried this and I may
have gotten some detail wrong.

Or if the two CLs are unrelated, just use a branch. For example, "git
codereview change BRANCHNAME" will give you a new branch.

Ian

[dav...@gmail.com]

Hi all -

  I'm encountering a problem like Ian's original assumption. I have C files that have code like this:

extern struct dds_state dds_st;

extern struct ad9361_rf_phy *ad9361_phy;

  and cgo doesn't see this (I'm clearly confused). I have been able to just pull many C headers and implementation files into my package directory. But none of the previous C files had an extern struct xxx instance in them. Pulling the C file WITH the extern struct definitions breaks everything.

  Might you give me a suggestion as to how to handle this? There must be a way, right?

Thanks!

[Ian Lance Taylor]

On Fri, Sep 18, 2015 at 5:02 PM, <dav...@gmail.com> wrote:
>
> I'm encountering a problem like Ian's original assumption. I have C files
> that have code like this:
>
> extern struct dds_state dds_st;
> extern struct ad9361_rf_phy *ad9361_phy;
>
> and cgo doesn't see this (I'm clearly confused). I have been able to just
> pull many C headers and implementation files into my package directory. But
> none of the previous C files had an extern struct xxx instance in them.
> Pulling the C file WITH the extern struct definitions breaks everything.
>
> Might you give me a suggestion as to how to handle this? There must be a
> way, right?

Can you clarify what you are trying to do? Are you building with
-buildmode=c-archive? What does your "C" comment look like? What
exactly is happening? What do you expect to happen instead?

Ian

[Andy]

Richard:

If you're asking about:

  "cmd/cgo: annotate named return struct members in comments "
  https://go-review.googlesource.com/#/c/13061/

It looks like that was merged at some point, but I'm not sure.

I abandoned my experiments with go/cgo when some of the limitations became clear.  I wouldn't ultimately be able to do what I'd wanted to with it.

---
Andy Maloney  //  https://asmaloney.com

twitter ~ @asmaloney

On Sun, Oct 16, 2016 at 12:51 PM, Richard Wilkes <raw.so...@gmail.com> wrote:

Andy,

What's the status of getting this change into Go? I have an issue that could benefit from this particular support and was curious if/when I'd be able to use it.

Thanks!

- Rich

[Andy]

Ah - ok.

As I recall, I submitted a CR for it but Ian pointed out an edge case I needed to handle. I never got to it because I changed direction and it didn't seem like anyone else was interested in this modification at the time.

I can dig out the diffs from my previous attempt if that would help get you started - let me know.

---
Andy Maloney  //  https://asmaloney.com

twitter ~ @asmaloney

On Sun, Oct 16, 2016 at 3:07 PM, Richard Wilkes <wil...@me.com> wrote:

Hi, Andy.

Thanks for replying. No, I was interested in the change to allow a struct defined in Go to be passed to C, even if it only supported basic types. Sounds like that part never made it in — certainly Go 1.7.1 doesn’t seem to allow me to do such things, which is unfortunate.

- Rich

[Andy]

Here is the original CR:

  https://go-review.googlesource.com/#/c/14060/

Locally I'd fixed the tests and added one for the unnamed struct case Ian pointed out - but I never checked this in (see attached diff).

So after this is brought in line with the current source I think it just needs to be modified to handle unnamed structs.

---
Andy Maloney  //  https://asmaloney.com

twitter ~ @asmaloney

On Sun, Oct 16, 2016 at 4:17 PM, Richard Wilkes <wil...@me.com> wrote:

Yes, that or just a link to the CR would be good. Thanks!

[Richard Wilkes]

Hi, Andy.

Thanks for replying. No, I was interested in the change to allow a struct defined in Go to be passed to C, even if it only supported basic types. Sounds like that part never made it in — certainly Go 1.7.1 doesn’t seem to allow me to do such things, which is unfortunate.

- Rich

On Oct 16, 2016, at 10:23 AM, Andy <asma...@gmail.com> wrote:

[Richard Wilkes]

Andy,

What's the status of getting this change into Go? I have an issue that could benefit from this particular support and was curious if/when I'd be able to use it.

Thanks!

- Rich

On Sunday, August 9, 2015 at 4:24:52 PM UTC-7, Andy Maloney wrote:

[Richard Wilkes]

Yes, that or just a link to the CR would be good. Thanks!

- Rich

On Oct 16, 2016, at 1:15 PM, Andy <asma...@gmail.com> wrote: