cgo vs ioctl system calls

[Tim K]

I'm looking for some expert opinions. Disclaimer: I did some C in a previous life and I'm new to Go. Yes, I know, maybe I should leave the topic to the experts :-)

Assume you have the following choices to interact with some hardware attached to a Linux server x86_64:

(1) Write a wrapper around a user space C API that in turn makes ioctl kernel calls to talk to the hardware

(2) Do ioctl calls directly from Go against the kernel

(Both the kernel driver and user space C API are developed in house, source is available and the developer who writes them both is right there ready to help)

Which way would you Go? What are the pros and cons of each one? It looks like either approach is "ugly" in its own way. One thing that is clear is that I will be creating a Go library that exposes an easy to use Go API to the rest of the application.

And my understanding is that making ioctl calls from Go does not involve cgo at all, it's just dealing directly with the syscall package and unsafe.Pointers (fun!)

I hope this is the start of a good discussion, I'd love to hear some opinions from those who were faced with the same decision and tried one or both approaches. Even better if there is some good sample code (I found some but I'd like to see some recommendations for code to look at that "does it the right way").

Thank you.

Tim

[Ian Lance Taylor]

I would normally make the ioctl calls directly from Go. No reason to
loop in C unless it's needed.

Ian

[Tim K]

Thanks Ian. At some point we would like to add event notifications from the kernel space to user space (when something interesting happens with the hardware, e.g. errors, plug/unplug, etc.) as opposed to the user space constantly polling for events. Any ideas what can be done with Go on the user space side to make an eventing mechanism work? I've done a bit of reading and I see at least epoll, SIGIO, netlink being mentioned. What would pair well with Go?

Tim

[Ian Lance Taylor]

On Thu, Dec 3, 2015 at 12:53 PM, Tim K <tim....@gmail.com> wrote:
>
> Thanks Ian. At some point we would like to add event notifications from the
> kernel space to user space (when something interesting happens with the
> hardware, e.g. errors, plug/unplug, etc.) as opposed to the user space
> constantly polling for events. Any ideas what can be done with Go on the
> user space side to make an eventing mechanism work? I've done a bit of
> reading and I see at least epoll, SIGIO, netlink being mentioned. What would
> pair well with Go?

I don't really know what your options are. SIGIO would work OK, but
it's an overloaded mechanism. I don't anything about netlink. If the
kernel can write data to a file descriptor, that would be very easy to
handle.

Ian

[Tim K]

Any good sample code for ioctl use from Go? Best practices, error handling, etc. from a project known to have done it right.

Thanks.

Tim

[Ian Lance Taylor]

On Thu, Dec 3, 2015 at 3:50 PM, Tim K <tim....@gmail.com> wrote:
>
> Any good sample code for ioctl use from Go? Best practices, error handling,
> etc. from a project known to have done it right.

I don't know that it is done right, but there is
golang.org/x/crypto/ssh/terminal.

Ian

[Nick Craig-Wood]

On 03/12/15 19:49, Tim K wrote:
> I'm looking for some expert opinions. Disclaimer: I did some C in a
> previous life and I'm new to Go. Yes, I know, maybe I should leave the
> topic to the experts :-)
>
> Assume you have the following choices to interact with some hardware
> attached to a Linux server x86_64:
>
> (1) Write a wrapper around a user space C API that in turn makes ioctl
> kernel calls to talk to the hardware
>
> (2) Do ioctl calls directly from Go against the kernel

I chose using a C wrapper when writing
https://github.com/ncw/go-nflog-acctd as the NFLOG protocol is quite
complicated and embedded into the C library. I did look at porting the
C code to go, but decided to go with cgo as it was rather complicated.

For a simple ioctl interface I'd go with embedding it in go, but watch
out for 32 vs 64 bit calling conventions!

--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick

[Tim K]

The third argument to the ioctl() call spooks me if the kernel needs to return a more complex struct, I'm not exactly sure how to deal with it form the Go side, how to allocate memory for it and then how to unpickle it into a Go struct. Simple examples that just return some raw bytes from the kernel I can see how it could be made to work, but more complicated structs I'm not sure.
Any "pointers" in the right direction? Pun intended...

Tim

[Ian Lance Taylor]

On Fri, Dec 4, 2015 at 10:02 AM, Tim K <tim....@gmail.com> wrote:
>
> The third argument to the ioctl() call spooks me if the kernel needs to
> return a more complex struct, I'm not exactly sure how to deal with it form
> the Go side, how to allocate memory for it and then how to unpickle it into
> a Go struct. Simple examples that just return some raw bytes from the kernel
> I can see how it could be made to work, but more complicated structs I'm not
> sure.
> Any "pointers" in the right direction? Pun intended...

For a complicated struct you should probably either use cgo or use
`cgo -godefs` to generate the structs in Go as the
golang.org/x/sys/unix package does.

Ian

[Tim K]

It seems that you can't completely escape using cgo. Even if the structs are 1-time generated with 'cgo -godefs' you still have to do "C" conversions for example from char* C strings to Go strings, potentially both ways as an ioctl() call may expect some data in some struct form. Does that sound correct or am I missing something?

Thanks.

Tim

[Ian Lance Taylor]

I don't think I understand what you mean, so one of us is missing
something. Definitely if an ioctl expects a C string--that is, a NUL
terminated sequence of bytes--then you need to give it that.
Similarly, if an ioctl returns such a thing, you need to do some work
to turn that into a Go string. That doesn't mean you need to use cgo
for that.

Still, if this all seems to be too much trouble, just use cgo. There
is no significant penalty for doing so.

Ian

[Tim K]

I see, it means that I'd have to write similar conversion functions to C.CString, C.GoString, ...
 

Still, if this all seems to be too much trouble, just use cgo.  There
is no significant penalty for doing so.

My first attempt will be to use cgo and maybe later attempt an ioctl impl. If the Go library has a good API interface the impl can be changed later. The Go user space code is not performance critical so I'm not worried about performance just what's easier to implement and easier to debug when things don't work.

Thanks again.

Tim