Ioctl Syscall with go and c

[Serguei Bezverkhi (sbezverk)]

Hello,

I am converting some C code to Go and hit an issue with one particular Syscall:

In C:

device = ioctl(group, 0x3b6a, path);
where path is char[N]

In Go:
ioctlId := 0x3b6a
device, _, errno := syscall.Syscall(
syscall.SYS_IOCTL,
uintptr(group),
uintptr(unsafe.Pointer(&ioctlId)),
uintptr(unsafe.Pointer(pciDevice)),
)
Where pciDevice is *string with exactly the same value as path in C.

When I run Go bits on the same h/w, same OS, same everything, it fails with "errno 22 (invalid argument)". It seems that the issue is how string gets passed to Syscall, but I could not find any examples how to do it correctly.
Appreciate some advice here.

Thank you
Serguei

[Tamás Gulácsi]

Is it \0-terminated?

[Serguei Bezverkhi (sbezverk)]

I changed code a little bit to be able to examine variables:

 

 func GetGroupFD(group int, pciDevice *string) (int, error) {

    ioctlId := 0x3b6a

    var buffer uintptr

    buffer = uintptr(unsafe.Pointer(pciDevice))

    device, _, errno := syscall.Syscall(

        syscall.SYS_IOCTL,

        uintptr(group),

        uintptr(unsafe.Pointer(&ioctlId)),

        buffer,

    )

 

(gdb) p &buffer

$8 = (uintptr *) 0xc4200c7b88

(gdb) x/g 0xc4200c7b88

0xc4200c7b88:   0x000000c4200c7d08

(gdb) x/g 0x000000c4200c7d08

0xc4200c7d08:   0x000000c4200122d0

(gdb) x/g 0x000000c4200122d0

0xc4200122d0:   0x3a31383a30303030

 

0x30303030 is start of expected string “0000:81:11.1”,  first 4 bytes are not part of it. Anybody knows where those 4 bytes could have come from? And how to get rid of them?

Thank you

Serguei

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Max]

Hi Serguei,

a Go string or *string do not correspond to a C char *

You must pass the address of the first byte of a sufficiently large buffer:

func GetGroupFD(group int, pciDevice *string) (int, error) {

  const N = 256

  var buffer [N]byte

  device, _, errno := syscall.Syscall(
        syscall.SYS_IOCTL,
        uintptr(group),
        uintptr(unsafe.Pointer(&ioctlId)),

        &buffer[0],
    )

   /* if ioctl() is successful, find '\0' in buffer[] and copy the relevant portion in *pciDevice */

[Serguei Bezverkhi (sbezverk)]

Hi Max,

 

Thanks for the suggestion, unfortunately it did not help, see below:

 

func GetGroupFD(group int, pciDevice *string) (int, error) {

    fmt.Printf("VFIO_GROUP_GET_DEVICE_FD() returned: %04x\n", VFIO_GROUP_GET_DEVICE_FD())

    buffer := make([]byte, len(*pciDevice)+1)

    for i, c := range *pciDevice {

        buffer[i] = uint8(c)

    }

    buffer[len(*pciDevice)] = 0x0

    fmt.Printf("pciDevice: %s\n", string(buffer))

    device, _, errno := syscall.Syscall(

        syscall.SYS_IOCTL,

        uintptr(group),

        uintptr(VFIO_GROUP_GET_DEVICE_FD()),

        uintptr(unsafe.Pointer(&buffer[0])),

    )

    if errno != 0 {

        return 0, fmt.Errorf("fail to get file descriptor for %d with errno: %+v", group, errno)

    }

    return int(device), nil

}

 

Any other suggestions?

Thank you

Serguei

 

From: <golan...@googlegroups.com> on behalf of Max <massimilia...@gmail.com>

Date: Thursday, August 23, 2018 at 10:46 AM
To: golang-nuts <golan...@googlegroups.com>

--

[jake...@gmail.com]

First off, I don't know how you are posting your code samples, but they are unreadable in my Firefox and Chrome on windows.

Second, I would point out that the system you use for converting a "string" to the buffer will fail if there are any non-ascii characters.

Third, OT, but I wonder why pciDevice is a string pointer, instead of a string?

Personally, I don't see anything decisively incorrect, but there is a lot of context that is missing. If you wanted to indicate your OS, and provide a complete working sample, I'm sure I, or someone else could help more.

One thing that jumps out at me is the use of  uintptr(unsafe.Pointer(&ioctlId)), in your initial example. It is hidden by the function  VFIO_GROUP_GET_DEVICE_FD(), which is not provided, in your larger example. Are you sure you want the pointer to that value, and not the actual value? Maybe uintptr(ioctlId)) instead. Seems more standard for IOCTL to me.

And, of course the changes you made to create an appropriate string buffer are necessary.

[Serguei Bezverkhi (sbezverk)]

Hello Jake,

 

Apologies about formatting, here is correctly formatted code:

 

// GetGroupFD gets File descriptor for a specified by PCI address device

func GetGroupFD(group int, pciDevice string) (int, error) {

                buffer := make([]byte, len(pciDevice)+1)

                for i, c := range pciDevice {

                                buffer[i] = uint8(c)

                }

                ioctlID := VFIO_GROUP_GET_DEVICE_FD()

                fmt.Printf("VFIO_GROUP_GET_DEVICE_FD() returned: %04x\n", ioctlID)

                buffer[len(pciDevice)] = 0x0

                fmt.Printf("pciDevice: %s\n", string(buffer))

                device, _, errno := syscall.Syscall(

                                syscall.SYS_IOCTL,

                                uintptr(group),

                                uintptr(unsafe.Pointer(&ioctlID)),

                                uintptr(unsafe.Pointer(&buffer[0])),

                )

                if errno != 0 {

                                return 0, fmt.Errorf("fail to get file descriptor for %d with errno: %+v", group, errno)

                }

                return int(device), nil

}

 

I have also addressed your suggestions without any success. It is latest Centos 7.5.

 

Thank you

Serguei

[Tamás Gulácsi]

Cany you give a complete example?

https://play.golang.org/p/ZQSf-PwMtd9

says

"device=18446744073709551615 errno=inappropriate ioctl for device"

[Serguei Bezverkhi (sbezverk)]

I copied the whole thing to https://github.com/sbezverk/vfio

Running it without actual vfio/iommu might be problematic though.

Before this specific ioctl can be run 2 open calls must succeed. See func main.

Thank you for looking into this problem
Serguei

Sent from my iPhone

[Tamás Gulácsi]

Ok, clearly I cannot test it - either I don't have the required setup, or the needed knowledge of what the hell is VFIO.

Some googling around says:

VFIO_GROUP_GET_DEVICE_FD - _IOW(VFIO_TYPE, VFIO_BASE + 6, char)
 *
 * Return a new file descriptor for the device object described by
 * the provided string.  The string should match a device listed in
 * the devices subdirectory of the IOMMU group sysfs entry.  The
 * group containing the device must already be added to this context.
 * Return: new file descriptor on success, -errno on failure.
 * Availability: When attached to container
 */

The strange thing in vfio.go GetGroupFD is the use of *string - as you don't want to modify that path, you shouldn't use a pointer.
But that's nothing to do with the ioctl.

The fill of buffer, and the use seems legit.

IF you have a working C program, I'd try to strace both C and Go and see what's different.

[jake...@gmail.com]

Probably unrelated to your problem, but IIUC, the syscall package is "deprecated", or "frozen". According to the syscall documentation:

This package is locked down. Code outside the standard Go repository should be migrated to use the corresponding package in the golang.org/x/sys repository. That is also where updates required by new systems or versions should be applied.

So, while unlikely, it is possible that is the problem. In any case, unless I misunderstand, the preferred package for you to use is golang.org/x/sys/unix.

[Serguei Bezverkhi (sbezverk)]

Hi Jake,

 

Ioctl implementation in unix package seems a bit limited, how do you pass for example a required struct if “func IoctlSetInt(fd int, req uint, value int)” allows only to accept int?? If this package had a func accepting uintptr as a parameter, then I guess that could work.

 

Thank you

Serguei

 

 

From: <golan...@googlegroups.com> on behalf of "jake...@gmail.com" <jake...@gmail.com>

Date: Friday, August 24, 2018 at 12:09 PM
To: golang-nuts <golan...@googlegroups.com>

--

[Ian Lance Taylor]

On Sun, Aug 26, 2018 at 6:59 AM, 'Serguei Bezverkhi (sbezverk)' via
golang-nuts <golan...@googlegroups.com> wrote:
>
> Ioctl implementation in unix package seems a bit limited, how do you pass
> for example a required struct if “func IoctlSetInt(fd int, req uint, value
> int)” allows only to accept int?? If this package had a func accepting
> uintptr as a parameter, then I guess that could work.

There are several variants of ioctl in the golang.org/x/sys/unix
package. Search for Ioctl in https://godoc.org/golang.org/x/sys/unix.
The intent is to handle ioctl in a type-safe manner, as much as is
possible. There is always the escape hatch of unix.Syscall(SYS_IOCTL,
...).

Ian

[jake...@gmail.com]

I was aware that your parameters would not fit the IoctlXXXX functions. As Ian pointed out, you would need to use golang.org/x/sys/unix/Syscall() and friends.