Unexpected cgo argument has Go pointer to Go pointer

[Phil Noel]

Hello,

I've been coding up a Windows based application and hit a problem that I sort of understand however not how to resolve.

The debug + panic is

TranslateMessage Msg: &win.Msg{Wnd:(win.HWnd)(0x5a0cdc), Message:0x113, WParam:0x1, LParam:0, Time:0x28699637, Pt:win.Point{X:1174, Y:488}}

TranslateMessage CMsg: &win._Ctype_struct_tagMSG{hwnd:(win._Ctype_HWND)(0x5a0cdc), message:0x113, _:[4]uint8{0x0, 0x0, 0x0, 0x0}, wParam:0x1, lParam:0, time:0x28699637, pt:win._Ctype_struct_tagPOINT{x:1174, y:488}, _:[4]uint8{0x0, 0x0, 0x0, 0x0}}

Size: SizeOf(*Msg) 48 = Sizeof(*CMsg) 48

panic: runtime error: cgo argument has Go pointer to Go pointer

goroutine 1 [running, locked to thread]:

win.TranslateMessage.func1(0xc0420760f0, 0x29)

C:/progs/ake_g/src/win/funcs.go:282 +0x59

win.TranslateMessage(0xc0420760f0, 0x0)

C:/progs/ake_g/src/win/funcs.go:282 +0x1da

ake.(*Platform).Poll(0xc0420760c0)

C:/progs/ake_g/src/ake/platform.go:84 +0x7c

ake.(*Application).Start(0xc042046420)

C:/progs/ake_g/src/ake/system.go:68 +0x10d

main.main()

C:/progs/ake_g/src/shooty/shooty.go:157 +0xc5

I have created a wrapped around the Windows TranslateMessage function. I am passing in my version of the Msg structure (that I have recreated in Go) which is then cast to C.MSG structure as constructed via cgo.

func TranslateMessage(msg *Msg) bool {
   fmt.Printf("TranslateMessage Msg: %#v\n", msg)
   c_msg := (*C.MSG)(unsafe.Pointer(msg))
   fmt.Printf("TranslateMessage CMsg: %#v\n", c_msg)
   fmt.Printf("Size: SizeOf(*Msg) %d = Sizeof(*CMsg) %d\n", unsafe.Sizeof(*msg), unsafe.Sizeof(*c_msg))

   rc := C.TranslateMessage(c_msg)

   return rc != WinFalse
}

The debug shows that both these structures have the same memory layout and are the same size (though the cgo version correctly has forced padding).

The C.TranslateMessage (generated by cgo) is

//line c:\progs\ake_g\src\win\funcs.go:274
func TranslateMessage(msg *Msg) bool {
                  fmt.Printf("Msg: %#v\n", msg)
                  c_msg := (*_Ctype_struct_tagMSG)(unsafe.Pointer(msg))
                  fmt.Printf("CMsg: %#v\n", c_msg)
                  fmt.Printf("Size: %d = %d\n", unsafe.Sizeof(*msg), unsafe.Sizeof(*c_msg))
//line c:\progs\ake_g\src\win\funcs.go:281

//line c:\progs\ake_g\src\win\funcs.go:280
   rc := func(_cgo0 *_Ctype_struct_tagMSG) _Ctype_WINBOOL {
//line c:\progs\ake_g\src\win\funcs.go:280
      _cgoCheckPointer(_cgo0)
//line c:\progs\ake_g\src\win\funcs.go:280
      return _Cfunc_TranslateMessage(_cgo0)
//line c:\progs\ake_g\src\win\funcs.go:280
   }(c_msg)
//line c:\progs\ake_g\src\win\funcs.go:283

//line c:\progs\ake_g\src\win\funcs.go:282
   return rc != WinFalse
}

I believe the error is that _cgoCheckPointer is scanning the _cgo0 variable (ie c_msg, ie type *_Ctype_struct_tagMSG) and detecting the hwnd:(win._Ctype_HWND)(0x5a0cdc) value 0x5a0cdc is a pointer to something on the Go heap.

In reality this value is assigned by Windows and is actually the window handle.

Initially I was just going to force my win.HWnd to be a uintptr (as effectively its a handle). However the cgo code knows its actually an opaque pointer and thus treats it as a real pointer (unsafe.Pointer). So I think forcing my code to use a uintptr won't help. Also it seems that general guidelines are to use unsafe.Pointer if its a pointer (or appears to be).

This is the structure that cgo generates based on the window.h header file

type _Ctype_struct_tagMSG struct {
   hwnd   _Ctype_HWND
   message    _Ctype_UINT
   _  [4]byte
   wParam _Ctype_WPARAM
   lParam _Ctype_LPARAM
   time   _Ctype_DWORD
   pt _Ctype_struct_tagPOINT
   _  [4]byte
}

type _Ctype_HWND *_Ctype_struct_HWND__

type _Ctype_struct_HWND__ struct {
   unused _Ctype_int
}

Its also worth mentioning that even though the C header has HWND as a struct HWND__* in reality the HWND can be align on a 2 byte boundary (ie its not a real pointer/8 byte boundary).

I am running this with

go version go1.8.3 windows/amd64

Also go vet (not surprisingly) doesn't find any problems with my code (mainly because this pointer/handle is runtime generated by Windows).

Does anyone have any suggestions about how to work around this (other than creating a fake windows.h)?

The problem occurs roughly 5% of the time when I start my program.

Thanks

Phil

[Ian Lance Taylor]

On Tue, Aug 29, 2017 at 8:53 AM, Phil Noel <ph...@noels.org.uk> wrote:
>
> I believe the error is that _cgoCheckPointer is scanning the _cgo0 variable
> (ie c_msg, ie type *_Ctype_struct_tagMSG) and detecting the
> hwnd:(win._Ctype_HWND)(0x5a0cdc) value 0x5a0cdc is a pointer to something on
> the Go heap.
>
> In reality this value is assigned by Windows and is actually the window
> handle.
>
> Initially I was just going to force my win.HWnd to be a uintptr (as
> effectively its a handle). However the cgo code knows its actually an opaque
> pointer and thus treats it as a real pointer (unsafe.Pointer). So I think
> forcing my code to use a uintptr won't help. Also it seems that general
> guidelines are to use unsafe.Pointer if its a pointer (or appears to be).

I didn't try to fully understand this code, but this point seems
problematic. From Go's perspective, a value is either a pointer or
not. There can't be any waffling on that. If win.HWnd is a handle,
not a memory address, then you must treat it as a uintptr. If it is a
pointer--if you could reliably convert it to a pointer type via
unsafe.Pointer and dereference it to get valid data--then you must
treat it as a pointer. From Go's perspective there is no such thing
as an opaque pointer or a value that appears to be a pointer. It's a
pointer or it isn't, and type must correspond.

Ian

[Phil Noel]

Thanks Ian,

I thought this maybe the offical answer. I also couldn't see how Go could recognise this as a "not real" pointer.

Its unfortunate that the main windows header (windows.h) has this defined as a pointer and therefore cgo will treat it as a pointer. So even if I force my Go version of the structure (win.Msg) to use uintptr (ie win.HWnd) it will fail once it hits the cgo code (which will use the pointer based on the windows.h file ie *_Ctype_struct_HWND__). This is because my Go Msg structure is cast to the cgo version C.MSG (based on windows.h)

I think for my problem I will just create a mini version of windows.h that has only the functions I require. I can then pretend that MSG is an opaque pointer so cgo won't see inside it (therefore unable to check the fake pointer). Or I can wrap the troubling functions inside my own C code to hide the windows part of this.

I'm also a little surprised I could not find anyone else with this problem as TranslateMessage is pretty much a standard Windows GUI required function.

Thanks for your quick reply.

Phil

[Phil Noel]

As a FYI for anyone else who may have this problem.

After doing some more reading/searching another solution is to not use cgo at all. 

By using syscall.NewLazyDLL and dynamically loading the function, then with the loaded function just use func.Call(). This completely skips cgo and thus all the checks. Though of course you have to becareful to not accidently pass a Go pointer to a Go pointer.

Phil

[brainman]

On Wednesday, 30 August 2017 03:54:10 UTC+10, Phil Noel wrote:

 

By using syscall.NewLazyDLL and dynamically loading the function, then with the loaded function just use func.Call().

Yes, that is what everyone is doing, including Go standard library. You can also use syscall.Syscall, syscall.Syscall6 and others. For an example see %GOROOT%\src\syscall\zsyscall_windows.go.

Alex