issues about struct align

[xjdrew]

Hello everyone,

Code as below, playground url(https://play.golang.org/p/XSx--6uF0E):

package main


import "fmt"
import "unsafe"


type A struct {
 a uint8
 b uint64
}
func main() {
 a := &A{}
 fmt.Println(unsafe.Sizeof(a))
 fmt.Println(unsafe.Sizeof(*a))
}

When I run the code in go playground, I get what as i expected.

4
16

Program exited.

but when I test the code in my own environment(linux i386 & win32, both), I get a unreasonable result as below:

$ go version 
go version go1.8.1 linux/386 
$ go run align.go
4 
12

Then I test a similar code in C, the result is same as go playground.

typedef struct _aaa { UINT8 a; UINT64 b; }AAA;

printf("%d",sizeof(int*)); // 4
printf("%d",sizeof(AAA)); // 16

uint64 should aligin in 8 bytes, why it aligns in 4 bytes in go1.8.1.linux-386?

Could anyone tell me how I can get the same result with go playground?

Best regards,

drew

[Jan Mercl]

On Fri, May 19, 2017 at 1:29 PM xjdrew <xj....@gmail.com> wrote:

> uint64 should aligin in 8 bytes, why it aligns in 4 bytes in go1.8.1.linux-386?

https://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C_structs_on_x86

> Could anyone tell me how I can get the same result with go playground?

Why should the playground output the same results when the GOARCH and/or GOOS differs?

--

-j

[Wojciech S. Czarnecki]

Dnia 2017-05-19, o godz. 04:28:53
xjdrew <xj....@gmail.com> napisał(a):

> uint64 should aligin in 8 bytes, why it aligns in 4 bytes in
> go1.8.1.linux-386?

uint64 member IS 8 bytes wide. It is aligned to the machine word boundary
which is 4B (32/8) for 32bit architecture and 8B for 64b one (64/8).

> Could anyone tell me how I can get the same result with go playground?

You can not. Go playground uses 64b platform.

--
Wojciech S. Czarnecki
^oo^ OHIR-RIPE

[xjdrew]

To Jan & Ohir, thanks for information.

Do you run my code example in GO plagground?  The pointer size of `a` is 4 bytes, it shows go playground use 32bit go. As my test, the go playground environment is:

GOOS: nacl 
GOARCH: amd64p32

the Machine is amd64, but run 32bit GO.

My Machine is amd64 too, I download this version of GO: https://storage.googleapis.com/golang/go1.8.1.windows-386.zip

I think my envvironment is similiar to GO playground.

Maybe my question should be: Keep the pointer size is 4 bytes, Which version of GO binary I shoud use to make uint64 aligns to 8 bytes?

Best Regards,

xjdrew

[xjdrew]

Here is my c code test:

drew@drew-PC MSYS /e/examples/c
$ cat align.c
#include <stdio.h>


typedef struct _A {
        char a;
        long long b;
} A;


int main() {
        printf("%d\n", sizeof(int*));
        printf("%d\n", sizeof(A));
}


drew@drew-PC MSYS /e/examples/c
$ gcc -o align align.c


drew@drew-PC MSYS /e/examples/c
$ ./align.exe
4
16

the gcc verison is:

$ gcc -v
Using built-in specs.
COLLECT_GCC=D:\msys64\mingw32\bin\gcc.exe
COLLECT_LTO_WRAPPER=D:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/6.3.0/lto-wrapper.exe
Target: i686-w64-mingw32

Best regards,

xjdrew

[peterGo]

xjdrew,

The Go Playground is (GOOS/GOARCH) nacl/amd64p32 which is a 64 bit architecture, where the address space is limited to a 4GB (32 bit) window. It has 8 byte (64 bit) data structure alignment.

linux/386 and windows/386 are 32 bit architectures, where the address space is 4GB (32 bit). They have 4 byte (32 bit) data structure alignment.

Peter

[xjdrew]

peterGo, thanks.

How can i use this kind of Go in windows? my machine is 64bit also. If I download the amd64 Go, the pointer size will be 8 bytes.

My real issue is , when I call win32 api, uint64 type in the struct of win32 is aligned to 8 bytes. 

I have to pad the go struct by manual to make it work with win32 api.

best regards,

xjdrew

[Dan Kortschak]

This smells like the XY problem. What are you actually trying to do and
how have to tried to do it (actual code, not simplified analogues)?

[Marvin Renich]

* xjdrew <xj....@gmail.com> [170522 02:06]:

> How can i use this kind of Go in windows? my machine is 64bit also. If I
> download the amd64 Go, the pointer size will be 8 bytes.
>
> My real issue is , when I call win32 api, uint64 type in the struct of
> win32 is aligned to 8 bytes.
> I have to pad the go struct by manual to make it work with win32 api.
>

> > On Friday, May 19, 2017 at 7:28:54 AM UTC-4, xjdrew wrote:
> >> Code as below, playground url(https://play.golang.org/p/XSx--6uF0E):
> >>
> >> package main
> >>
> >> import "fmt"
> >> import "unsafe"
> >>
> >> type A struct {
> >> a uint8
> >> b uint64
> >> }
> >> func main() {
> >> a := &A{}
> >> fmt.Println(unsafe.Sizeof(a))
> >> fmt.Println(unsafe.Sizeof(*a))
> >> }

As Dan said, you have not given us enough information to provide you
with useful help (at least not without a lot of good guessing on our
part as to what you are really trying to do). I am guessing that you
are importing "golang.org/x/sys/windows" and using syscall to call
functions in DLLs. If so, what DLL and what entry? What is the C
structure you are trying to pass to that entry? Must it work in both
win32 and win64?

When you are mixing Go and C ABIs, there is likely to be some manual
alignment, especially if you are trying to do this on multiple
OS/architecture combinations (e.g. win32 and win64).

I believe (but am not 100% positive) that on all platforms supported by
Go, the following will produce the same alignment:

type A struct {
a uint8

_ uint32
b uint64
}

...Marvin

[xjdrew]

Marvin, thanks.

What i'm doing is similar to your guessing.

I use syscall.Syscall to call win32 api, and encounter some padding issues.

I have resolve my problems by mannual padding days ago, but I want to know if there is a more graceful way to finish that.

One of the Apis is:

DWORD WINAPI FwpmFilterAdd0(
  _In_            HANDLE              engineHandle,
  _In_      const FWPM_FILTER0        *filter,
  _In_opt_        SECURITY_DESCRIPTOR sd,
  _Out_opt_       UINT64              *id
);

I have call with a FWPM_FILTER0 struct as below:

typedef struct FWPM_FILTER0_ {
  GUID                   filterKey;
  FWPM_DISPLAY_DATA0     displayData;
  UINT32                 flags;
  GUID                   *providerKey;
  FWP_BYTE_BLOB          providerData;
  GUID                   layerKey;
  GUID                   subLayerKey;
  FWP_VALUE0             weight;
  UINT32                 numFilterConditions;
  FWPM_FILTER_CONDITION0 *filterCondition;
  FWPM_ACTION0           action;
  union {
    UINT64 rawContext;                 // makes this union aligns to 8 bytes
    GUID   providerContextKey;
  };
  GUID                   *reserved;
  UINT64                 filterId;     // filterId aligns to 8 bytes
  FWP_VALUE0             effectiveWeight;
} FWPM_FILTER0;

In win32, I have to define the sturct as below:

type FWPM_FILTER0 struct {

    FilterKey GUID

    DisplayData FWPM_DISPLAY_DATA0

    Flags uint32

    ProviderKey *GUID

    ProviderData FWP_BYTE_BLOB

    LayerKey GUID

    SubLayerKey GUID

    Weight FWP_VALUE0

    NumFilterConditions uint32

    FilterCondition *FWPM_FILTER_CONDITION0

    Action FWPM_ACTION0

    /*

        union {

         UINT64 rawContext;

         GUID providerContextKey;

        };

    */

    reserved1 uint32 // mannul padding

    ProviderContextKey GUID // the max size field of that anonymous union

    Reserved *GUID

    reserved2 uint32 // mannul padding

    FilterId uint64

    EffectiveWeight FWP_VALUE0

}

Now I can run my code in 32bit go. But I hope my code can run 64bit go without any conditional compile.

best regards,

xjdrew

[Marvin Renich]

* xjdrew <xj....@gmail.com> [170523 02:31]:

I think manual padding is appropriate here. Note that the standard
library uses manual padding in syscall in a couple ztypes_*.go files.

Looking more closely at the MSDN docs for FwpmFilterAdd0 and
FWPM_FILTER0, if I were doing this, I might use «RawContext uint64»
instead of «ProviderContextKey GUID», which would be properly aligned
without manual padding. I would then convert this value manually to a
GUID if the appropriate flag were set in Flags. This would, of course,
depend on my expected use of that context field.

Also note that in Go, the idiomatic way to pad is with the blank
identifier, «_», as in «_ uint32», rather than using «reserved1 uint32».

...Marvin

[Marvin Renich]

* Marvin Renich <mr...@renich.org> [170523 08:58]:

> Looking more closely at the MSDN docs for FwpmFilterAdd0 and
> FWPM_FILTER0, if I were doing this, I might use «RawContext uint64»
> instead of «ProviderContextKey GUID», which would be properly aligned
> without manual padding. I would then convert this value manually to a
> GUID if the appropriate flag were set in Flags. This would, of course,
> depend on my expected use of that context field.

Of course, that would have to be a struct with two uint64 fields so that
it would be the correct size and have the full GUID.

...Marvin

[xjdrew]

Marvin, thanks.

It seems I have to keep my manual padding!

best regards,

xjdrew

[T L]

http://www.tapirgames.com/blog/golang-memory-alignment