sync/atomic 64-bit alignment

[enormouspenguin]

Hi,

In http://golang.org/pkg/sync/atomic/#pkg-note-BUG, I quoted:

"...The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned..."

If I have an array of structs, are the first words of each and every structs in the array automatically 64-bit aligned or just the first one? Do I need to know the size of a word for each build target in order to make it work correctly? If so, how can I programatically get the size?

Thanks very much!

[Matt Harden]

http://golang.org/pkg/unsafe/#Sizeof will tell you the size of a type. Also see AlignOf and OffsetOf. I don't think you can rely on every struct in a slice or array to be aligned. You can add padding where necessary to ensure alignment. Of course alignment isn't guaranteed to remain the same across different compilers or different versions of the same compiler.

My suggestion would be to split the 64 bit fields out into their own array/slice; then this will be 64-bit aligned on all platforms.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[enormouspenguin]

On Saturday, July 25, 2015 at 10:54:20 PM UTC+7, Matt Harden wrote:

http://golang.org/pkg/unsafe/#Sizeof will tell you the size of a type. Also see AlignOf and OffsetOf.

Thanks Matt, but I think you misread my question. I know about Sizeof, Alignof and Offsetof but what I want to know is how to find out the size of a *word*. Is it the same size as an int or uint var?

 

I don't think you can rely on every struct in a slice or array to be aligned. You can add padding where necessary to ensure alignment. Of course alignment isn't guaranteed to remain the same across different compilers or different versions of the same compiler.

The piece of doc I quoted is so confusing. Is a struct in an allocated array considered allocated struct? If so, should all the structs in the array have its first word 64-bit aligned? The confusion is killing me.

 

My suggestion would be to split the 64 bit fields out into their own array/slice; then this will be 64-bit aligned on all platforms.

By fields do you mean all the fields of primitive types only or fields that could be struct (on and on recursively)? If I have to split all the fields to get 64-bit aligned guarantee, then what would we need struct for? Couldn't rearrange (and add padding if necessary) inside a struct be enough?

[enormouspenguin]

Also, another source of my confusion: Is array and slice treated equally in the quoted doc?

[Dave Cheney]

> If I have an array of structs, are the first words of each and every structs in the array automatically 64-bit aligned or just the first one? 

Potentially just the first one, it depends if the size of each element, plus padding, is a multiple of 8.

> Do I need to know the size of a word for each build target in order to make it work correctly? 

Potentially, you may have to special case // +build arm, 386

> If so, how can I programatically get the size?

Would something like this work ? http://play.golang.org/p/SXfSY_3cig

[Ian Lance Taylor]

On Sat, Jul 25, 2015 at 11:10 AM, enormouspenguin <kimkh...@gmail.com> wrote:
> On Saturday, July 25, 2015 at 10:54:20 PM UTC+7, Matt Harden wrote:
>>
>> http://golang.org/pkg/unsafe/#Sizeof will tell you the size of a type.
>> Also see AlignOf and OffsetOf.
>
> Thanks Matt, but I think you misread my question. I know about Sizeof,
> Alignof and Offsetof but what I want to know is how to find out the size of
> a *word*. Is it the same size as an int or uint var?

Yes.

>> I don't think you can rely on every struct in a slice or array to be
>> aligned. You can add padding where necessary to ensure alignment. Of course
>> alignment isn't guaranteed to remain the same across different compilers or
>> different versions of the same compiler.
>
>
> The piece of doc I quoted is so confusing. Is a struct in an allocated array
> considered allocated struct? If so, should all the structs in the array have
> its first word 64-bit aligned? The confusion is killing me.

The quote is "The first word in a global variable or in an allocated
struct or slice can be relied upon to be 64-bit aligned." The key
phrase there is "allocated", not "struct or slice." The first word in
a struct or slice that is allocated via new or via a composite literal
will be 64-bit aligned. A struct inside a slice or array is not
allocated by itself, and no particular alignment is guaranteed beyond
that of unsafe.AlignOf.

Ian

[Ian Lance Taylor]

On Sat, Jul 25, 2015 at 11:17 AM, enormouspenguin <kimkh...@gmail.com> wrote:
> Also, another source of my confusion: Is array and slice treated equally in
> the quoted doc?

Yes: the first word of an allocated array will be aligned to a 64-bit boundary.

Ian

[enormouspenguin]

Just to make sure that I got it right in my mind:

+ If I have an array (or slice) of pointers to structs (all the pointers is not nil), does that means that I get the guarantee of 64-bit aligned first word for each and every referenced structs? Also, if unsafe.Sizeof(unsafe.Pointer(&v)) == 8 (64 bits), are all of the pointers in the array automatically 64-bit aligned too?

+ Could I rely on build tags to conditionally add necessary padding for specific target archs at compile-time? Such as:

file1.go:

// +build amd64

package main

type S struct {
        x int64
        y int //or uint or uintptr or unsafe.Pointer
}

var (
        s S //s.x is 64-bit aligned by default
        arrS [100]S //x in all array items are 64-bit aligned.
)

file2.go:

// +build 386 arm

package main

type S struct {

x int64

y int //or uint or uintptr or unsafe.Pointer

_ int32 //padding, could also be uint32, float32, rune, int, uint, uintptr or unsafe.Pointer

}

/*

Could I shorten it like this?

type S struct {

x int64

y, _ int

}

*/

var (

s S //s.x is 64-bit aligned by default

arrS [100]S //x in all array items are 64-bit aligned.

)

[Ian Lance Taylor]

On Sun, Jul 26, 2015 at 1:04 AM, enormouspenguin <kimkh...@gmail.com> wrote:
>
> + If I have an array (or slice) of pointers to structs (all the pointers is
> not nil), does that means that I get the guarantee of 64-bit aligned first
> word for each and every referenced structs?

Yes.

> Also, if
> unsafe.Sizeof(unsafe.Pointer(&v)) == 8 (64 bits), are all of the pointers in
> the array automatically 64-bit aligned too?

Yes. Ideally, though, you should check unsafe.Alignof.

> + Could I rely on build tags to conditionally add necessary padding for
> specific target archs at compile-time? Such as:

Yes.

> /*
> Could I shorten it like this?
>
> type S struct {
> x int64
> y, _ int
> }
> */
>
> var (
> s S //s.x is 64-bit aligned by default
> arrS [100]S //x in all array items are 64-bit aligned.
> )

On a 32-bit system, yes.

Ian

[Mateusz Czapliński]

On Saturday, July 25, 2015 at 4:44:36 PM UTC+2, enormouspenguin wrote:

In http://golang.org/pkg/sync/atomic/#pkg-note-BUG, I quoted:

"...The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned..."

@core-team: is this, or could it be, checked by `go vet`, taking into account all the slice/goarch/whatever-related special cases?

Thanks,
/M.

[Ian Lance Taylor]

I assume you are asking: could go vet check whether a value passed to
a sync/atomic function is not 64-bit aligned? The answer to that
question is yes. It could not be 100% reliable, of course, but it
could catch a number of cases. I suggest that you open an issue for
that at https://golang.org/issue. Thanks.

Ian

[Mateusz Czapliński]

Done: http://golang.org/issue/11891
Thanks
/M.

[enormouspenguin]

On Tuesday, July 28, 2015 at 3:10:31 AM UTC+7, Mateusz Czapliński wrote:

Done: http://golang.org/issue/11891
Thanks
/M.

Great job firing an issue. It's always better to have an automated check.

All this padding, aligning making me feel like I am in C or C++ world, although I know only a tiny bit about them. I think that's one of the beauties of Go - usually high and sometimes low enough to play without the fear that C or C++ bring.

[titarooss]

I'd like to see alignment directives in the language instead of padding, but I'm not sure this is possible. Alignment issues were brought up by a few clients, and my only answer was padding. Everyone of them complained and called it ugly but I had nothing for them. I'm not an expert in golang and I'm still looking for possible answers.

[Egon]

On Thursday, 30 July 2015 04:55:16 UTC+3, titarooss wrote:

I'd like to see alignment directives in the language instead of padding, but I'm not sure this is possible. Alignment issues were brought up by a few clients, and my only answer was padding. Everyone of them complained and called it ugly but I had nothing for them. I'm not an expert in golang and I'm still looking for possible answers.

At the moment you probably could create something like a vetter that checks whether fields have the correct alignment, e.g.

type Example struct {

}

[Egon]

type Example struct {

     X byte

     Y int64  // align to 64

}

And I don't think it's just alignment, it's about cache locality and the whole SOA vs AOS issues. "jai" had some interesting ideas with respect to it.