About 64bits alignment

[T L]

the sync/atomic docs, https://golang.org/pkg/sync/atomic/, says in the end of the docs

On x86-32, the 64-bit functions use instructions unavailable before the Pentium MMX. 

On non-Linux ARM, the 64-bit functions use instructions unavailable before the ARMv6k core.

On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. 

The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned.

The last line says the first word in a global variable or in an allocated struct or slice is 64-bit aligned for sure.
But what does an allocated struct or slice means? A struct or slice allocated on heap, not stack?

[Jan Mercl]

On Wed, Feb 1, 2017 at 5:04 PM T L <tapi...@gmail.com> wrote:

> But what does an allocated struct or slice means? A struct or slice allocated on heap, not stack?

There's no heap nor stack from the POV of the language spec. Allocated means appropriate amount of storage is now used by the instance. Everything else is an implementation detail and has no influence on the semantics.

--

-j

[T L]

So allocated means addressable?
 

--

-j

[T L]

Why does WaitGroup.state method check 64-bit alignment at run time?
Why not make the state field as the first word of WaitGroup struct?

// https://golang.org/src/sync/waitgroup.go?s=1857:1892#L20

type WaitGroup struct {

	noCopy noCopy

	// 64-bit value: high 32 bits are counter, low 32 bits are waiter count.

	// 64-bit atomic operations require 64-bit alignment, but 32-bit

	// compilers do not ensure it. So we allocate 12 bytes and then use

	// the aligned 8 bytes in them as state.

	state1 [12]byte

	sema   uint32

}
 

func (wg *WaitGroup) state() *uint64 {

	if uintptr(unsafe.Pointer(&wg.state1))%8 == 0 {

		return (*uint64)(unsafe.Pointer(&wg.state1))

	} else {

		return (*uint64)(unsafe.Pointer(&wg.state1[4]))

	}

}

[Ian Lance Taylor]

On Fri, Feb 3, 2017 at 5:38 AM, T L <tapi...@gmail.com> wrote:
> Why does WaitGroup.state method check 64-bit alignment at run time?
> Why not make the state field as the first word of WaitGroup struct?
>
> // https://golang.org/src/sync/waitgroup.go?s=1857:1892#L20
>
> type WaitGroup struct {
>
> noCopy noCopy
>
> // 64-bit value: high 32 bits are counter, low 32 bits are waiter count.
>
> // 64-bit atomic operations require 64-bit alignment, but 32-bit
>
> // compilers do not ensure it. So we allocate 12 bytes and then use
>
> // the aligned 8 bytes in them as state.

Doesn't this comment explain the problem?

Ian

[Matt Harden]

Doesn't the statement "32-bit compilers to not ensure [64 bit alignment at the start of an allocation]" contradict the sync/atomic statement "The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned."?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Harden]

Never mind; I just realized that a WaitGroup is not necessarily at the start of an allocation or global variable.

[T L]

On Saturday, February 4, 2017 at 11:03:11 AM UTC+8, Matt Harden wrote:

Never mind; I just realized that a WaitGroup is not necessarily at the start of an allocation or global variable.

Not very get it.
Do you mean WaitGroup will be embedded in other types?
So we can't use the 64bit atomic functions for all exported types in libraries?
 

[Matt Harden]

https://play.golang.org/p/VWysBwbPCu

The above program prints 64-bit aligned addresses on the playground, but that's a 64 bit platform (with 32 bit pointers). On a 32 bit platform x.j is not guaranteed to be 64-bit aligned. The same would be true if we substituted a WaitGroup for int64 in that example.

[Axel Wagner]

On Sat, Feb 4, 2017 at 5:49 AM, T L <tapi...@gmail.com> wrote:

On Saturday, February 4, 2017 at 11:03:11 AM UTC+8, Matt Harden wrote:

Never mind; I just realized that a WaitGroup is not necessarily at the start of an allocation or global variable.

Not very get it.
Do you mean WaitGroup will be embedded in other types?
So we can't use the 64bit atomic functions for all exported types in libraries?

You can, but must make sure that they are aligned properly. Either like WaitGroup does, or, for example, by using the common idiom of factory functions that do the allocation (or by just putting it on the user to make sure).

 

 

On Fri, Feb 3, 2017 at 7:01 PM Matt Harden <matt....@gmail.com> wrote:

Doesn't the statement "32-bit compilers to not ensure [64 bit alignment at the start of an allocation]" contradict the sync/atomic statement "The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned."?

On Fri, Feb 3, 2017 at 6:44 AM Ian Lance Taylor <ia...@golang.org> wrote:

On Fri, Feb 3, 2017 at 5:38 AM, T L <tapi...@gmail.com> wrote:
> Why does WaitGroup.state method check 64-bit alignment at run time?
> Why not make the state field as the first word of WaitGroup struct?
>
> // https://golang.org/src/sync/waitgroup.go?s=1857:1892#L20
>
> type WaitGroup struct {
>
> noCopy noCopy
>
> // 64-bit value: high 32 bits are counter, low 32 bits are waiter count.
>
> // 64-bit atomic operations require 64-bit alignment, but 32-bit
>
> // compilers do not ensure it. So we allocate 12 bytes and then use
>
> // the aligned 8 bytes in them as state.

Doesn't this comment explain the problem?

Ian

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[T L]

On Saturday, February 4, 2017 at 2:51:53 PM UTC+8, Axel Wagner wrote:

On Sat, Feb 4, 2017 at 5:49 AM, T L <tapi...@gmail.com> wrote:

On Saturday, February 4, 2017 at 11:03:11 AM UTC+8, Matt Harden wrote:

Never mind; I just realized that a WaitGroup is not necessarily at the start of an allocation or global variable.

Not very get it.
Do you mean WaitGroup will be embedded in other types?
So we can't use the 64bit atomic functions for all exported types in libraries?

You can, but must make sure that they are aligned properly. Either like WaitGroup does, or, for example, by using the common idiom of factory functions that do the allocation (or by just putting it on the user to make sure).

Get it. This is quite bug prone to write programs on 64bit OSes and the programs are expected to run on 32bit OSes too.
 

[T L]

Part of. Ok I get it now.

BTW, I have another question, can an allocated local int64 value be relied upon to be 64-bit aligned?

 

[T L]

By search the usages of 64bit functions in atomic package in go source, I think the answer should be yes.
And the go source also imply any 8-byte sized fields, and the fields followed 8-byte sized fields, in any struct are also relied upon to be 64-bit aligned.
 

 

[T L]

If this is true, why not define WaitGroup.state as uint64 directly?

The following is the code in go src which thinks 8-byte sized fields in structs are also 64-bit aligned.

// syscall/net_nacl.go

type netFile struct {
    defaultFileImpl
    proto      *netproto
    sotype     int
    listener   *msgq
    packet     *msgq
    rd         *byteq
    wr         *byteq
    rddeadline int64
    wrdeadline int64
    addr       Sockaddr
    raddr      Sockaddr
}
...

func SetReadDeadline(fd int, t int64) error {
    f, err := fdToNetFile(fd)
    if err != nil {
        return err
    }
    atomic.StoreInt64(&f.rddeadline, t)
    return nil
}
...

func SetWriteDeadline(fd int, t int64) error {
    f, err := fdToNetFile(fd)
    if err != nil {
        return err
    }
    atomic.StoreInt64(&f.wrdeadline, t)
    return nil
}

 

 

 

[T L]

On Saturday, February 4, 2017 at 2:18:44 PM UTC+8, Matt Harden wrote:

https://play.golang.org/p/VWysBwbPCu

The above program prints 64-bit aligned addresses on the playground, but that's a 64 bit platform (with 32 bit pointers). On a 32 bit platform x.j is not guaranteed to be 64-bit aligned. The same would be true if we substituted a WaitGroup for int64 in that example.

The go source shown in my last comment doesn't think so.
 

[Axel Wagner]

The spec says this: https://golang.org/ref/spec#Size_and_alignment_guarantees

Now, I can't really read from this whether this applies to fields or not. If it does, it would seem to me, that, indeed, WaitGroup wouldn't need to use the trick it does.

For the example you name: It isn't particularly telling, as on nacl (at least on x86{,-64}, don't know a lot about ARM) pointers have 32 bits and so does int, meaning both rddeadline and wrdeadline are indeed 64-bit aligned.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[T L]

On Saturday, February 4, 2017 at 6:32:52 PM UTC+8, Axel Wagner wrote:

The spec says this: https://golang.org/ref/spec#Size_and_alignment_guarantees

Now, I can't really read from this whether this applies to fields or not. If it does, it would seem to me, that, indeed, WaitGroup wouldn't need to use the trick it does.

For the example you name: It isn't particularly telling, as on nacl (at least on x86{,-64}, don't know a lot about ARM) pointers have 32 bits and so does int, meaning both rddeadline and wrdeadline are indeed 64-bit aligned.

Yes, there are 6 4-byte fields before the two 8-byte fields.

Ok, then another question, is it safe to embed expvar.Float type in 32bit OSes?
https://golang.org/src/expvar/expvar.go?s=1577:1608#L55
 

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Lars Seipel]

On Sat, Feb 04, 2017 at 01:30:49AM -0800, T L wrote:
> Get it. This is quite bug prone to write programs on 64bit OSes and the
> programs are expected to run on 32bit OSes too.

Well, there's a reason the sync/atomic package docs have this statement
right at the top:

> These functions require great care to be used correctly. Except for special,
> low-level applications, synchronization is better done with channels or the
> facilities of the sync package.

[Axel Wagner]

I find this an inappropriate answer. Yes, they need care, but it should at least be possible to figure out how to use them from reading documentation (carefully). I tend to agree, that neither the spec nor the documentation of the atomic package is particularly helpful about these questions; at least I fail to read them in a way that unambiguously answers them.

Without an authoritative and unambiguous description about what is guaranteed it is impossible to implement this robustly and if the spec isn't sufficiently unambiguous, it's natural to ask questions. Saying "you are not supposed to use them, unless you know how to use them" is, in context of all of this, just shutting people out.

FWIW, I read the spec, Dave Cheney's blog post about padding, the therein referenced #599 and the atomic-documentation, I find the whole situation still somewhat confusing. To try to interpret all of this and answer OPs question:

I believe, that fields do not necessarily need to be properly aligned (explaining 599). However, *structs* have a necessary alignment, as required by their fields. So, if you do

type S struct {

    A uint32

    B uint64

}

then there is no guarantee, that B is 64-bit aligned. The spec tells us, that unsafe.Alignof(s.A) will return 4 ("The functions Alignof and Sizeof take an expression x of any type and return the alignment or size, respectively, of a hypothetical variable v as if v was declared via var v = x", s.A is of type uint32 and such a variable has alignment 4), unsafe.Alignof(s.B) will return 8 and thus, unsafe.Alignof(s) will return 8 ("For a variable x of struct type: unsafe.Alignof(x) is the largest of all the values unsafe.Alignof(x.f) for each field f of x, but at least 1").

This means, that the compiler needs to align S on 64 bits. Now, #599 tells us, that on some archs s.B will *not* be 64 bit aligned. This is in accordance with the spec though, as confusing as that seems; the spec says only, how a variable of the same type as s.B must be aligned, not how s.B itself must be aligned (this is, where the confusion stems from). So a compiler might or might not insert padding.

What all of this means, I believe, is

* The atomic-documentation (maybe?) tells you, that the first field in a struct can be relied upon to be aligned. It says "the first word", though, and from what I can tell, there is nothing preventing a compiler to insert arbitrary padding at the beginning of a struct, technically speaking.

* Thus you can guarantee that s.B is 8-byte aligned, by switching the fields around

* Thus, your question about the embedding of expvar.Float seems to be "yes, it's safe", as the struct itself must be 64-bit aligned, even if embedded, and the 64-bit field is the only one contained

* Your question about WaitGroup seems to be answerable by referring to the fact, that technically the noCopy field could misalign the data. But I tend to agree, that this reading would allow us to use a uint64, if we switch noCopy and state. Dave Cheney's blog post about padding wouldn't create technical problems here, because noCopy isn't the last field, thus gets no storage allocated…

* Also, all of that reading seems to suggest that there really is no reliable, guaranteed way, to have multiple fields with a guaranteed alignment. So, if you need multiple variables to be accessed by atomics, you are out of luck.

Now, all of that being said, my reading could also be wrong. In any case, in my opinion, the fact that there is considerable guesswork involved, seems to suggest, that the spec is either ambiguous or insufficient to actually allow safe usage of atomics. It would be nice, if it could be a bit more explicit about what happens to fields. I understand, though, that we don't want to restrict too much the flexibility of a compiler to insert padding. But maybe a guarantee about the first field (not "word", what does that even mean?) of a struct being aligned, even the first fields (plural), if they all have the same alignment as the struct, would be clear enough *and* compatible with what compilers are already doing.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Matt Harden]

On Sat, Feb 4, 2017 at 7:34 AM 'Axel Wagner' via golang-nuts <golan...@googlegroups.com> wrote:

I find this an inappropriate answer. Yes, they need care, but it should at least be possible to figure out how to use them from reading documentation (carefully). I tend to agree, that neither the spec nor the documentation of the atomic package is particularly helpful about these questions; at least I fail to read them in a way that unambiguously answers them.

Without an authoritative and unambiguous description about what is guaranteed it is impossible to implement this robustly and if the spec isn't sufficiently unambiguous, it's natural to ask questions. Saying "you are not supposed to use them, unless you know how to use them" is, in context of all of this, just shutting people out.

FWIW, I read the spec, Dave Cheney's blog post about padding, the therein referenced #599 and the atomic-documentation, I find the whole situation still somewhat confusing. To try to interpret all of this and answer OPs question:

I believe, that fields do not necessarily need to be properly aligned (explaining 599). However, *structs* have a necessary alignment, as required by their fields. So, if you do

type S struct {

    A uint32

    B uint64

}

then there is no guarantee, that B is 64-bit aligned. The spec tells us, that unsafe.Alignof(s.A) will return 4 ("The functions Alignof and Sizeof take an expression x of any type and return the alignment or size, respectively, of a hypothetical variable v as if v was declared via var v = x", s.A is of type uint32 and such a variable has alignment 4), unsafe.Alignof(s.B) will return 8 and thus, unsafe.Alignof(s) will return 8 ("For a variable x of struct type: unsafe.Alignof(x) is the largest of all the values unsafe.Alignof(x.f) for each field f of x, but at least 1").

This means, that the compiler needs to align S on 64 bits. Now, #599 tells us, that on some archs s.B will *not* be 64 bit aligned. This is in accordance with the spec though, as confusing as that seems; the spec says only, how a variable of the same type as s.B must be aligned, not how s.B itself must be aligned (this is, where the confusion stems from). So a compiler might or might not insert padding.

What all of this means, I believe, is

* The atomic-documentation (maybe?) tells you, that the first field in a struct can be relied upon to be aligned. It says "the first word", though, and from what I can tell, there is nothing preventing a compiler to insert arbitrary padding at the beginning of a struct, technically speaking.

* Thus you can guarantee that s.B is 8-byte aligned, by switching the fields around

* Thus, your question about the embedding of expvar.Float seems to be "yes, it's safe", as the struct itself must be 64-bit aligned, even if embedded, and the 64-bit field is the only one contained

Wait, I don't think you've explained why expvar.Float would be aligned even if embedded in another struct. I don't think that's guaranteed. The size and alignment guarantees in the spec only make guarantees about variables, not struct fields (even if they are themselves structs).

 

* Your question about WaitGroup seems to be answerable by referring to the fact, that technically the noCopy field could misalign the data. But I tend to agree, that this reading would allow us to use a uint64, if we switch noCopy and state.

I don't agree with this either. Why do you think that structs embedded in other structs are guaranteed to be aligned?

 

Dave Cheney's blog post about padding wouldn't create technical problems here, because noCopy isn't the last field, thus gets no storage allocated…

* Also, all of that reading seems to suggest that there really is no reliable, guaranteed way, to have multiple fields with a guaranteed alignment. So, if you need multiple variables to be accessed by atomics, you are out of luck.

Here I disagree again. As long as all fields in a struct variable before yours are n*64 bits wide, you can be sure it will be 64 bit aligned. Perhaps this needs to be documented. Also, doesn't this contradict your statements above that structs embedded in other structs have alignment guarantees?

Now, all of that being said, my reading could also be wrong. In any case, in my opinion, the fact that there is considerable guesswork involved, seems to suggest, that the spec is either ambiguous or insufficient to actually allow safe usage of atomics. It would be nice, if it could be a bit more explicit about what happens to fields.

I agree! :-)

 

I understand, though, that we don't want to restrict too much the flexibility of a compiler to insert padding. But maybe a guarantee about the first field (not "word", what does that even mean?) of a struct being aligned, even the first fields (plural), if they all have the same alignment as the struct, would be clear enough *and* compatible with what compilers are already doing.

On Sat, Feb 4, 2017 at 3:31 PM, Lars Seipel <lars....@gmail.com> wrote:

On Sat, Feb 04, 2017 at 01:30:49AM -0800, T L wrote:
> Get it. This is quite bug prone to write programs on 64bit OSes and the
> programs are expected to run on 32bit OSes too.

Well, there's a reason the sync/atomic package docs have this statement
right at the top:

> These functions require great care to be used correctly. Except for special,
> low-level applications, synchronization is better done with channels or the
> facilities of the sync package.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Ian Lance Taylor]

On Sat, Feb 4, 2017 at 7:33 AM, Axel Wagner
<axel.wa...@googlemail.com> wrote:
>
> I believe, that fields do not necessarily need to be properly aligned
> (explaining 599). However, *structs* have a necessary alignment, as required
> by their fields. So, if you do
> type S struct {
> A uint32
> B uint64
> }
> then there is no guarantee, that B is 64-bit aligned. The spec tells us,
> that unsafe.Alignof(s.A) will return 4 ("The functions Alignof and Sizeof
> take an expression x of any type and return the alignment or size,
> respectively, of a hypothetical variable v as if v was declared via var v =
> x", s.A is of type uint32 and such a variable has alignment 4),
> unsafe.Alignof(s.B) will return 8 and thus, unsafe.Alignof(s) will return 8
> ("For a variable x of struct type: unsafe.Alignof(x) is the largest of all
> the values unsafe.Alignof(x.f) for each field f of x, but at least 1").

The spec does not say that unsafe.Alignof(s.B) will return 8. In
fact, on 32-bit targets, with the gc toolchain, it will return 4.

I think the spec and docs do provide enough information to use
sync/atomic correctly. "The first word in a global variable or in an
allocated struct or slice can be relied upon to be 64-bit aligned."
That is sufficient to ensure that your code works correctly. It does
mean that certain kinds of operations don't work. sync.WaitGroup
plays the games it does to avoid introducing an extra allocation.

There is probably something to be fixed here, to permit higher
efficiency without requiring trickiness like that of sync.WaitGroup,
but I'm not sure how it should be implemented.

Ian

[Axel Wagner]

Basically, to most of your disagreements: I'm wildly guessing, trying to make sense of the information I do have :) Just one specific answer:

On Sat, Feb 4, 2017 at 11:05 PM, Matt Harden <matt....@gmail.com> wrote:

Wait, I don't think you've explained why expvar.Float would be aligned even if embedded in another struct. I don't think that's guaranteed. The size and alignment guarantees in the spec only make guarantees about variables, not struct fields (even if they are themselves structs).

My first point: I agree with you about the spec. I'm citing the documentation for sync/atomic, which appears to make a stronger guarantee than the spec. Most of my other points are derived from that. But I do agree, that it's a liberal reading. Basically, I'm trying to *somehow* derive something useful about fields from somewhere.

[Axel Wagner]

On Sat, Feb 4, 2017 at 11:13 PM, Ian Lance Taylor <ia...@golang.org> wrote:

On Sat, Feb 4, 2017 at 7:33 AM, Axel Wagner
<axel.wa...@googlemail.com> wrote:
>
> I believe, that fields do not necessarily need to be properly aligned
> (explaining 599). However, *structs* have a necessary alignment, as required
> by their fields. So, if you do
> type S struct {
>     A uint32
>     B uint64
> }
> then there is no guarantee, that B is 64-bit aligned. The spec tells us,
> that unsafe.Alignof(s.A) will return 4 ("The functions Alignof and Sizeof
> take an expression x of any type and return the alignment or size,
> respectively, of a hypothetical variable v as if v was declared via var v =
> x", s.A is of type uint32 and such a variable has alignment 4),
> unsafe.Alignof(s.B) will return 8 and thus, unsafe.Alignof(s) will return 8
> ("For a variable x of struct type: unsafe.Alignof(x) is the largest of all
> the values unsafe.Alignof(x.f) for each field f of x, but at least 1").

The spec does not say that unsafe.Alignof(s.B) will return 8.  In
fact, on 32-bit targets, with the gc toolchain, it will return 4.

To me, that seems to contradict the combination of "The functions Alignof and Sizeof take an expression x of any type and return the alignment or size, respectively, of a hypothetical variable v as if v was declared via var v = x" and the table at the very bottom, saying an uint64 has Alignof 8. The latter seems to strongly imply that a "var v = s.B" must be 8-byte aligned, as v is a variable (even an allocated one; one of the few mentions of "allocate" in the spec :) ) of type uint64. Then, the former would imply that the same happens to s.B, as that is how the function is defined.

I think the spec and docs do provide enough information to use
sync/atomic correctly.  "The first word in a global variable or in an
allocated struct or slice can be relied upon to be 64-bit aligned."

In that context, the original question in this thread was "what does 'allocated' mean and I find that more and more a fair question" :)

 

That is sufficient to ensure that your code works correctly.  It does
mean that certain kinds of operations don't work.  sync.WaitGroup
plays the games it does to avoid introducing an extra allocation.

I don't understand. Where is the extra allocation, if you where to change state to be an uint64 and just dereference it, instead of using unsafe.Pointer casting in state()?

Am I understanding it correctly that you are saying a) yes, fields like, say expvar.Float can not be embedded, unless you take special care (recursively) of having it aligned and b) the only safe way to use atomic with fields would be, to put a pointer in and allocate with new (and that's the "extra allocation" WaitGroup is trying to avoid?). 

I would find that situation understandable, but also pretty unfortunate. It seems very easy to make a mistake that way (embedding a type, not knowing that it relies on atomics). It also means, it's harder to make the zero value of a struct useful; while having a uint64 intiialized to zero is a perfectly fine default for a lot of cases, needing to allocate it with new requires a constructor.

I would hope we could at least a) recommend that (and maybe the WaitGroup trick) as "the" ways to use atomic somehow and b) have some kind of check to detect misuse. The memory layout of all structs is statically known, so it should be theoretically possible to detect usages of atomic.AddUint64(&s.f) (or the like) with a misaligned field, no?

[Ian Lance Taylor]

On Sat, Feb 4, 2017 at 2:38 PM, Axel Wagner
<axel.wa...@googlemail.com> wrote:
> On Sat, Feb 4, 2017 at 11:13 PM, Ian Lance Taylor <ia...@golang.org> wrote:
>>
>> The spec does not say that unsafe.Alignof(s.B) will return 8. In
>> fact, on 32-bit targets, with the gc toolchain, it will return 4.
>
>
> To me, that seems to contradict the combination of "The functions Alignof
> and Sizeof take an expression x of any type and return the alignment or
> size, respectively, of a hypothetical variable v as if v was declared via
> var v = x" and the table at the very bottom, saying an uint64 has Alignof 8.
> The latter seems to strongly imply that a "var v = s.B" must be 8-byte
> aligned, as v is a variable (even an allocated one; one of the few mentions
> of "allocate" in the spec :) ) of type uint64. Then, the former would imply
> that the same happens to s.B, as that is how the function is defined.

The table at the very bottom is only type sizes, not type alignments.

>> That is sufficient to ensure that your code works correctly. It does
>> mean that certain kinds of operations don't work. sync.WaitGroup
>> plays the games it does to avoid introducing an extra allocation.
>
>
> I don't understand. Where is the extra allocation, if you where to change
> state to be an uint64 and just dereference it, instead of using
> unsafe.Pointer casting in state()?
>
> Am I understanding it correctly that you are saying a) yes, fields like, say
> expvar.Float can not be embedded, unless you take special care (recursively)
> of having it aligned and b) the only safe way to use atomic with fields
> would be, to put a pointer in and allocate with new (and that's the "extra
> allocation" WaitGroup is trying to avoid?).
>
> I would find that situation understandable, but also pretty unfortunate. It
> seems very easy to make a mistake that way (embedding a type, not knowing
> that it relies on atomics). It also means, it's harder to make the zero
> value of a struct useful; while having a uint64 intiialized to zero is a
> perfectly fine default for a lot of cases, needing to allocate it with new
> requires a constructor.

Yes.

> I would hope we could at least a) recommend that (and maybe the WaitGroup
> trick) as "the" ways to use atomic somehow and b) have some kind of check to
> detect misuse. The memory layout of all structs is statically known, so it
> should be theoretically possible to detect usages of atomic.AddUint64(&s.f)
> (or the like) with a misaligned field, no?

Sounds like a good cmd/vet check.

Ian

[T L]

Ian, thanks for the answers.

But I still not very confirm on many points.

Up to now, there are two places mention the alignments in Go.

The first is in the end of Go spec:
 

Size and alignment guarantees

For the numeric types, the following sizes are guaranteed:

type                                 size in bytes

byte, uint8, int8                     1
uint16, int16                         2
uint32, int32, float32                4
uint64, int64, float64, complex64     8
complex128                           16

The following minimal alignment properties are guaranteed:

1. For a variable x of any type: unsafe.Alignof(x) is at least 1.
2. For a variable x of struct type: unsafe.Alignof(x) is the largest of all the values unsafe.Alignof(x.f) for each field f of x, but at least 1.
3. For a variable x of array type: unsafe.Alignof(x) is the same as unsafe.Alignof(x[0]), but at least 1.

A struct or array type has size zero if it contains no fields (or elements, respectively) that have a size greater than zero. Two distinct zero-size variables may have the same address in memory.

The second is at the end of sync/atomic docs:

On x86-32, the 64-bit functions use instructions unavailable before the Pentium MMX.

On non-Linux ARM, the 64-bit functions use instructions unavailable before the ARMv6k core.

On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned.

I feel the two are not enough to remove all my confusions.

So could you help me remove the following confusions:

1. What does the "allocated struct or slice" mean?

Currently, I think it means the structs or slices created by new, or the structs or slices escape to heap.

Is my understanding right?

2. Are local 8-bytes variables 64bit aligned on 32bit OSes?

I found there are many usages of the 64bit functions of atomic package being used on local 8-bytes variables in go source.

So I think the answer is yes. Right?

3. Are expvar.Int and expvar.Float safe to be embedded in other structs on 32bit OSes?

I think the answer is no. Is my opinion right?

[Ian Lance Taylor]

Those cases are "allocated struct or slice," yes. The phrase also
includes variables defined with a struct or slice type.

> 2. Are local 8-bytes variables 64bit aligned on 32bit OSes?
>
>
> I found there are many usages of the 64bit functions of atomic package being
> used on local 8-bytes variables in go source.
>
> So I think the answer is yes. Right?

Yes.

> 3. Are expvar.Int and expvar.Float safe to be embedded in other structs on
> 32bit OSes?
>
>
> I think the answer is no. Is my opinion right?

You could embed them as the first field of a struct (if you were then
careful to not embed that struct, (except as the first field)).

It would not be portable to embed them as anything other than the first field.

I think this is problematic, and it would be nice to figure out a way to fix it.

Ian

[T L]

Thanks! I think I almost get it.

In short, when structs and slices are used as the non-first fields of other structs,
then the struct and slice fields may be not 64bit aligned on 32bit OSes.

But what about arrays? Can the first word in an allocated array be relied upon to be 64-bit aligned?

[T L]

In fact, I still have another question needs to be clarify: what does the "word" in "The first word in a global variable or in an allocated struct or slice" mean?
A field with the same length with int values?
 

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 11:55 AM, T L <tapi...@gmail.com> wrote:
>
> In short, when structs and slices are used as the non-first fields of other
> structs,
> then the struct and slice fields may be not 64bit aligned on 32bit OSes.
>
> But what about arrays? Can the first word in an allocated array be relied
> upon to be 64-bit aligned?

Yes.

Ian

[Ian Lance Taylor]

On Sun, Feb 5, 2017 at 8:15 PM, T L <tapi...@gmail.com> wrote:
>
> In fact, I still have another question needs to be clarify: what does the
> "word" in "The first word in a global variable or in an allocated struct or
> slice" mean?
> A field with the same length with int values?

In this case it means a 64-bit value stored in memory, as mentioned in
the previous sentence.

Ian

[T L]

Thanks for the clarification!

[T L]

I installed a 32bit VM and found that local int64 and [N]int64 variables are not guaranteed to be 64bit aligned.
But the magic is, if the local int64 variables are passed to atomic 64bit functions, then they become 64bit aligned for sure.
Quite magic.

It looks when local int64 variables are escaped to heap if their addresses are passed to atomic 64bit functions.
And it looks 64bit words allocated on heap are always 64bit aligned, even on 32bit OSes.

 

[T L]

The same is for local structs which first field is a 64bit word. Such local structs are also not guaranteed to be 64bit aligned.
But if the addresses of the 64bit word fields are passed to atomic 64bit functions, then the local structs will escape to heap,
so the local structs become 64bit aligned on heap.

 

[T L]

The same for local slices.

The full code:

// go version go1.7.5 linux/386

package main

import (
    "sync/atomic"
)

func fa() {
    println("========== fa")
    var b bool
    var arr [5]int64
    println(&b, &arr) // 0x1843bf73 0x1843bf74
}

func fa2() {
    println("========== fa2")
    var b bool
    var arr [5]int64
    println(&b, &arr) // 0x1843bf57 0x184120f0
    atomic.LoadInt64(&arr[0])
}

func fb() {
    println("========== fb")
    var b bool
    var x = new(int64)
    println(&b, &x) // 0x1843bf73 0x1843bf74
}

func fb2() {
    println("========== fb2")
    var b bool
    var x int64
    println(&b, &x) // 0x1843bf57 0x184120f0
    atomic.LoadInt64(&x)
}

func fc() {
    println("========== fc")
    var b bool
    var slc = make([]int64, 1)
    println(&b, &slc[0]) // 0x1843bf5b 0x1843bf5c
}

func fc2() {
    println("========== fc2")
    var b bool
    var slc = make([]int64, 1)
    println(&b, &slc[0]) // 0x1843bf37 0x1840e0e0
    atomic.LoadInt64(&slc[0])
}

func fd() {
    type T struct {
        x int64
    }
    println("========== fd")
    var b bool
    var t T
    println(&b, &t.x) // 0x1843bf33 0x1843bf34
}

func fd2() {
    type T struct {
        x int64
    }
    println("========== fd2")
    var b bool
    var t T
    println(&b, &t.x) // 0x1843bf37 0x1840e0f0
    atomic.LoadInt64(&t.x)
}

func fd3() {
    type T struct {
        x int64
    }
    println("========== fd3")
    var b bool
    var t = new(T)
    println(&b, &t.x) // 0x1843bf33 0x1843bf34
}

func main() {
    fa()
    fa2()

    fb()
    fb2()

    fc()
    fc2()

    fd()
    fd2()
    fd3()
}

 

[T L]

On Thursday, February 2, 2017 at 12:03:59 AM UTC+8, T L wrote:

the sync/atomic docs, https://golang.org/pkg/sync/atomic/, says in the end of the docs

On x86-32, the 64-bit functions use instructions unavailable before the Pentium MMX. 

On non-Linux ARM, the 64-bit functions use instructions unavailable before the ARMv6k core.

On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. 

Does the "ARM" here include ARMv8 (64-bit)?
 

The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned.

The last line says the first word in a global variable or in an allocated struct or slice is 64-bit aligned for sure.
But what does an allocated struct or slice means? A struct or slice allocated on heap, not stack?

[Michael Hudson-Doyle]

On 29 March 2017 at 04:44, T L <tapi...@gmail.com> wrote:

On Thursday, February 2, 2017 at 12:03:59 AM UTC+8, T L wrote:

the sync/atomic docs, https://golang.org/pkg/sync/atomic/, says in the end of the docs

On x86-32, the 64-bit functions use instructions unavailable before the Pentium MMX. 

On non-Linux ARM, the 64-bit functions use instructions unavailable before the ARMv6k core.

On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. 

Does the "ARM" here include ARMv8 (64-bit)?

On a 64 bit platform, 64-bit integers are naturally aligned on 64 bits so you shouldn't have to make any extra effort to arrange for it to happen.

Cheers,

mwh 

The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned.

The last line says the first word in a global variable or in an allocated struct or slice is 64-bit aligned for sure.
But what does an allocated struct or slice means? A struct or slice allocated on heap, not stack?

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Dave Cheney]

Arm means arm as in linux/arm.

[T L]

On Wednesday, March 29, 2017 at 5:04:26 AM UTC+8, Dave Cheney wrote:

Arm means arm as in linux/arm.

so WinRT is not supported?
but will?

[Dave Cheney]

I don't know of anyone working on windows/arm.

[T L]

It looks the latest gc (1.10) always makes 64-bit words aligned on i386 OSes.
Can any Go team member confirm this?
 

[T L]

I mean "always makes 64-bit words 64-bit aligned on i386 OSes."
 

[Jan Mercl]

On Thu, Mar 15, 2018 at 3:29 PM T L <tapi...@gmail.com> wrote:

> I mean "always makes 64-bit words 64-bit aligned on i386 OSes."

AFAIK, that's not the case, ie. not always.

--

-j

[T L]

But it looks the results from Go playground (which is a 32-bit OS env)
show all 64-bit words are 64-bit aligned on 32-bit OS.

https://play.golang.org/p/J2TT4yskRQT

[Ian Lance Taylor]

The playground currently runs in the amd64p32 environment, which is to
say with a 64-bit CPU with 32-bit pointers. This is done in order to
use NativeClient as a sandbox technology. Try running your program on
real hardware with GOARCH=386.

Ian

[T L]

Thanks for the clarification.

[john...@gmail.com]

I still can't find anything "authoritative and unambiguous" on this.  Therefore apps are going to be released with bugs in them.  Can someone please say specifically when it is _not_ safe to use 64-bit atomic operations?

When it is unsafe on a 64-bit OS?

When it is unsafe on a 32-bit OS?

John

On Sunday, February 5, 2017 at 4:34:05 AM UTC+13, Axel Wagner wrote:

I find this an inappropriate answer. Yes, they need care, but it should at least be possible to figure out how to use them from reading documentation (carefully). I tend to agree, that neither the spec nor the documentation of the atomic package is particularly helpful about these questions; at least I fail to read them in a way that unambiguously answers them.

Without an authoritative and unambiguous description about what is guaranteed it is impossible to implement this robustly and if the spec isn't sufficiently unambiguous, it's natural to ask questions. Saying "you are not supposed to use them, unless you know how to use them" is, in context of all of this, just shutting people out.

FWIW, I read the spec, Dave Cheney's blog post about padding, the therein referenced #599 and the atomic-documentation, I find the whole situation still somewhat confusing. To try to interpret all of this and answer OPs question:

I believe, that fields do not necessarily need to be properly aligned (explaining 599). However, *structs* have a necessary alignment, as required by their fields. So, if you do

type S struct {

    A uint32

    B uint64

}

then there is no guarantee, that B is 64-bit aligned. The spec tells us, that unsafe.Alignof(s.A) will return 4 ("The functions Alignof and Sizeof take an expression x of any type and return the alignment or size, respectively, of a hypothetical variable v as if v was declared via var v = x", s.A is of type uint32 and such a variable has alignment 4), unsafe.Alignof(s.B) will return 8 and thus, unsafe.Alignof(s) will return 8 ("For a variable x of struct type: unsafe.Alignof(x) is the largest of all the values unsafe.Alignof(x.f) for each field f of x, but at least 1").

This means, that the compiler needs to align S on 64 bits. Now, #599 tells us, that on some archs s.B will *not* be 64 bit aligned. This is in accordance with the spec though, as confusing as that seems; the spec says only, how a variable of the same type as s.B must be aligned, not how s.B itself must be aligned (this is, where the confusion stems from). So a compiler might or might not insert padding.

What all of this means, I believe, is

* The atomic-documentation (maybe?) tells you, that the first field in a struct can be relied upon to be aligned. It says "the first word", though, and from what I can tell, there is nothing preventing a compiler to insert arbitrary padding at the beginning of a struct, technically speaking.

* Thus you can guarantee that s.B is 8-byte aligned, by switching the fields around

* Thus, your question about the embedding of expvar.Float seems to be "yes, it's safe", as the struct itself must be 64-bit aligned, even if embedded, and the 64-bit field is the only one contained

* Your question about WaitGroup seems to be answerable by referring to the fact, that technically the noCopy field could misalign the data. But I tend to agree, that this reading would allow us to use a uint64, if we switch noCopy and state. Dave Cheney's blog post about padding wouldn't create technical problems here, because noCopy isn't the last field, thus gets no storage allocated…

* Also, all of that reading seems to suggest that there really is no reliable, guaranteed way, to have multiple fields with a guaranteed alignment. So, if you need multiple variables to be accessed by atomics, you are out of luck.

Now, all of that being said, my reading could also be wrong. In any case, in my opinion, the fact that there is considerable guesswork involved, seems to suggest, that the spec is either ambiguous or insufficient to actually allow safe usage of atomics. It would be nice, if it could be a bit more explicit about what happens to fields. I understand, though, that we don't want to restrict too much the flexibility of a compiler to insert padding. But maybe a guarantee about the first field (not "word", what does that even mean?) of a struct being aligned, even the first fields (plural), if they all have the same alignment as the struct, would be clear enough *and* compatible with what compilers are already doing.

On Sat, Feb 4, 2017 at 3:31 PM, Lars Seipel <lars....@gmail.com> wrote:

On Sat, Feb 04, 2017 at 01:30:49AM -0800, T L wrote:
> Get it. This is quite bug prone to write programs on 64bit OSes and the
> programs are expected to run on 32bit OSes too.

Well, there's a reason the sync/atomic package docs have this statement
right at the top:

> These functions require great care to be used correctly. Except for special,
> low-level applications, synchronization is better done with channels or the
> facilities of the sync package.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Ian Lance Taylor]

On Fri, Jan 25, 2019 at 2:59 PM <john...@gmail.com> wrote:
>
> I still can't find anything "authoritative and unambiguous" on this. Therefore apps are going to be released with bugs in them. Can someone please say specifically when it is _not_ safe to use 64-bit atomic operations?
>
> When it is unsafe on a 64-bit OS?
> When it is unsafe on a 32-bit OS?

Speaking pedantically, it's not the OS that matters, it's the
processor (in Go terms, the GOARCH value).

The exact rules can be seen at the bottom of
https://golang.org/pkg/sync/atomic/ , in the "Bugs" section.

Ian

[Marvin Renich]

* Ian Lance Taylor <ia...@golang.org> [190125 18:11]:

In that section it says "The first word in a variable or in an allocated
struct, array, or slice can be relied upon to be 64-bit aligned." Does
that mean that an embedded struct in an allocated struct may not be
aligned? As in

type A struct {
a uint64
}

type B struct {
b int8
d A
}

type C struct {
c int8
e *A
}

var x = B{3, A{4}}
var y = C{6, &A{7}}

x.d.a might not be aligned, but y.e.a will be?

Thanks...Marvin

[Ian Lance Taylor]

Yes.

Ian

[T L]

Is the bug zone outdated now. How about the support on other 32-bit archs? Such as mips?

On Wednesday, February 1, 2017 at 12:03:59 PM UTC-4, T L wrote:

the sync/atomic docs, https://golang.org/pkg/sync/atomic/, says in the end of the docs

On x86-32, the 64-bit functions use instructions unavailable before the Pentium MMX. 

On non-Linux ARM, the 64-bit functions use instructions unavailable before the ARMv6k core.

On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. 

The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned.

[Ian Lance Taylor]

On Thu, Feb 7, 2019 at 3:49 PM T L <tapi...@gmail.com> wrote:
>
> Is the bug zone outdated now. How about the support on other 32-bit archs? Such as mips?

The bug description is not out of date.

Yes, 32-bit MIPS also requires 8 byte alignment for the 64-bit
operations. I sent https://golang.org/cl/161697 to update the docs.

Ian

[T L]

Thanks for the clarification.