threaded R/O access to built in map

[Warren Bare]

Hi Folks,

I understand built in maps are not thread safe.  

Does anyone know if it is safe to have a bunch of threads simultaneously accessing a single map if they are just doing read only lookups?  Specifically, if the map is created before the separate go routines try to use the map to resolve keys into values.

Many thanks!!

Warren

[Caleb Spare]

Yes, it's safe to do concurrent reads as long as none of them are concurrent with any writes.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[andrey mirtchovski]

the RWMutex in sync is designed specifically for this purpose: many
readers, infrequent updaters. Locking and unlocking for readers should
be cheap.

[Jan Mercl]

On Wed Feb 25 2015 at 20:44:37 Warren Bare <warre...@gmail.com> wrote:

> Does anyone know if it is safe to have a bunch of threads

> simultaneously accessing a single map if they are just doing read only 

> lookups? Specifically, if the map is created before the separate go 

> routines try to use the map to resolve keys into values.

Not every data structure is automatically safe for R/O-like concurrent access through its methods, but AFAICT, Go's map implementation is safe in this case.

-j

[Warren Bare]

Outstanding.  Thanks for your help guys.

Warren

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/86N_xtgcZqo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

[David Anderson]

Unless I'm missing something, the Go memory model does not guarantee that the map implementation will always be safe for concurrent read-only access. So, per the "don't be clever" advice in the memory model docs, I'd wrap the map access in an RWMutex that's only ever used in read mode. Plus, then if you do need to update it, you can already do so safely.

- Dave

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Caleb Spare]

​Concurrent reads are safe. It would be bizarre if readonly access required synchronization. The memory model document is about what happens in the presence of writes (see, for example, the introductory paragraph).

I agree with adonovan from another thread here: the "slap a lock on it" advice is given too often. In this case it's a huge overhead to pay for no reason.

-Caleb

[Ian Lance Taylor]

On Wed, Feb 25, 2015 at 2:02 PM, David Anderson <da...@natulte.net> wrote:
>

> Unless I'm missing something, the Go memory model does not guarantee that
> the map implementation will always be safe for concurrent read-only access.
> So, per the "don't be clever" advice in the memory model docs, I'd wrap the
> map access in an RWMutex that's only ever used in read mode. Plus, then if
> you do need to update it, you can already do so safely.

Concurrent reads of maps are safe. A map variable is like any other
variable. I assume you believe that concurrent reads of slices and
arrays are OK. The same is true for maps.

Ian

[Rob Pike]

But you may need to synchronize after initialization and before reading. That is, initialize, sync memory, and then you can read in parallel all you like.

I believe the transition from init() to main() should sufficient to do this syncing, but that's not said anywhere. Is it true?

-rob

[Caleb Spare]

I believe the transition from init() to main() should sufficient to do this syncing, but that's not said anywhere. Is it true?

​It's mentioned in the memory model:

​"​

The start of the function main.main happens after all init functions have finished.

​"​

[Rob Pike]

Great.

-rob

[Matt Harden]

A future map implementation might optimize reads by reordering a structure based on the pattern of reads. It's not unreasonable to think a map implementation might do this. If maps are guaranteed to be safe for concurrent reads, and will remain so, this should be documented somewhere.

[Ian Lance Taylor]

On Thu, Feb 26, 2015 at 4:50 AM, Matt Harden <matt....@gmail.com> wrote:
>
> A future map implementation might optimize reads by reordering a structure
> based on the pattern of reads. It's not unreasonable to think a map
> implementation might do this. If maps are guaranteed to be safe for
> concurrent reads, and will remain so, this should be documented somewhere.

It's documented in the memory model. A map is a value like any other
value.

Ian

[Matt Harden]

The question is not whether a map is a value. The question is whether retrieving values from a map is safe to do concurrently in multiple goroutines. It's obvious for slices that calling b := x[1] and a := x[2] in separate goroutines is safe, because a slice is "merely" a pointer into an array, a length, and a capacity. Just as concurrent reads from an array are safe, concurrent reads from a slice are safe. I can also get a pointer to a slice element, because this is just a location in memory. Maps are free to move their elements around in memory and so it's not valid Go to take the address of a map element.

This is not so obvious for maps, because a creating a performant map implementation is much more complex than a slice. A map implementation of x["abcd"] might mutate some internal structure like an LRU cache or it might slightly restructure a tree to make future gets from the same key faster. Since it is nowhere stated in the Go Memory Model or the Go Language Specification that retrieving map values from the same map concurrently is safe, a Go runtime could use a map implementation that is not safe for concurrent read access to its values. Such an implementation would still conform to the spec. If that is not the case, please point out what in the spec or the memory model document rules out such behavior? The memory model doc does not mention maps at all.

Regards,

Matt

[Rob Pike]

If all writes happen before any reads, it's safe.

-rob

[Matt Harden]

With respect, I know it's safe in the current implementation. My point is that this is not guaranteed by the language spec or the memory model. The implementation could change. An alternative implementation might not be safe.

I'm going to stop now; this conversation isn't getting anywhere.

[Ian Lance Taylor]

On Sat, Feb 28, 2015 at 8:49 PM, Matt Harden <matt....@gmail.com> wrote:
>

> The question is not whether a map is a value. The question is whether
> retrieving values from a map is safe to do concurrently in multiple
> goroutines.

A map is a value. Reading from a map is a read of a value. The
memory spec guarantees that concurrent reads of values are OK.
Therefore concurrent reads of maps are OK.

> This is not so obvious for maps, because a creating a performant map
> implementation is much more complex than a slice. A map implementation of
> x["abcd"] might mutate some internal structure like an LRU cache or it might
> slightly restructure a tree to make future gets from the same key faster.
> Since it is nowhere stated in the Go Memory Model or the Go Language
> Specification that retrieving map values from the same map concurrently is
> safe, a Go runtime could use a map implementation that is not safe for
> concurrent read access to its values. Such an implementation would still
> conform to the spec.

No, it wouldn't.

> If that is not the case, please point out what in the
> spec or the memory model document rules out such behavior? The memory model
> doc does not mention maps at all.

It doesn't need to.

Ian

[Jan Mercl]

On Mon, Mar 2, 2015 at 5:12 AM Ian Lance Taylor <ia...@golang.org> wrote:

> A map is a value. Reading from a map is a read of a value. The

> memory spec guarantees that concurrent reads of values are OK.
> Therefore concurrent reads of maps are OK.

I believe the concern is about

        m := map[T]U{}

        ...

        n := m // Reading a map value (A)

vs

        v := m[k] // Reading a value from a map (B)

Case A is obviously covered by the memory model. The later case is defined by the specs as an index expression and the concept of "reading a value" is not mentioned in that part of the specs nor it's IMO 100% clear if it should if not specified explicitly as such - in the case of maps. Technical reasons for the uncertainity feeling were stated before by others (LRU cache for example) and IIRC, one such proposal by Brad is already somewhere on the issue list.

-j

[atd...@gmail.com]

The question of a LRU cache/map retrieval is a little bit of a red herring. I expect that it would be taken care of in the implementation internals. Otherwise that would be a language change even for a single read. The end user doesn't need to be aware about that.

[Ian Lance Taylor]

I would say that an index expression is clearly reading the value
being indexed. What else could it be?

All aggregate types--maps, slices, arrays, and even structs--are in
exactly the same position with regard to the memory spec. We don't
think the memory spec needs to say anything about slices, arrays, and
structs, because it is "obvious" that concurrent reads of those types
are OK. But if you think that, then you are confusing the underlying
implementation with the actual language. And that same confusion is
driving this sense that the memory spec needs to say something special
about maps.

All aggregate types are the same with regard to the memory spec: they
are all values. Concurrent reads are always OK. Concurrent writes
are not. There is no need to say anything special about maps.

Ian

[Dmitry Vyukov]

On Mon, Mar 2, 2015 at 7:32 PM, Ian Lance Taylor <ia...@golang.org> wrote:
> On Sun, Mar 1, 2015 at 11:50 PM, Jan Mercl <0xj...@gmail.com> wrote:
>> On Mon, Mar 2, 2015 at 5:12 AM Ian Lance Taylor <ia...@golang.org> wrote:
>>
>>> A map is a value. Reading from a map is a read of a value. The
>>> memory spec guarantees that concurrent reads of values are OK.
>>> Therefore concurrent reads of maps are OK.
>>
>> I believe the concern is about
>>
>> m := map[T]U{}
>>
>> ...
>>
>> n := m // Reading a map value (A)
>>
>> vs
>>
>> v := m[k] // Reading a value from a map (B)
>>
>> Case A is obviously covered by the memory model. The later case is defined
>> by the specs as an index expression and the concept of "reading a value" is
>> not mentioned in that part of the specs nor it's IMO 100% clear if it should
>> if not specified explicitly as such - in the case of maps. Technical reasons
>> for the uncertainity feeling were stated before by others (LRU cache for
>> example) and IIRC, one such proposal by Brad is already somewhere on the
>> issue list.
>
> I would say that an index expression is clearly reading the value
> being indexed. What else could it be?
>
> All aggregate types--maps, slices, arrays, and even structs--are in
> exactly the same position with regard to the memory spec. We don't

Maps and arrays are not:

x[1] = 1
x[2] = 2

works for arrays, but not for maps.

Slices and arrays are also different:

x[1] = 1
y = x

is ok for slices, but not for arrays.

> think the memory spec needs to say anything about slices, arrays, and
> structs, because it is "obvious" that concurrent reads of those types
> are OK. But if you think that, then you are confusing the underlying
> implementation with the actual language. And that same confusion is
> driving this sense that the memory spec needs to say something special
> about maps.
>
> All aggregate types are the same with regard to the memory spec: they
> are all values. Concurrent reads are always OK. Concurrent writes
> are not. There is no need to say anything special about maps.
>
> Ian
>

[Ian Lance Taylor]

On Mon, Mar 2, 2015 at 9:04 AM, Dmitry Vyukov <dvy...@google.com> wrote:
> On Mon, Mar 2, 2015 at 7:32 PM, Ian Lance Taylor <ia...@golang.org> wrote:
>
>> All aggregate types--maps, slices, arrays, and even structs--are in
>> exactly the same position with regard to the memory spec. We don't
>
> Maps and arrays are not:
>
> x[1] = 1
> x[2] = 2
>
> works for arrays, but not for maps.
>
> Slices and arrays are also different:
>
> x[1] = 1
> y = x
>
> is ok for slices, but not for arrays.

I think I've managed to figure out what you are saying. You mean that
two different goroutines can concurrently execute

x[1] = 1
x[2] = 2

for an array, but not for a map. That is a good point. This is a
consequence of the spec's rule that each individual element of an
array, slice, or struct is a variable
(http://golang.org/ref/spec#Variables).

My larger point is still correct, though.

Ian

[Dmitry Vyukov]

The larger point is correct, but the details are tricky enough.

For example, when you write to a slice you write the element value
_and_ read the slice value. So it is OK to do:

s[1] = 1
tmp = s

Maps are even more interesting, there is a value for map handle and
there is a _single_ abstract value that represents all keys and
values. So it is NOT ok to do:

m[1] = 1
m[2] = 2

but OK to do:

m[1] = 1
tmp = m

I can imagine that unwillingness to simply state this explicitly in
the spec can cause confusion for Go users.

[adon...@google.com]

On Monday, 2 March 2015 11:33:45 UTC-5, Ian Lance Taylor wrote:

All aggregate types--maps, slices, arrays, and even structs--are in
exactly the same position with regard to the memory spec.  We don't
think the memory spec needs to say anything about slices, arrays, and
structs, because it is "obvious" that concurrent reads of those types
are OK.

I think it's not entirely obvious.  A reasonable map implementation might cache the most-recently accessed entry to make patterns like this faster:

seen := make(map[string]bool)

if !seen[x] {

    seen[x] = true

    ....

}

This would make operations that are logically read-only (lookup, iteration) into write operations as far as memory is concerned, and thus maps would not be safe even for concurrent read access; an exclusive Mutex lock, not a multi-reader RWMutex lock, would be needed to serialize read operations.

Of course, Go does not do that, and the spec says so if you read carefully, but I wouldn't say it's obvious that it doesn't.

[Ian Lance Taylor]

On Tue, Mar 3, 2015 at 8:23 AM, <adon...@google.com> wrote:
> On Monday, 2 March 2015 11:33:45 UTC-5, Ian Lance Taylor wrote:
>>
>> All aggregate types--maps, slices, arrays, and even structs--are in
>> exactly the same position with regard to the memory spec. We don't
>> think the memory spec needs to say anything about slices, arrays, and
>> structs, because it is "obvious" that concurrent reads of those types
>> are OK.
>
>
> I think it's not entirely obvious.

I was saying that it is "obvious" with regard to slices, arrays, and
structs.

Ian

[adon...@google.com]

On Tuesday, 3 March 2015 12:47:59 UTC-5, Ian Lance Taylor wrote:

I was saying that it is "obvious" with regard to slices, arrays, and structs.

So you were.  And I suppose my feeling that maps were different was exactly the confusion of language and implementation you were talking about.

Carry on.

 

[Warren Bare]

Since this conversation has died down, I just wanted to say thank you to everyone that contributed such great information.  This was a super informative thread and I love the fact that there are people on this list knowledgable enough to really make well formed arguments on each side.  I learned a lot more than I expected when I first posted the question.

Thanks again!

Warren

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/86N_xtgcZqo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

[Matt Harden]

Alan,

I feel I've read the spec carefully and I don't think it does state unambiguously that concurrent reads from a map are safe. Can you point out the wording that made you believe that? Hopefully you can change my mind.

Thanks,

Matt

[Ian Lance Taylor]

On Thu, Mar 12, 2015 at 12:30 PM, Matt Harden <matt....@gmail.com> wrote:
>
> I feel I've read the spec carefully and I don't think it does state
> unambiguously that concurrent reads from a map are safe. Can you point out
> the wording that made you believe that? Hopefully you can change my mind.

Let me turn that around on you. Where in the spec is there any
license for concurrent reads of a map to be unsafe?

The memory model implies that concurrent reads of a "variable" are
safe. It implies this by not making any restriction on concurrent
reads.

The spec defines a "variable" (http://golang.org/ref/spec#Variables)
and it is clear that any type that can be written in Go is at least
one variable. Some variables are structured and contain
sub-variables; this is true of array, slice, and struct types. So a a
map variable is an unstructured variable within the meaning of the
spec, and therefore of the memory model.

Since the memory model permits concurrent reads of variables, it
permits concurrent reads of maps.

Ian

[Alan Donovan]

Hi Ian,

I think I agree with Matt that the current spec and memory model are incomplete in this regard.  The question is: what constitutes a read of a variable?

Variables of aggregate type---structs and arrays---have sub-variables that can be independently pointed to, read, and updated.  The spec allows not only concurrent reads of distinct sub-variables, but concurrent writes to them too.  You can desugar all a[i] and s.f operations to address arithmetic and loads and stores of memory locations (scalar variables) and reason about concurrency that way.  This is an unavoidable consequence of the spec.  Implementations must represent these data structures in the obvious way, not as (say) JavaScript-style sparse mappings.  "Read operations" must be memory loads.

In contrast, for maps, although there are clearly many sub-variables representing each element and other bookkeeping, none of them can be pointed to, read, or written directly.  We have to use the m[k], m[k]=v, and range operations which clearly do a lot more than a simple load, store, or pointer increment.  The spec gives implementations a lot of freedom to choose the representation of a map.  The proposition that m[k] and range operations do not store values to memory may be true, it may be obvious, it may be consistent with the other data types, but it is not a logical consequence of the current spec.

cheers

alan

[Jan Mercl]

On Thu, Mar 12, 2015 at 8:42 PM Ian Lance Taylor <ia...@golang.org> wrote:

> Let me turn that around on you. Where in the spec is there any

> license for concurrent reads of a map to be unsafe?

> The memory model implies that concurrent reads of a "variable" are
> safe. It implies this by not making any restriction on concurrent

> reads.

> The spec defines a "variable" (http://golang.org/ref/spec#Variables)
> and it is clear that any type that can be written in Go is at least

> one variable. Some variables are structured and contain

> sub-variables; this is true of array, slice, and struct types. So a a

> map variable is an unstructured variable within the meaning of the

> spec, and therefore of the memory model.

> Since the memory model permits concurrent reads of variables, it
> permits concurrent reads of maps.

Given

        m := map[T]U{}

m is a variable. Performing for example

        n := m // (1)

reads the variable m and no one suspects this is not safe to do concurrently. However the indexing operation, for example

        v := m[k] // (2)

is a two step process. The first step is read access if the variable m and the second one is the indexing operation per se, using the just read value of m to find the value associated with k within what m represents. I, for one, do not see any obvious reason why the guarantees of the first step should be implicitly applied to the second step as those two steps are reasonable to expect to be to be different. Reading the m variable can be as simple as the single instruction load into a CPU register from a single word sized memory location. Determining which value within m is associated with k is, in the general case, a function/method call with complexity not really comparable to a single instruction (of the first step). This function call can potentially mutate state of m (or what m represents indirectly being a pointer), eg. cache last k->v mapping etc.

I admit that the uncertain feeling comes mostly from not abstracting away from the implementation details, but even then, I still would point out that the view of two steps of (2) vs the single step of (1) is somewhat legitimate even when not thinking about any specific map implementation.

Not pushing for any specs change. Just wanted to share why I'm not completely comfortable with saying (1) equals (2) wrt the memory model.

If one would want to be pedantic: The memory model[0] does not ever mention the indexing operation or array, slice or map things in particular, so it is in a certain sense possible to claim the indexing operation might not be covered by the memory model.

[0]: http://golang.org/ref/mem

[1]: http://golang.org/ref/spec#Index_expressions

-j

[Ian Lance Taylor]

On Thu, Mar 12, 2015 at 1:28 PM, Alan Donovan <adon...@google.com> wrote:
>
> Variables of aggregate type---structs and arrays---have sub-variables that
> can be independently pointed to, read, and updated. The spec allows not
> only concurrent reads of distinct sub-variables, but concurrent writes to
> them too. You can desugar all a[i] and s.f operations to address arithmetic
> and loads and stores of memory locations (scalar variables) and reason about
> concurrency that way. This is an unavoidable consequence of the spec.
> Implementations must represent these data structures in the obvious way, not
> as (say) JavaScript-style sparse mappings. "Read operations" must be memory
> loads.
>
> In contrast, for maps, although there are clearly many sub-variables
> representing each element and other bookkeeping, none of them can be pointed
> to, read, or written directly. We have to use the m[k], m[k]=v, and range
> operations which clearly do a lot more than a simple load, store, or pointer
> increment. The spec gives implementations a lot of freedom to choose the
> representation of a map. The proposition that m[k] and range operations do
> not store values to memory may be true, it may be obvious, it may be
> consistent with the other data types, but it is not a logical consequence of
> the current spec.

In my opinion, there aren't any sub-variables of a map. The spec says
clearly that there are sub-variables of slices, arrays, and structs.
It makes no such statement for maps. Here I'm obviously using the
word "variable" in the somewhat technical sense used in the spec,
where it basically something whose address can be taken.

A map is a single variable.

Ian

[atd...@gmail.com]

I think it is an implementation concern : even if a read from a map consisted effectively in multiple operations, some involving writes, you could expect some internal synchronization to guarantee that there were no torn reads. Otherwise the map implementation would be broken given the spec http://golang.org/ref/spec#Expressions especially since the spec does not make a distinction here about a read happening in a single vs. multithreaded context.

But then again, I also understand the potential confusion, the F.A.Q. saying that maps are not safe for concurrent access without making any distinction (http://golang.org/doc/faq#atomic_maps). Someone reading fast may draw the conclusion that it is not safe even for concurrent read only accesses.

No strong opinion on this. The spec is obviously correct but if it helps, something could perhaps be mentioned somewhere else.

[Alan Donovan]

On 12 March 2015 at 18:42, Ian Lance Taylor <ia...@golang.org> wrote:

In my opinion, there aren't any sub-variables of a map.  The spec says
clearly that there are sub-variables of slices, arrays, and structs.
It makes no such statement for maps.  Here I'm obviously using the
word "variable" in the somewhat technical sense used in the spec,
where it basically something whose address can be taken.

A map is a single variable.

Hi Ian,

we agree that read-only access to maps should be safe from multiple goroutines, but I can't see how it's valid to conclude that from the spec.  You mention that the spec uses the word "variable" to mean something whose address can be taken; I think that's a good definition.  But nowhere does the spec even remotely suggest that a map is a variable!

Worse, I don't think it's possible to derive even the basic idea that a value of 'map' type has reference semantics, or that copying a value of 'map' type creates an alias to a single piece of state so that operations on either copy are observed by the other; this is a frequent point of confusing for Go novices.  It's not clear that Go's maps aren't value types in the style of C++ collections.  The nil value is a hint that map is a reference type, but not conclusive.

I think the the burden is on the spec to state these assumptions:

- that a 'map' value is a reference to an opaque data structure

- that copying a 'map' value creates a new alias to the same instance of that data structure

- that this data structure is a variable for the purposes of the memory model

- that m[k] and range operations on a map are reads of that variable, for the same purposes

- and that m[k]=v and delete operations are writes.

- that a 'map' value may be safely represented by an unsafe.Pointer.

cheers

alan

[Ian Lance Taylor]

On Fri, Mar 13, 2015 at 4:07 PM, Alan Donovan <adon...@google.com> wrote:
>
> we agree that read-only access to maps should be safe from multiple
> goroutines, but I can't see how it's valid to conclude that from the spec.
> You mention that the spec uses the word "variable" to mean something whose
> address can be taken; I think that's a good definition. But nowhere does
> the spec even remotely suggest that a map is a variable!

You could equally well say that the spec doesn't suggest that an int
is a variable. I think that http://golang.org/ref/spec#Variables has
to be read as saying that "var m map[k]v" give you a variable. When I
say "a map is a variable" I do of course loosely mean that a variable
of map type counts as a variable.

> Worse, I don't think it's possible to derive even the basic idea that a
> value of 'map' type has reference semantics, or that copying a value of
> 'map' type creates an alias to a single piece of state so that operations on
> either copy are observed by the other; this is a frequent point of confusing
> for Go novices. It's not clear that Go's maps aren't value types in the
> style of C++ collections. The nil value is a hint that map is a reference
> type, but not conclusive.

Yes, this is http://golang.org/issue/5803 .

> I think the the burden is on the spec to state these assumptions:
> - that a 'map' value is a reference to an opaque data structure
> - that copying a 'map' value creates a new alias to the same instance of
> that data structure
> - that this data structure is a variable for the purposes of the memory
> model

I agree.

> - that m[k] and range operations on a map are reads of that variable, for
> the same purposes
> - and that m[k]=v and delete operations are writes.

Not necessary in my opinion. I just think people are letting their
understanding of the implementation color their assumptions about the
language. But I admit that I'm getting tired of this argument.

> - that a 'map' value may be safely represented by an unsafe.Pointer.

I don't think this should be stated by the spec at all. I don't think
ordinary code should assume this to be true.

Ian

[Caleb Spare]

Yes, this is http://golang.org/issue/5803 .

You mean https://golang.org/issue/5083.

[Ian Lance Taylor]

On Fri, Mar 13, 2015 at 4:49 PM, Caleb Spare <ces...@gmail.com> wrote:
>> Yes, this is http://golang.org/issue/5803 .
>
>
> You mean https://golang.org/issue/5083.

Indeed I do. Thanks.

Ian

[adon...@google.com]

On Friday, 13 March 2015 19:44:34 UTC-4, Ian Lance Taylor wrote:

On Fri, Mar 13, 2015 at 4:07 PM, Alan Donovan <adon...@google.com> wrote:
>
> we agree that read-only access to maps should be safe from multiple
> goroutines, but I can't see how it's valid to conclude that from the spec.
> You mention that the spec uses the word "variable" to mean something whose
> address can be taken; I think that's a good definition.  But nowhere does
> the spec even remotely suggest that a map is a variable!

You could equally well say that the spec doesn't suggest that an int
is a variable.  I think that http://golang.org/ref/spec#Variables has
to be read as saying that "var m map[k]v" give you a variable.  When I
say "a map is a variable" I do of course loosely mean that a variable
of map type counts as a variable.

Of course every "var" decl gives you a variable; the question is about "non addressable" map values such as make(map[k]v). 

At least some of the confusion here arises from the distinction between map-the-reference and map-the-datastructure, a distinction the spec does not yet make but must, as you point out in the bug link.  A map and an int are categorically different; a map is more like a pointer.  Whereas a pointer is a reference to an ordinary variable, a map is a reference to the "map data structure", which is like a variable for our purposes.

 

> - that m[k] and range operations on a map are reads of that variable, for
> the same purposes
> - and that m[k]=v and delete operations are writes.

Not necessary in my opinion.  I just think people are letting their
understanding of the implementation color their assumptions about the
language.  But I admit that I'm getting tired of this argument.

My remark didn't presume any implementation.  More generally, I'm a little baffled why this seems like a matter of opinion.  Isn't the point of the spec to make things clear enough that reasonable people don't disagree?

Anyway, sorry to tire you out.  I'll file a spec change bug since that's the proper place for it.

 

> - that a 'map' value may be safely represented by an unsafe.Pointer.

I don't think this should be stated by the spec at all.  I don't think
ordinary code should assume this to be true.

Well, the reflection API depends upon it, and I recently asked Russ about it and he said it was a safe assumption.  So either the spec or Russ needs to be changed. :)

[atd...@gmail.com]

Maybe, if we don't use the term "read" but "expression" as in the spec, it might make things clearer.

In that case, m[k] is just an expression, an index expression.

An expression specifies the computation of a value by applying operators and functions to operands. (spec)

 

In my understanding, the spec does not preclude any change in implementation provided its requirements are still fulfilled. You could conceptually have an implementation of the map operations subject to internal race conditions if it was conserving correctness for the whole program.

For the question of being safe for concurrent read-only access, it would stem from the index expression section allotted to maps :

if the map contains an entry with key x, a[x] is the map value with key x and the type of a[x] is the value type of M

Indeed, in a concurrent program, if there is a value being assigned at the same time ( a write) the expression is being evaluated ( a read), there will be a torn read (and that's expected behaviour even though racy, because it is effectively the value in memory at some point). But otherwise, this requirement makes safe concurrent read-only map access a necessity for correct implementations, unless I'm mistaken.

As an additional perhaps interesting datapoint, I've been playing around replacing the go hashmap implementation with a lock free one that is safe for concurrent reads/writes. (the writes performance was horrendous though, allocations due to heavy RCU usage were too expensive). 

Bottom line is that value retrievals from the map (reads) involved some writes (because the implementation was moving things around to optimize space). Yet, the end user is not and does not need to be aware of this.

Anyway, I will leave the rest of the discussion to you guys, as I don't think I belong to this side of the discussion anymore. 

But if you allow an external point of view, I do think that the specification should be separate from the implementation.

Defining "reference type" in the spec is more than welcome. Coercing it into being represented by an unsafe.Pointer for maps ?  It might freeze the choice of implementation unnecessarily (even if it is low risk in this case).

I welcome any correction for any stupidities I might have written too :o)

A.