Auto Make a named return map

309 views
Skip to first unread message

Kevin Chadwick

unread,
Mar 11, 2021, 7:25:02 PM3/11/21
to golang-nuts
I find named returns produce more readable code but I avoid them when returning a map.

Why doesn't go auto init or make an empty map for a named return to avoid the potential chance of a panic due to operating on a nil return?

Axel Wagner

unread,
Mar 12, 2021, 2:16:59 AM3/12/21
to Kevin Chadwick, golang-nuts
On Fri, Mar 12, 2021 at 1:24 AM Kevin Chadwick <m8il...@gmail.com> wrote:
I find named returns produce more readable code but I avoid them when returning a map.

Personally, I would recommend avoiding naked returns in all cases. In the best case, a naked return is less explicit and requires readers to think about what is returned. In the worst case, a naked return can introduce subtle shadowing bugs (this has happened in production code, multiple times).

I don't think the convenience justifies the cost.

Why doesn't go auto init or make an empty map for a named return to avoid the potential chance of a panic due to operating on a nil return?

It auto-initializes every variable to its zero value. It would be very strange to make an exception for maps. And named returns aren't really different from regular variables in this regard.

I also don't think there are many cases where you'd *want* to implicitly return an empty, non-nil map. A nil map says "there is no map", whereas an empty map says "there is a map, but it has no elements". It seems very appropriate that if you want to say the latter, you have to explicitly do so.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANNfpqe5VmDfd53YMHDecOEt-XNH3efCVGAa1Z3q6qJJbDQYsw%40mail.gmail.com.

Kevin Chadwick

unread,
Mar 12, 2021, 6:51:38 AM3/12/21
to golang-nuts
On 3/12/21 7:16 AM, Axel Wagner wrote:
> I find named returns produce more readable code but I avoid them when
> returning a map.
>
>
> Personally, I would recommend avoiding naked returns in all cases. In the best
> case, a naked return is less explicit and requires readers to think about what
> is returned. In the worst case, a naked return can introduce subtle shadowing
> bugs (this has happened in production code, multiple times).
>
> I don't think the convenience justifies the cost.

Shadowing is an issue that needs to be dealt with, in any case. Think about =
less readable.

>
> Why doesn't go auto init or make an empty map for a named return to avoid
> the potential chance of a panic due to operating on a nil return?
>
>
> It auto-initializes every variable to its zero value. It would be very strange
> to make an exception for maps. And named returns aren't really different from
> regular variables in this regard.
>
> I also don't think there are many cases where you'd *want* to implicitly return
> an empty, non-nil map. A nil map says "there is no map", whereas an empty map
> says "there is a map, but it has no elements". It seems very appropriate that if
> you want to say the latter, you have to explicitly do so.

I would argue that a string could be nil but isn't and that being 0 len is a
good thing. Ergo, it is strange for a map to be doing premature optimisation,
when you could test it for 0 size. More likely, it is an artifact of make coming
to the language later on and perhaps is tied by the go promise, unfortunately?
It might have also been a not tiny amount of work for a relatively small benefit.

We are encouraged to use make to initialise maps. I also replace err.Error()
with a function that returns "". Incidentally, I would be happy with err != "".
err.Error() has also lead to bugs in stdlib. So maybe I just have a different
point of view? Maybe null safety will come, eventually as it was quite high on
the survey and likely increasing as more languages provide this feature and
users acknowledge it.

Are there any better tools than golang-shadow and golang-nilness, for these
problems?

Kevin Chadwick

unread,
Mar 12, 2021, 7:32:28 AM3/12/21
to golang-nuts
On 3/12/21 11:51 AM, Kevin Chadwick wrote:
> Incidentally, I would be happy with err != "".

Of course I do appreciate the wrapping/error testing. Though I use little/no
more than could be accomplished with typed strings.Contains, personally.

Axel Wagner

unread,
Mar 12, 2021, 7:34:40 AM3/12/21
to Kevin Chadwick, golang-nuts
On Fri, Mar 12, 2021 at 12:51 PM Kevin Chadwick <m8il...@gmail.com> wrote:
I would argue that a string could be nil but isn't and that being 0 len is a
good thing. Ergo, it is strange for a map to be doing premature optimisation,
when you could test it for 0 size. More likely, it is an artifact of make coming
to the language later on and perhaps is tied by the go promise, unfortunately?
It might have also been a not tiny amount of work for a relatively small benefit.

This sounds like if you are suggesting to make the zero value of a map an empty map, which is different from initializing a map variable to an empty map if it a named return. That suggestion has come up a lot, both on this list and on the github issue tracker. It's, unfortunately, not as easy as it sounds. Here is some prior discussion:

Are there any better tools than golang-shadow and golang-nilness, for these
problems?

I don't know. I don't even know either of those.  


--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

Kevin Chadwick

unread,
Mar 12, 2021, 7:43:29 AM3/12/21
to golang-nuts
On 3/12/21 12:33 PM, Axel Wagner wrote:
> This sounds like if you are suggesting to make the zero value of a map an empty
> map, which is different from initializing a map variable to an empty map if it a
> named return. That suggestion has come up a lot, both on this list and on the
> github issue tracker. It's, unfortunately, not as easy as it sounds. Here is
> some prior discussion:
> https://groups.google.com/g/golang-nuts/c/SjuhSYDITm4/m/Z013vH5qDQAJ
> <https://groups.google.com/g/golang-nuts/c/SjuhSYDITm4/m/Z013vH5qDQAJ>
> https://groups.google.com/g/golang-nuts/c/5_8E9OblIho/m/b9O038mzBQAJ
> <https://groups.google.com/g/golang-nuts/c/5_8E9OblIho/m/b9O038mzBQAJ>
>

Thank You. Sorry, I should have searched.

> Are there any better tools than golang-shadow and golang-nilness, for these
> problems?
>
>
> I don't know. I don't even know either of those.  

They are in x/tools. I only found them today.

Kevin Chadwick

unread,
Mar 12, 2021, 6:01:56 PM3/12/21
to golan...@googlegroups.com

>This sounds like if you are suggesting to make the zero value of a map
>an
>empty map, which is different from initializing a map variable to an
>empty
>map if it a named return. That suggestion has come up a lot, both on
>this
>list and on the github issue tracker. It's, unfortunately, not as easy
>as
>it sounds. Here is some prior discussion:
>https://groups.google.com/g/golang-nuts/c/SjuhSYDITm4/m/Z013vH5qDQAJ
>https://groups.google.com/g/golang-nuts/c/5_8E9OblIho/m/b9O038mzBQAJ
>

>> And another reason is that it's very convenient for the zero value >> for all types to be literally a sequence of zero bytes. We could
>> never figure out how to preserve that property while not requiring >> the make call for map values. In the very early days what we call >> maps now were written as pointers, so you wrote *map[int]int.
>> We moved away from that when we realized that no one ever
>> wrote `map` without writing `*map`. That simplified many things >> but it left this issue behind as a complication.

>> The most plausible fix that I know for this is to invent a map
>> function that is similar to the append function, but I haven't yet
>> seen a good design for that.

I do not need to understand the difficulties that exist, though I struggle to understand the zero bytes property as surely even an empty string has a 4 or 8 byte pointer. Perhaps it is the memory pointed to. Maybe useful for compaction etc..

My original thinking was that either the function call or initialisation could run make under the covers. From Ian in those threads copied above "While not requiring the make call".

Why not require make by calling it, behind the scenes?

Axel Wagner

unread,
Mar 12, 2021, 7:03:57 PM3/12/21
to Kevin Chadwick, golang-nuts
On Sat, Mar 13, 2021 at 12:01 AM Kevin Chadwick <m8il...@gmail.com> wrote:
I do not need to understand the difficulties that exist, though I struggle to understand the zero bytes property as surely even an empty string has a 4 or 8 byte pointer. 

A string is effectively a
struct{
    data *byte
    len int
}
If you zero that, you get a nil pointer and a 0 length, which is a valid representation of an empty string.

The same is true for every Go type. A nil pointer is all zero bytes (so pointing at address 0, which is not accessible), maps/funcs/chans are de-facto pointers (we often say "they are pointer-shaped"), so their zero value is also just a pointer to address 0. A nil slice is a
struct {
    data *T
    len int
    cap int
}
which, if zeroed, points at 0 with length/capacity 0. And so on.
 
The zero value of any Go type is just whatever it is represented as in memory, with all bytes set to 0.

My original thinking was that either the function call or initialisation could run make under the covers. 
From Ian in those threads copied above "While not requiring the make call".

Why not require make by calling it, behind the scenes?

But that is exactly the loss of "the zero value is all 0 bytes" we are talking about. It would mean if you write `var m map[int]int` the compiler needs to create instructions to initialize `m` to something that is not zero bytes (a.k.a. "call make behind the scenes"). And if you do `m := make([]map[int]int, 1e6)` it needs to run those instructions a million times - instead of just getting a block of 0-bytes from the runtime.

Of course it's possible to abandon this property. Lots of languages (dare I say most languages) don't have it. We'd just prefer not to.
 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

Axel Wagner

unread,
Mar 13, 2021, 3:04:03 AM3/13/21
to Matthew Holiday, golang-nuts
On Sat, Mar 13, 2021 at 1:24 AM Matthew Holiday <matthew...@nytimes.com> wrote:
I don't think we should change creation; what about having the first insert make the map if it's nil?

It seems that would be fairly transparent.

I assume you accidentally hit "Reply" instead of "Reply All".

This isn't as simple either, unfortunately:

func F(m map[int]int) {
    m[42] = 23
}

func main() {
    var m map[int]int
    F(m)
    fmt.Println(m[42])
}

F might create the map, but the change has to be propagated back somehow - and all it got is a nil-pointer.

No offense, but again, if it was that simple, it would have already happened :) No one is arguing that having to explicitly initialize a map (the same, BTW, goes for channels) is not awkward. But any solution comes with its own set of problems, be it having to write `*map` everywhere, introducing call-by-reference semantics, but only for some types, not for others or significant performance penalties. It's been over a decade and we didn't find a panacea yet - it's becoming increasingly unlikely that we find one. And if we where to go through the churn of orchestrating a deep language change like this, breaking almost every Go program out there, it really has to be worth it and not just trade one wart for another.
 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfGUjnqSnBmKZ%3DH6UHWmK%3DSFTigNLnYiBi%3DcT2Km2UzskQ%40mail.gmail.com.


--
Matt Holiday
Senior Gopher, Marketing Technologies

620 Eighth Avenue

New York, NY 10018

matthew...@nytimes.com

Kevin Chadwick

unread,
Mar 13, 2021, 8:14:46 AM3/13/21
to golang-nuts

>
> I don't think we should change creation; what about having the first insert
> make the map if it's nil?
>
> It seems that would be fairly transparent.
>

This wouldn't solve the problem that I saw. Whilst, an easy personal fix is to
make sure coders always make a map whenever used. I figured a user might quite
easily think that := on a returned map would always be safe, like with a string.
Making an empty map just to return it and later making a map of a set size,
seems odd. I also do not see why you would ever initialise a nil map.

After looking into the named return discussion, including Dave Cheney's blog. I
also considered that :=, seemed to be the real problem in the end and IMO, the
feature with least to offer.

>
> My original thinking was that either the function call or
> initialisation could run make under the covers. 
>
> From Ian in those threads copied above "While not requiring the make
> call".
>
> Why not require make by calling it, behind the scenes?
>
>
> But that is exactly the loss of "the zero value is all 0 bytes" we are
> talking about.
AFAICT, it looks like I misread Ian's response as if make could always be used
then it would solve the problem rather than being a cause of a compiler
complication.

Axel Wagner

unread,
Mar 13, 2021, 8:41:29 AM3/13/21
to Kevin Chadwick, golang-nuts
On Sat, Mar 13, 2021 at 2:14 PM Kevin Chadwick <m8il...@gmail.com> wrote:

>
>     I don't think we should change creation; what about having the first insert
>     make the map if it's nil?
>
>     It seems that would be fairly transparent.
>

This wouldn't solve the problem that I saw.

It really is just another way to say "the zero value of a map should be an empty map". That is "it's initialized on the first write" was a suggestion on how to implement those semantics, because to the programmer there would no longer be a distinction between an empty map and a "nil" map. Currently, the only difference is that you can't write to a nil map (as well as comparing it to nil). That difference would disappear.

As an aside: It's the same for slices, currently. An empty slice and a nil slice are identical in every way, except for comparison to nil. 

AFAICT, it looks like I misread Ian's response as if make could always be used
then it would solve the problem rather than being a cause of a compiler
complication.

Compiler complication is not the concern. It would be easy to build any of the suggested semantics into the language. It is just that none of the suggestions so far seem clearly better - that is, they all come with their own downsides.

The concern about the zero value consisting of 0 bytes is more about performance. As long as that's the case, initialization is very close to free. As soon as the zero value becomes some non-zero bytes, they have to be written and that incurs a cost. 

When we say "the zero value of a map should only contain zero bytes", what we mean is that if I write
var m map[int]int
that should a) reserve some memory for m (currently that's as many bytes as a pointer needs) and b) those bytes should be zero after that.
If "make was called implicitly" or anything like that, those bytes wouldn't be zero anymore - they would contain a pointer to an allocated map header.

It's as simple as that. You might feel that preserving that property is not important or less important than having a useful zero value for maps. That's fine. But that's the property Ian is talking about.

 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

Kevin Chadwick

unread,
Mar 13, 2021, 9:22:30 AM3/13/21
to golang-nuts
On 3/13/21 1:40 PM, Axel Wagner wrote:
>
> Compiler complication is not the concern. It would be easy to build any of the
> suggested semantics into the language. It is just that none of the suggestions
> so far seem clearly better - that is, they all come with their own downsides.
>

OK, I really meant, GC complexity, etc. Perhaps it does not affect that either.

> The concern about the zero value consisting of 0 bytes is more about
> performance. As long as that's the case, initialization is very close to free.
> As soon as the zero value becomes some non-zero bytes, they have to be written
> and that incurs a cost. 
>
> When we say "the zero value of a map should only contain zero bytes", what we
> mean is that if I write
> var m map[int]int
> that should a) reserve some memory for m (currently that's as many bytes as a
> pointer needs) and b) those bytes should be zero after that.
> If "make was called implicitly" or anything like that, those bytes wouldn't be
> zero anymore - they would contain a pointer to an allocated map header.
>
> It's as simple as that. You might feel that preserving that property is not
> important or less important than having a useful zero value for maps. That's
> fine. But that's the property Ian is talking about.


In that case, as far as I understand it, which may be a limited understanding. I
disagree with this being a useful property. Replacing a map of 0 length with one
that is larger is not a problem. Checking the length of a map also makes more
sense to me than nil.

Though I personally do not agree with Nulls in JSON either. An intentional empty
is always more useful. Similarly, nulls when permitted always being
unintentional is more useful in my eyes.

Axel Wagner

unread,
Mar 13, 2021, 9:48:42 AM3/13/21
to Kevin Chadwick, golang-nuts
On Sat, Mar 13, 2021 at 3:22 PM Kevin Chadwick <m8il...@gmail.com> wrote:
In that case, as far as I understand it, which may be a limited understanding. I
disagree with this being a useful property.

That is your prerogative. I was simply trying to explain why maps work the way they do (and ISTM that if you want to change the way they work, you should really try to understand why they work this way first). I wasn't trying to convince you it's a good thing.
 
Replacing a map of 0 length with one
that is larger is not a problem. Checking the length of a map also makes more
sense to me than nil.

Neither of these seems relevant. I agree that if the zero value of a map was a non-nil empty map, that you could check the length to see if it is empty by checking it's length. And that comparison to nil would likely be obsolete (though, again, slices work that way and still have a separate nil value and that doesn't seem to bother people).
I also don't think anyone disagrees that it would be nice to have the zero value of a map be simply an empty map (that you can write to).

Neither of these have anything to do with whether or not the zero value is represented by 0-bytes, though.
There are ways to get these properties that would leave the zero value as 0-bytes - for example, a linked list of key-value pairs that uses an append-like builtin.
There are ways that get these properties while destroying the property that the zero value is 0-bytes - for example your suggestion of implicitly initializing maps using `make`.

What you describe are the language-level semantics of maps - can a programmer compare them to nil, or can they write to it. But this property is not about semantics, it's an implementation restriction. It's just that we can't find a way to implement better semantics, without losing this property or incurring some other significant downsides.

Though I personally do not agree with Nulls in JSON either. An intentional empty
is always more useful. Similarly, nulls when permitted always being
unintentional is more useful in my eyes.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

Jan Mercl

unread,
Mar 13, 2021, 10:05:51 AM3/13/21
to Kevin Chadwick, golang-nuts
On Fri, Mar 12, 2021 at 1:24 AM Kevin Chadwick <m8il...@gmail.com> wrote:

> Why doesn't go auto init or make an empty map for a named return to avoid the potential chance of a panic due to operating on a nil return?

A non-zero value map value must be allocated. All well-known Go
compilers allocate maps in heap. Allocating automagically a map that
may end up never used is a waste of resources.

Kevin Chadwick

unread,
Mar 13, 2021, 11:52:24 AM3/13/21
to golang-nuts
Very little resources, unless the map is actually used and not for long. If you really need to control gos memory use, you need to preallocate arrays in a long standing manner anyway, not for maps though, as you can't delete without realloc.

Jan Mercl

unread,
Mar 13, 2021, 12:01:11 PM3/13/21
to Kevin Chadwick, golang-nuts
On Sat, Mar 13, 2021 at 5:52 PM Kevin Chadwick <m8il...@gmail.com> wrote:

> Very little resources, unless the map is actually used and not for long. If you really need to control gos memory use, you need to preallocate arrays in a long standing manner anyway, not for maps though, as you can't delete without realloc.

How did you figure out "very little resources"?

What if there's a hot loop called a billion times doing something and
returning a non zero map in the very last call? We're then talking
about probably measurable slowdowns and gigabytes of garbage the
collector has to deal with. Allocating "preemptively" might be
sometimes a good idea, but that's a decision better left to humans as
the compiler usually does not have enough info to make that decision
right.

Ignoring for now other techniques that might get the cost of this
down. They exist.

Kevin Chadwick

unread,
Mar 13, 2021, 12:14:35 PM3/13/21
to golang-nuts
If you are doing something a billion times then whatever you are doing is more expensive than allocating a zero map. You can also avoid that return if really necessary, which I doubt. It is right for optimisation steps to increase, over run of the mill stuff.

Nonsense around performance is the reason that c sucks.

Axel Wagner

unread,
Mar 13, 2021, 12:27:55 PM3/13/21
to golang-nuts
On Sat, Mar 13, 2021 at 6:14 PM Kevin Chadwick <m8il...@gmail.com> wrote:
Nonsense around performance is the reason that c sucks.

Lack of "nonsense around performance" is also the reason why many people use Go instead of, say, Python.
 
I'm once again not sure what your objective is here. No one is trying to argue that it is impossible to introduce semantics like you propose.
We are only trying to give you the reasons why that's currently not done.

You can always file a proposal to change the semantics. But I would strongly suggest to a) get acquainted with the reasons given to you for why maps work the way they do and b) have good answers ready when they come up in the discussion. At least better answers than just dismissing them. If your only response to someone saying that performance would suffer to much if we had to allocate a map for every zero value is "that is performance nonsense", my prediction is that your proposal would not get very accepted.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

Kevin Chadwick

unread,
Mar 15, 2021, 9:19:18 AM3/15/21
to golang-nuts
On 3/13/21 5:27 PM, 'Axel Wagner' via golang-nuts wrote:

>  
> I'm once again not sure what your objective is here. No one is trying to argue
> that it is impossible to introduce semantics like you propose.
> We are only trying to give you the reasons why that's currently not done.
>
> You can always file a proposal to change the semantics. But I would strongly
> suggest to a) get acquainted with the reasons given to you for why maps work the
> way they do and b) have good answers ready when they come up in the discussion.
> At least better answers than just dismissing them. If your only response to
> someone saying that performance would suffer to much if we had to allocate a map
> for every zero value is "that is performance nonsense", my prediction is that
> your proposal would not get very accepted.

That was not my answer or "only response"!

>> If you are doing something a billion times then whatever you are doing is
>> more expensive than allocating a zero map. You can also avoid that return if
>> really necessary, which I doubt. It is right for optimisation steps to
>> increase, over run of the mill stuff.

e.g. append vs array

>> Nonsense around performance is the reason that c sucks.

perhaps I should have said, nonsense by default, but it is inferred from context


> Lack of "nonsense around performance" is also the reason why many people use
> Go instead of, say, Python.

Whilst you have wrongly called some things in this thread irrelevant.
That is irrelevant to this discussion as the reasons that Python is slower have
very little to do with this discussion. Whilst issues like this and ones already
fixed by go, do pertain to c.

I shall see if the staticcheck SA5000 linter can be improved to track returns or
submit a proposal at some point or wait for null safety to fix this issue.

Axel Wagner

unread,
Mar 15, 2021, 9:53:15 AM3/15/21
to Kevin Chadwick, golang-nuts
On Mon, Mar 15, 2021 at 2:19 PM Kevin Chadwick <m8il...@gmail.com> wrote:
Whilst you have wrongly called some things in this thread irrelevant.

To be clear: I meant they are not relevant to the discussion of the property that a zero value is represented by 0 bytes.
I explained why (they are questions about the semantics of using maps, whereas the memory representation is an implementation detail) and I stand by that.

That is irrelevant to this discussion as the reasons that Python is slower have
very little to do with this discussion.

Certainly, if "C gives up too much convenience for the sake of performance" is pertinent to the discussion, then so is "Python gives up too much performance for the sake of convenience"? I would tend to agree that *neither* is particularly relevant (as comparing languages is relatively futile) but saying one is and one isn't seems an arbitrary double-standard to me.

To repeat, I didn't really try to make a case one way or another. I agree that it would be nice to not have nil-maps in the language. I also agree that we should not ignore the performance impact of that. I'm just trying to explain why things are the way they are.

Whilst issues like this and ones already
fixed by go, do pertain to c.

I shall see if the staticcheck SA5000 linter can be improved to track returns or
submit a proposal at some point or wait for null safety to fix this issue.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages