Golang inconsistencies

[jemeshsu]

This article highlights one aspect of Golang (New & Make) which might
drive new users away:
http://seanerussell.blogspot.com/2011/06/now-we-get-to-annoying-aspects-of-go.html

I'm new to Go but hope experts can weight in as to why Go is designed
this way. Some harsh comments from Reddit threads:
http://www.reddit.com/r/programming/comments/ii3yb/now_we_get_to_the_annoying_aspects_of_go/

[Ian Lance Taylor]

jemeshsu <jeme...@gmail.com> writes:

It's been discussed several times before--see the list archives. No
specific solution has ever seemed completely satisfactory.

Ian

[John Asmuth]

Not to disagree specifically, but a quick search of the group archives yielded many +1s. Is there a summary of the objection, somewhere?

[Jessta]

On Thu, Jul 7, 2011 at 3:57 AM, John Asmuth <jas...@gmail.com> wrote:
> Not to disagree specifically, but a quick search of the group archives
> yielded many +1s. Is there a summary of the objection, somewhere?

"In short: new allocates memory, make initializes the slice, map, and
channel types."
Some structs need initialization to be useful. If would be very
inconsistent if make() initialized slices,maps, and channels, but only
allocated memory for structs. There would also have to be some magic
where make() sometimes returned a pointer and sometimes returned a
value depending on what you asked it to do.

The current case is consistent with make() and new() performing distinct tasks.

--
=====================
http://jessta.id.au

[chris dollin]

On 6 July 2011 18:02, jemeshsu <jeme...@gmail.com> wrote:
> This article highlights one aspect of Golang (New & Make) which might
> drive new users away:
> http://seanerussell.blogspot.com/2011/06/now-we-get-to-annoying-aspects-of-go.html
>
> I'm new to Go but hope experts can weight in as to why Go is designed
> this way.

I've never really understood what the problem with Go's distinguishing
make() and new() is. You use make() to create -- that is, construct values
for -- slices, channels, and maps. You use new() when you want to allocate
a pointer to a fresh copy of the zero value of the pointed-to type, and you
almost never actually need to use it. Those are two different kinds of
action.

The proposals I've seen fly past essentially all trade the bump
in the carpet that is the new/make distinction for a different bump,
bigger but more obscure.

Chris

--
Chris "not a Go designer" Dollin

[John Asmuth]

The idea was that you wouldn't make a struct, you'd make a pointer to it, which would initialize the contents. new(T) would become make(*T).

[andyb...@gmail.com]

make() is used with the types that have a degree of reference semantics without being pointers. Needing to use make() reminds users that they are special.

(But it would be nice if the zero value for maps and channels were usable, like the zero value for slices is.)

[andyb...@gmail.com]

I suppose there is a good technical reason why maps and channels can't be initialized automatically the first time they're used. I just don't know what it is.

[andrey mirtchovski]

> I suppose there is a good technical reason why maps and channels can't be
> initialized automatically the first time they're used. I just don't know
> what it is.

a channel needs to know its buffer size.

[Jessta]

The zero values for channels are 'usable', you can use a nil channel
in a select and it will be ignored.
It would be nice if:
map['something'] = "something"
make()'d the map if it wasn't already make()'d

--
=====================
http://jessta.id.au

[chris dollin]

For starters, it would be impossible to have a nil map/channel.

And it would be difficult to see the point of initialisation, since any
use might be that point.

Chris

--
Chris "allusive" Dollin

[chris dollin]

On 6 July 2011 20:20, Jessta <jes...@jessta.id.au> wrote:

> It would be nice if:
> map['something'] = "something"
> make()'d the map if it wasn't already make()'d

I think you would then get complaints from people from the seqence:

var M someMapType
someFunction(M)
... look at M ...

...
func init(M' someMapType) { M'[K] = V }

where M would not be initialised.

[bflm]

On Wednesday, July 6, 2011 7:02:17 PM UTC+2, jemeshsu wrote:
> This article highlights one aspect of Golang (New & Make) which might
> drive new users away:
> http://seanerussell.blogspot.com/2011/06/now-we-get-to-annoying-aspects-of-go.html

There is probably only a finite number of reasonable ways how to
define the semantics of 'new' and 'make' (and/or the existence of them
both/one of them only). The current semantics of those builtins is IMO
almost trivial (for a programmer with experiences in a true-pointer-
type-exists language). Pleasing the new users is good - e.g. from the
"marketing" POW, not necessairly from the language designer(s) POW -
which hopefuly is close to the POW of the "power" users of Go. And to
criticize a language design, one may not yet have a good graps off, is
easy, regardeless of [un]fairness of such comparison..

> I'm new to Go but hope experts can weight in as to why Go is designed
> this way. Some harsh comments from Reddit threads:
> http://www.reddit.com/r/programming/comments/ii3yb/now_we_get_to_the_annoying_aspects_of_go/

That's a bit of "mission [close to] impossible". To understand the
reasons - one has to understand some amount of the Go concepts. Once
one gets familiar with those concepts, the very same question quite
probably - vanishes ;-) I'm aware this sounds like cheating/fallacy/
tautology/choose your favorite... Not in any way because of my
intention.

[Steven Blenkinsop]

On Wednesday, July 6, 2011, John Asmuth <jas...@gmail.com> wrote:
> The idea was that you wouldn't make a struct, you'd make a pointer to it, which would initialize the contents. new(T) would become make(*T).

(ignoring the past 20 minutes of rapid fire discussion)

+1 ;-) make([]T, 5) only initializes the slice, not the elements. I
don't see why make(*T) would be expected to behave any different.
Arguments against this either conflate pointers with what they are
pointing to, or are getting confused by aspects of the implementation
which aren't part of the language definition. A pointer is a reference
to a single element. A slice is a reference to multiple ordered
elements [belonging to an array]. The amount of initialization for
each of these two concepts is implementation dependent, and you cannot
dismiss that both have to be initialized by saying one is more complex
than the other.

By the same token, I'd like to see pointer composite literals
(`(*int){5}`), but acknowledge that they aren't necessary (there's
only a few cases where they'd be useful), and as such, might just be
consistency without reason (which is bad). They would be useful for
making an in-place copy to pass to a function, though.

As far as stumbling blocks go, make/new isn't all that bad compared to
`:=`, but I find the false dichotomy intellectually annoying
(obviously not a language design argument :), and we've seen that the
distinction does confuse new users... People who get past this
confusion come to accept that new *makes* a pointer to the type passed
to it. Using make for pointers would remove this problem entirely,
since the code is identical to the sentence describing it.

[Hilco Wijbenga]

What I've never understood is why that distinction *requires* two
different functions/operators? Doesn't the compiler know it's dealing
with a slice/map/channel? What prevents new(slice/map/channel) from
simply doing what make(slice/map/channel) does today? Or the other way
around, of course.

[peter]

I haven't figured this out yet.

These seem to do the same thing:

m1 := map[string] int{}
m2 := make(map[string] int)

And so do these:

p1 := new(map[string] int)
p1 = &m1
p2 := &m1
p3 := &(map[string] int{})
p3 = &m1

Or are they different?

When do you actually need to use new or make?

--
Peter

[Rob 'Commander' Pike]

http://golang.org/doc/go_faq.html#new_and_make

[Andy Balholm]

On Jul 6, 2011, at 2:56 PM, peter wrote:

> I haven't figured this out yet.
>
> These seem to do the same thing:
>
> m1 := map[string] int{}
> m2 := make(map[string] int)
>
> And so do these:
>
> p1 := new(map[string] int)
> p1 = &m1
> p2 := &m1
> p3 := &(map[string] int{})
> p3 = &m1
>
> Or are they different?

m1 and m2 should be indistinguishable.

By the time your second code block finishes, p1, p2, and p3 should all be the same (pointers to m1), but two uninitialized maps will have been allocated and left on the heap to be garbage-collected.

> When do you actually need to use new or make?

You should seldom (if ever) need to use new with a map type, because there is little need for a pointer to a map. Maps already act a lot like references without the extra indirection.

You need to use make every time you create a map. Even if for some reason you allocated a map with new, you would still need to initialize it with make before you could use it:

p4 := new(map[string]int)
*p4 = make(map[string]int)

[Andy Balholm]

On Jul 6, 2011, at 12:21 PM, chris dollin wrote:
> For starters, it would be impossible to have a nil map/channel.
>
> And it would be difficult to see the point of initialisation, since any
> use might be that point.

I see. A nil slice can be used without being initialized, since it automatically (because of zeroed memory) has a length and capacity of 0. But initializing maps and channels would violate Go's philosophy of not doing things behind your back.

[Peter Kleiweg]

Andy Balholm schreef op de 6e dag van de hooimaand van het jaar 2011:

>
> On Jul 6, 2011, at 2:56 PM, peter wrote:
>
> > I haven't figured this out yet.
> >
> > These seem to do the same thing:
> >
> > m1 := map[string] int{}
> > m2 := make(map[string] int)
> >
> > And so do these:
> >
> > p1 := new(map[string] int)
> > p1 = &m1
> > p2 := &m1
> > p3 := &(map[string] int{})
> > p3 = &m1
> >
> > Or are they different?
>
> m1 and m2 should be indistinguishable.
>
> By the time your second code block finishes, p1, p2, and p3
> should all be the same (pointers to m1), but two uninitialized
> maps will have been allocated and left on the heap to be
> garbage-collected.

I noticed I can add items to the maps m1 and m2, and to the one
pointed to by p3, before I do p3 = &m1
That makes three initialized maps.

Where are the uninitialized maps? Or do you mean empty maps?

I can't add items to the map where p1 is pointing to before I do
p2 := &m1

Before, p2 seems to be just a pointer pointing nowhere. But,
this isn't correct? It actually points to a map, but you can't
use it, you can't put things in it. Then what is it it points
to?

But if I use new for a struct, I get a pointer to a newly
created instance of that struct, with all values set to zero.

This looks as if two different language things are used for
something that is actually just an implementation issue.

> > When do you actually need to use new or make?
>
> You should seldom (if ever) need to use new with a map type,
> because there is little need for a pointer to a map. Maps
> already act a lot like references without the extra
> indirection.

Are there other variables that require new? I can also create a
struct without new:

type t struct { a, b, c int }
tt := t{}

> You need to use make every time you create a map. Even if for
> some reason you allocated a map with new, you would still need
> to initialize it with make before you could use it:
>
> p4 := new(map[string]int)
> *p4 = make(map[string]int)

I thought make created the map. How can it initialize a map that
is already created?

This is indeed a tricky subject for a newcomer like me.

--
Peter Kleiweg
http://pkleiweg.home.xs4all.nl/

[Peter Kleiweg]

Kyle Lemons schreef op de 6e dag van de hooimaand van het jaar 2011:

> > When do you actually need to use new or make?
> >

> well, you *need* to use make if you're creating a slice, map, or channel
> that has a specified internal size or capacity. These aren't available as a
> part of a composite literal, and the cases in which you need them are pretty
> distinct from the cases in which a composite literal is used. You also
> cannot create a *int without using new, not that you should be creating one
> anyway (again unless you know what you're doing). I should also mention
> that using new on an interface type is not common, despite looking identical
> to a new of any other named type.

I haven't studied channels yet. I understand the need to use
make for a slice or array. What can I do with make to create a
map that I can't do without? You cannot specify a capacity for a
map, can you?

> A simpler way to express it might be this: Use a composite literal when you
> know most of what the value is going to be. Use new when you really just
> need a pointer to a newly allocated object and are going to fool around with
> its value on the fly. Use make when you need an empty map/slice or channel
> or want to give it a capacity, length, or size---just like the previous
> case, you will be constructing its value on the fly.

I think I will avoid using new for the time being.

[eaburns]

On Wednesday, July 6, 2011 3:14:00 PM UTC-7, Andy Balholm wrote:

You need to use make every time you create a map. Even if for some reason you allocated a map with new, you would still need to initialize it with make before you could use it:

p4 := new(map[string]int)
*p4 = make(map[string]int)

Now I am a bit confused:

Why would I ever need to use make for a map if I can just use 'm := map[string]int{}' instead?

Why would I ever need to use new for a map if I can just use 'p := &map[string]int{}' instead?

The same questions go for slices too.

It seems to me that the only reason to use make/new is when you want to create a channel.  This is because, as far as I am aware, channels cannot be created as a literal.

Ethan

[eaburns]

On Wednesday, July 6, 2011 3:56:37 PM UTC-7, eaburns wrote:

The same questions go for slices too.

I take that back.  As was already pointed out, slices need make because you may want to specify a capacity. 

[David Symonds]

On Thu, Jul 7, 2011 at 8:56 AM, eaburns <burns...@gmail.com> wrote:

> Why would I ever need to use make for a map if I can just use 'm :=
> map[string]int{}' instead?
> Why would I ever need to use new for a map if I can just use 'p :=
> &map[string]int{}' instead?
> The same questions go for slices too.
> It seems to me that the only reason to use make/new is when you want to
> create a channel.  This is because, as far as I am aware, channels cannot be
> created as a literal.

make can take arguments to set a map's capacity, or set a slice's
length/capacity. You can't do that with a literal.

Dave.

[Rob 'Commander' Pike]

'New' is not some arbitrary piece of cruft you can ignore when it bothers you. It's the memory allocator.

I suggest using 'make' when you need to create a channel, slice, or map, and 'new' when you need to allocate new storage for a zero value of an item.

The composite literal syntax (&x{a,b,c}) is a convenience hack; under the covers it still calls 'new', and it only applies to composites (hence the name).

I get the feeling some people in this discussion do not appreciate the differences between pointers and values, or between zero values and initialized data structures. To use Go well, you must know how memory and addressing works.

-rob

[Peter Kleiweg]

On 7 jul, 00:51, Peter Kleiweg <pklei...@xs4all.nl> wrote:


> I haven't studied channels yet. I understand the need to use
> make for a slice or array. What can I do with make to create a
> map that I can't do without? You cannot specify a capacity for a
> map, can you?

O, I see you can. It's not in the tutorial, but it's in the language
specification.

--
Peter Kleiweghttp://pkleiweg.home.xs4all.nl/

[Peter Kleiweg]

On 7 jul, 01:00, "Rob 'Commander' Pike" <r...@google.com> wrote:

> I get the feeling some people in this discussion do not appreciate the differences between pointers and values, or between zero values and initialized data structures. To use Go well, you must know how memory and addressing works.

This is how I understand pointers and values from C:

struct T
s,
*sp;

sp = (struct T *) malloc (sizeof (struct T));

s.a = 1;
sp->a = 1;

But in Go, it seems to be more complex.


--
Peter

[skyb...@google.com]

On Wednesday, July 6, 2011 4:00:42 PM UTC-7, r wrote:

I get the feeling some people in this discussion do not appreciate the differences between pointers and values, or between zero values and initialized data structures. To use Go well, you must know how memory and addressing works.

There are some who don't understand it. Others do understand it but don't appreciate its worth, and therefore are annoyed at having to remember the distinction.  To people coming from other languages, uninitialized zero values look like an annoying low-level detail that would ideally be swept under the rug, rather than a fundamental concept that everyone needs to be aware of. Since many high-level languages don't have them, it appears at first glance that it should be easy to do.

- Brian

[Kyle Lemons]

I haven't studied channels yet. I understand the need to use
make for a slice or array. What can I do with make to create a
map that I can't do without? You cannot specify a capacity for a
map, can you?

You can indeed.  Have another look at the spec under #Making_slices_maps_and_channels

 

> A simpler way to express it might be this: Use a composite literal when you
> know most of what the value is going to be.  Use new when you really just
> need a pointer to a newly allocated object and are going to fool around with
> its value on the fly.  Use make when you need an empty map/slice or channel
> or want to give it a capacity, length, or size---just like the previous
> case, you will be constructing its value on the fly.

I think I will avoid using new for the time being.

That's probably fine.  You can't really avoid make, though. 

[Ian Lance Taylor]

Peter Kleiweg <pkle...@xs4all.nl> writes:

>> On Jul 6, 2011, at 2:56 PM, peter wrote:
>>
>> > I haven't figured this out yet.
>> >
>> > These seem to do the same thing:
>> >
>> > m1 := map[string] int{}
>> > m2 := make(map[string] int)
>> >
>> > And so do these:
>> >
>> > p1 := new(map[string] int)
>> > p1 = &m1
>> > p2 := &m1
>> > p3 := &(map[string] int{})
>> > p3 = &m1
>> >

...

> I can't add items to the map where p1 is pointing to before I do
> p2 := &m1
>
> Before, p2 seems to be just a pointer pointing nowhere. But,
> this isn't correct? It actually points to a map, but you can't
> use it, you can't put things in it. Then what is it it points
> to?

I'm not sure whether you mean p2 here. In general, if you have a
pointer to a map, and the pointer is not initialized, then it doesn't
point to anything. The uninitialized valuee for a pointer is nil.

> But if I use new for a struct, I get a pointer to a newly
> created instance of that struct, with all values set to zero.

Yes.

> This looks as if two different language things are used for
> something that is actually just an implementation issue.

In Go, a pointer can point to something, or it can be nil. An
uninitialized pointer is nil. I don't consider this to be an
implementation issue.

> Are there other variables that require new? I can also create a
> struct without new:
>
> type t struct { a, b, c int }
> tt := t{}

No types require "new". You would use "new" when you want to get a
pointer to an uninitialized value of some type. The code here is not
creating a pointer; if you wrote &t{}, it would create a pointer. An
example of using new would be something like
p := new(int)

>> You need to use make every time you create a map. Even if for
>> some reason you allocated a map with new, you would still need
>> to initialize it with make before you could use it:
>>
>> p4 := new(map[string]int)
>> *p4 = make(map[string]int)
>
> I thought make created the map. How can it initialize a map that
> is already created?

The new function does not create a map. It creates an uninitialized
value of a given type, and returns a pointer to it. The uninitialized
value for a map type is nil. If you want to add something to the map,
you need to give it a value; you can do that using make.

Ian

[Ian Lance Taylor]

Peter Kleiweg <pkle...@xs4all.nl> writes:

Go is the same, except that '->' is replaced by '.' and you use new
rather than malloc.

var s T
var sp *T
sp = new(T)
s.a = 1
sp.a = 1

Ian

[Namegduf]

Key word is "seems". Pointers and values behave in the same way in Go,
and Go also passes parameters by value, with the same consequences as
in C. There's no additional complexity here; the principle difference
is that . is used for members of both values and pointers, and method
receivers exist (but behave exactly like parameters do).

The complex part is initialisation, which is also tricky in C but not a
problem the language tries to solve.

new() performs no initialisation, returning a pointer to zeroed memory.
This is the usual way to create most types, built-in and user-defined,
on the heap.

make() performs initialisation of, and returns a value for one
of the built-in types that require initialisation.

pkg.NewSomething() functions are usually the way to provide
initialisation for a user-defined type, but unlike make(), often
returns a pointer.

I don't personally think there's anything wrong with this. Knowing
whether you need to use new() or initialise the type is a type-specific
thing you need to know for all types you use, but I've not seen any
solutions that extend beyond the built-in types, which are the least
problematic, as anyone who's used Go for any length of time can
remember them.

I think it would be more inconsistent for built-in types to be exempted
from this issue, technically speaking.

As for the blog post... it does not appear to be using a standard
meaning of inconsistency, instead substituting "surprised me", which I
think is a little presumptious, as well as wrong, as well as claiming
that he has to know "implementation details", which is also wrong.
Knowing you need to know whether types require initialisation is a tad
tricky at first, but is something he should have tried harder to
understand before blaming the language and throwing around unpleasant
words with disregard for their meaning.

The part on pointers seems to be speculative, as I cannot see the
proposed scenario in which the user doesn't know what the methods he is
calling do, even whether they modify the type at all, actually
occurring; regardless, if it somehow did, a glance at the signature
would tell him if it could modify the type. For any sizeable struct it's
obviously better to just pass around pointers to begin with.

At any rate, I don't feel the accusations of "inconsistency" hold any
strength. Reddit users may have harsh comments, but the opinions of
news site commenters aren't particularly worth caring about.

[Ian Lance Taylor]

skyb...@google.com writes:

Go has precise memory layout and it has pointers. Given that, what is
the value of an uninitialized pointer?

Personally, I view having an uninitialized value of map type be nil as
basically an efficiency hack. It lets us assume that that all
uninitialized variables can be zero. If an uninitialized map were not
nil--if creating a map did not require a call to make or the use of a
composite literal--then I don't see any way to retain the simplicity and
efficiency of having uninitialized values be zero.

Ian

[Steven Blenkinsop]

On Wednesday, July 6, 2011, Rob 'Commander' Pike <r...@google.com> wrote:
> I get the feeling some people in this discussion do not appreciate the differences between pointers and values, or between zero values and initialized data structures. To use Go well, you must know how memory and addressing works.

This is where I think `new(T)` is creating some harm. You can do `t :=
new(T)` and ignore that t isn't a T but is rather a *T. Obviously,
this isn't a problem for people who grok pointers, but for people who
are new to them, it gives them an avenue to ignore them, which just
creates problems. The benefit of `make(*T)` is that it forces people
to think of pointers as a distinct kind of type, like slices, maps,
and channels, rather than glossing over them as a quirky thing about
value types. I also don't think the parallel people draw to the Java
keyword "new" is all that helpful if they don't realize that Java
objects are really pointers.

[andyb...@gmail.com]

On Wednesday, July 6, 2011 3:42:49 PM UTC-7, Peter Kleiweg wrote:

I noticed I can add items to the maps m1 and m2, and to the one
pointed to by p3, before I do p3 = &m1
That makes three initialized maps.

Where are the uninitialized maps? Or do you mean empty maps?

 I made a mistake. There is only one uninitialized map (the one pointed to by p1). p2 would point to a map that is initialized but empty, because a map literal creates an initialized map.

[Kyle Lemons]

These seem to do the same thing:

So does "a+b" and "b+a".  Having more than one way to express the same code is nothing new.

 

m1 := map[string] int{}
m2 := make(map[string] int)

While these might make equivalent values, they are---idiomatically---quite different.  A composite literal is typically used when you know all or most of the values of the map, slice, or structure.  Make and new are more often used to create the value that will be built up through functions, loops, closures, channels, whatever.  So, in your case, m1 might be expected to be by itself, whereas m2 would be somewhat odd unless it were followed by a loop that was setting some computed map keys.

 

And so do these:

p1 := new(map[string] int)
p1 = &m1
p2 := &m1
p3 := &(map[string] int{})
p3 = &m1

Unless you really know what you're doing, it's rarely necessary to use & except in a composite structure literal.  You can generate a countably infinite number of equivalent expressions by simply adding extra &/*s. 

 

Or are they different?

When do you actually need to use new or make?

well, you *need* to use make if you're creating a slice, map, or channel that has a specified internal size or capacity.  These aren't available as a part of a composite literal, and the cases in which you need them are pretty distinct from the cases in which a composite literal is used.  You also cannot create a *int without using new, not that you should be creating one anyway (again unless you know what you're doing).  I should also mention that using new on an interface type is not common, despite looking identical to a new of any other named type.

A simpler way to express it might be this: Use a composite literal when you know most of what the value is going to be.  Use new when you really just need a pointer to a newly allocated object and are going to fool around with its value on the fly.  Use make when you need an empty map/slice or channel or want to give it a capacity, length, or size---just like the previous case, you will be constructing its value on the fly.

HTH

~K

[Han-Wen Nienhuys]

Personally the only thing that irks me with make() is having to write
the type again when initializing structs,.

Couldn't there be a polymorphic built-in that looks at the type of
argument to correctly initialize the object? eg.

type Foo struct {
slice map[string]*VeryLongTypeName
}
func NewFoo() *Foo {
f := &Foo{}
make(&f.slice) // shorter than: make(map[string]*VeryLongTypeName)
return f
}

--
Han-Wen Nienhuys
Google Engineering Belo Horizonte
han...@google.com

[Steven Blenkinsop]

On Wed, Jul 6, 2011 at 10:15 PM, Han-Wen Nienhuys <han...@google.com> wrote:

Personally the only thing that irks me with make() is having to write
the type again when initializing structs,.

Couldn't there be a polymorphic built-in that looks at the type of
argument to correctly initialize the object? eg.

type Foo struct {
 slice map[string]*VeryLongTypeName
}
func NewFoo() *Foo {
 f := &Foo{}
 make(&f.slice)  // shorter than: make(map[string]*VeryLongTypeName)
 return f

}

type ShortName map[string] *VeryLongTypeName 

f.slice := make(ShortName) // I wouldn't call a map "slice" though

[⚛]

This forum thread is amazing. It is like throwing a stick at a hornet
nest.

Simply: new(T) returns *T, while make(T) returns T. That is all that
is to know here.

On Jul 6, 7:02 pm, jemeshsu <jemes...@gmail.com> wrote:
> This article highlights one aspect of Golang (New & Make) which might
> drive new users away:http://seanerussell.blogspot.com/2011/06/now-we-get-to-annoying-aspec...
>
> I'm new to Go but hope experts can weight in as to why Go is designed
> this way. Some harsh comments from Reddit threads:http://www.reddit.com/r/programming/comments/ii3yb/now_we_get_to_the_...

[Rob 'Commander' Pike]

On Jul 7, 2011, at 5:20 PM, ⚛ wrote:

> Simply: new(T) returns *T, while make(T) returns T. That is all that
> is to know here.

No: new(T) returns a *T pointing to a zeroed T, while make(T) returns an initialized (not zeroed) T. In other words, new allocates; make initializes.

If you come from a language where new initializes, this can confuse you. If you come from a language where everything is an object, this can confuse you. If you come from a language where everything is a pointer but you pretend it's not, this can confuse you.

There are lots of languages with those properties. Go isn't one of them; instead Go reflects the nature of the underlying machine model.

-rob

[Adam Richardson]

Thank you! I know this is in the documentation (on the "Effective Go" page), but I was beginning to question my recall/understanding after reading the commentary within this thread.

Much better. And, for the record, I appreciate this language property.

Adam

[Gheorghe Postelnicu]

Hi Adam and Rob,

I like the transparency of the property as well. However, that transparency goes away when you are in the presence of a typedef-like statement which obscures the type of resource you're working with.

From my point of view, this thread seems to get lost into saying: why we have make and new, as opposed to addressing the issue of a potential language inconsistency. I am reading this with a lot of interest, as I started recently using the language.

Cheers,

Gheorghe

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Certainly this is a true statement. The question is whether it is the
best way to define things. A few notes on my part:

1) The EffectiveGo documentation recommends that you write initializers
as: NewMyType() *MyType
It explicitly uses:
func NewFile(fd int, name string) *File

And *all* of these "New" functions are returning initialized types. So
there is at least some confusion because:

new(MyType) is uninitialized but
NewMyType() is initialized

If new/make distinction is to be followed, it would make a lot more
sense to have:
MakeMyType() *MyType

2) 'make' can only be used for Go-defined types, and I don't think all
of them at that. (you can't "make(string, 10)" for example, though
partially that is because strings are immutable.)

But it does mean that I can't define some-special-sauce and get:
make(MyType)
return an initialized version of MyType.

This shows up in other areas, like "range" not working on things that
aren't built-in types.

3) Go seems to be moving away from obviously defined storage. In that
saying:

foo := MyType{}

Means it is allocated on the stack, unless you then do:
bar := &foo

At which point the previous 'foo' declaration now is allocated on the heap.

If the language wants to hide whether something is on the stack or on
the heap, then why do we need "new" at all? (What use cases of 'new'
aren't covered by &MyType{}. It is even fewer characters to type than
new(MyType).)

4) new/make gets more confusing after type declarations. It is pretty
obvious if you see:

x := make(map[string]string)

x := new(map[string]string) # is "obviously" wrong

It is no longer obvious after a type declaration, especially at a distance:
type Dict map[string]string

x := Dict{}
x := make(Dict)
x := new(Dict)

5) I think "deprecating" new and looking at a way to have make
initialize user-defined types would be nice. (especially as that like
leads the way to have 'range' work on more types, etc.) "magic" built-in
objects that have different semantics than all other objects tend to
chafe a bit. It may be a necessity, but thing like Python's magic
members (__init__, __len__, __getitem__) do add complexity, but avoid
the specialization, I think.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4VkdAACgkQJdeBCYSNAAMLhgCcCR8yyO61CWK/MqFoXYeHTYiq
1JYAoM+OrtgGV0E2volCJYWp+ThbW3OG
=ia0r
-----END PGP SIGNATURE-----

[chris dollin]

On 7 July 2011 12:00, John Arbash Meinel <jo...@arbash-meinel.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 7/7/2011 9:34 AM, Rob 'Commander' Pike wrote:
>>
>> On Jul 7, 2011, at 5:20 PM, ⚛ wrote:
>>
>>> Simply: new(T) returns *T, while make(T) returns T. That is all that
>>> is to know here.
>>
>> No: new(T) returns a *T pointing to a zeroed T, while make(T) returns an initialized (not zeroed) T. In other words, new allocates; make initializes.
>>
>> If you come from a language where new initializes, this can confuse you. If you come from a language where everything is an object, this can confuse you. If you come from a language where everything is a pointer but you pretend it's not, this can confuse you.
>>
>> There are lots of languages with those properties. Go isn't one of them; instead Go reflects the nature of the underlying machine model.
>>
>> -rob
>>
>
> Certainly this is a true statement. The question is whether it is the
> best way to define things. A few notes on my part:
>
> 1) The EffectiveGo documentation recommends that you write initializers
> as:  NewMyType() *MyType
> It explicitly uses:
>     func NewFile(fd int, name string) *File
>
> And *all* of these "New" functions are returning initialized types. So
> there is at least some confusion because:
>
>  new(MyType) is uninitialized but

It [the pointed-to storage] /is/ initialised, to a zero value.

>  NewMyType() is initialized

to something as its documentation specifies, same as for
new().

> If new/make distinction is to be followed, it would make a lot more
> sense to have:
>  MakeMyType() *MyType

Maybe. Now were just talking naming conventions.

> 2) 'make' can only be used for Go-defined types,

In fact only for chanels, maps, and slices. That's what it's
there for: to do things that can't otherwise be done, to be
the base case for non-zero values for those types.

> But it does mean that I can't define some-special-sauce and get:
>  make(MyType)
> return an initialized version of MyType.

You'll note that the language has /no/ built-in methods; which
is why:

> This shows up in other areas, like "range" not working on things that
> aren't built-in types.

> 3) Go seems to be moving away from obviously defined storage. In that
> saying:
>
>  foo := MyType{}
>
> Means it is allocated on the stack, unless you then do:
>  bar := &foo
>
> At which point the previous 'foo' declaration now is allocated on the heap.

Variables are allocated on the heap unless the compiler can show
that its correct to allocate them in the stack, which it can do
if their address is not taken (implicitly or explicitly). There's
no "at which point" in your example: foo is allocated on the heap
because its address is taken (and the compiler isn't smart
enough at present to see if, say, that address never leaks out
of the function its taken in).

> If the language wants to hide whether something is on the stack or on
> the heap, then why do we need "new" at all? (What use cases of 'new'
> aren't covered by &MyType{}.

ints, floats, complexes, strings.

> 4) new/make gets more confusing after type declarations. It is pretty
> obvious if you see:
>
>  x := make(map[string]string)
>
>  x := new(map[string]string) # is "obviously" wrong

Not in the least. It allocates a pointer to a zero-valued map.
That's a perfectly sensible thing to do.

> It is no longer obvious after a type declaration, especially at a distance:
>  type Dict map[string]string
>
>  x := Dict{}
>  x := make(Dict)
>  x := new(Dict)

I don't see the problem. All of those things make sense.
Knowing which one does what you want is a matter of what
you're trying to do.

Rather than trying to reconstruct this corner of the language,
I wonder if we can find a form of words for Effective Go and
the tutorial which is more effective at avoiding people
garden-pathing down misunderstandings. I'm probably
to close to the problem to see why it's a problem ...

Chris

--
Chris "allusive" Dollin

[bflm]

On Thursday, July 7, 2011 1:00:32 PM UTC+2, John Arbash Meinel wrote:

1) The EffectiveGo documentation recommends that you write initializers

as:  NewMyType() *MyType
It explicitly uses:
     func NewFile(fd int, name string) *File

And *all* of these "New" functions are returning initialized types. So
there is at least some confusion because:

  new(MyType) is uninitialized but
  NewMyType() is initialized

That's as much "confusion" as expecting a user defined function named 'Make' to have anything in common with the built in 'make'. 

If new/make distinction is to be followed, it would make a lot more
sense to have:
  MakeMyType() *MyType

Anyone is free to use such convention in his/her code. Go authors just happened to adopt a different convention. 

2) 'make' can only be used for Go-defined types, and I don't think all
of them at that.

Maps, channels and slices only.

(you can't "make(string, 10)" for example, though
partially that is because strings are immutable.)

No. 

But it does mean that I can't define some-special-sauce and get:
  make(MyType)
return an initialized version of MyType.

This shows up in other areas, like "range" not working on things that
aren't built-in types.

'range' works with "array, pointer to an array, slice, string, map, or channel.", i.e. also on user defined types.

3) Go seems to be moving away from obviously defined storage. In that
saying:

 foo := MyType{}

Means it is allocated on the stack, unless you then do:
 bar := &foo

Wrong. Stack or heap is never mentioned in Go specification. 

At which point the previous 'foo' declaration now is allocated on the heap.

If the language wants to hide whether something is on the stack or on
the heap, then why do we need "new" at all? (What use cases of 'new'
aren't covered by &MyType{}. It is even fewer characters to type than
new(MyType).)

There are not only structured types. You need 'new' for e.g. 'intptr := new(int)'.

4) new/make gets more confusing after type declarations. It is pretty
obvious if you see:

 x := make(map[string]string)

 x := new(map[string]string) # is "obviously" wrong

No, there is *nothing* wrong in the line above. It's perfectly legal and can easily make sense in code. 

It is no longer obvious after a type declaration, especially at a distance:
 type Dict map[string]string 

 x := Dict{}

A zero value of Dict iff Dict is a struct type.

 x := make(Dict)

An initialized (non zero) value of Dict iff Dict is a map or channel type.

 x := new(Dict)

A pointer to a zero value of Dict. Always.

5) I think "deprecating" new and looking at a way to have make
initialize user-defined types would be nice. (especially as that like
leads the way to have 'range' work on more types, etc.) "magic" built-in
objects that have different semantics than all other objects tend to
chafe a bit. It may be a necessity, but thing like Python's magic
members (__init__, __len__, __getitem__) do add complexity, but avoid
the specialization, I think.

Read Rob's and other posts. You can't throw 'new' from Go (i.e. a language with real value and real pointer types supported).

[Andrew Gerrand]

On Thursday, 7 July 2011 21:00:32 UTC+10, John Arbash Meinel wrote:

And *all* of these "New" functions are returning initialized types. So
there is at least some confusion because:

  new(MyType) is uninitialized but
  NewMyType() is initialized

No, new(MyType) returns a pointer to a zeroed value. NewMyType() is a function call. The latter is a convention, not a rule.

2) 'make' can only be used for Go-defined types, and I don't think all
of them at that. (you can't "make(string, 10)" for example, though
partially that is because strings are immutable.)

 Make can only initialize the Go built-in types that require initialization: slices, maps, and channels. They are special.

But it does mean that I can't define some-special-sauce and get:
  make(MyType)
return an initialized version of MyType.

No, you can't. You can only use make to initialize user-defined slice, map, and channel types.

This shows up in other areas, like "range" not working on things that
aren't built-in types.

Kind-of. Range works on slices, strings (a special type of slice), maps, and channels. (There's a pattern here.) 

3) Go seems to be moving away from obviously defined storage. In that
saying:

 foo := MyType{}

Means it is allocated on the stack, unless you then do:
 bar := &foo

At which point the previous 'foo' declaration now is allocated on the heap.

This isn't quite correct. We're working on compiler optimizations to put as much on the stack as possible. That includes values initialized with new(T) or &T{}, when possible.

If the language wants to hide whether something is on the stack or on
the heap,

The language isn't doing anything of the sort. This is an implementation detail, not part of the language.  

4) new/make gets more confusing after type declarations. 

It doesn't, really. This whole issue is the realm of rookie misunderstanding. The rules are simple and reflect the way things actually work. By understanding make and new work, you also understand how the type system works in a fundamental way. By exposing this "inconsistency" to the programmer we give them control and visibility over how their program actually works.    

5) I think "deprecating" new and looking at a way to have make
initialize user-defined types would be nice. (especially as that like
leads the way to have 'range' work on more types, etc.) "magic" built-in
objects that have different semantics than all other objects tend to
chafe a bit.

But these types _are_ special and they _do_ have different semantics. Why hide the reality? To me, _that_ would be confusing and inconsistent.

Andrew

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...

>> 5) I think "deprecating" new and looking at a way to have make
>> initialize user-defined types would be nice. (especially as that like
>> leads the way to have 'range' work on more types, etc.) "magic" built-in
>> objects that have different semantics than all other objects tend to
>> chafe a bit.
>>
> But these types _are_ special and they _do_ have different semantics. Why
> hide the reality? To me, _that_ would be confusing and inconsistent.
>
> Andrew
>

But why do they *need* to be different and have different semantics.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4VmL0ACgkQJdeBCYSNAAPqcACgyKKchnSPx0ofLl6ScJJtLDpZ
qucAn3btejSj+cXYbru2U8Aj+kic18Oj
=rGLx
-----END PGP SIGNATURE-----

[chris dollin]

On 7 July 2011 12:23, bflm <befeleme...@gmail.com> wrote:

> 'range' works with "array, pointer to an array, slice, string, map, or
> channel.", i.e. also on user defined types.

Not on structs, not on pointers to structs, not on typenamed ints or
strings or whatever. So if one were to create ones own container
type, one couldn't have range work on it (to iterate over all the
elements) without having to introduce channels & goroutines.

[Miguel Pignatelli]

On 07/07/11 12:00, John Arbash Meinel wrote:
> It is no longer obvious after a type declaration, especially at a distance:
> type Dict map[string]string
>
> x := Dict{}
> x := make(Dict)
> x := new(Dict)
>

This argument has arisen many times and I fully disagree with it.
Even "at a distance" you *must* know what is the type of Dict (you are
using it, after all!), and knowing its type it is fairly obvious which
to use.

Even if there is only "new" or only "make" you must know the type of
Dict to know what you are getting from new/make (initialization?, just
allocation?).

For me, having both makes the code simpler and more readable. If I see:

x := make(Dict)

I know that Dict is a slice, map or channel.

M;

[bflm]

The obviuos (I thought) meaning is ... "also on user defined types (of the just before explicitly enumerated type classes)" 

[bflm]

On Thursday, July 7, 2011 1:41:36 PM UTC+2, M; wrote:

For me, having both makes the code simpler and more readable. If I see:

x := make(Dict)

I know that Dict is a slice, map or channel.

Nitpicking: Not a slice without at least two arguments to make. 

[⚛]

On Jul 7, 9:34 am, "Rob 'Commander' Pike" <r...@google.com> wrote:
> On Jul 7, 2011, at 5:20 PM, ⚛ wrote:
>
> > Simply: new(T) returns *T, while make(T) returns T. That is all that
> > is to know here.
>
> No: new(T) returns a *T pointing to a zeroed T, while make(T) returns an initialized (not zeroed) T. In other words, new allocates; make initializes.

Indeed, as I already wrote: this forum thread is amazing.

> If you come from a language where new initializes, this can confuse you. If you come from a language where everything is an object, this can confuse you. If you come from a language where everything is a pointer but you pretend it's not, this can confuse you.

If you come from a language where "functions, interfaces, slices,
channels, and maps" are behind the curtain implemented as pointers
[see http://golang.org/doc/go_spec.html#The_zero_value] and you are at
the same time using * to distinguish pointer-types from value-types,
it will confuse people.

The confusion really has its roots in the fact that Go channels and
maps are not implemented as values. They are implemented as pointers.
So "x := new(map[int]int)" will in fact return a pointer to (a
structure containing) a pointer, and therefore (x != nil) and (*x ==
nil).

Go uses * to indicate a pointer. But there is nothing in "map[int]int"
which indicates the hidden pointer used to implement the map. AND
*THAT* IS CONFUSING.

Anybody denying that the above paragraphs are the accurate description
of the primary root of the confusion is wrong. The cause of the
confusion is in Go, not outside of Go - unlike some Go authors seem be
thinking.

[bflm]

On Thursday, July 7, 2011 1:51:41 PM UTC+2, ⚛ wrote:

Anybody denying that the above paragraphs are the accurate description
of the primary root of the confusion is wrong. The cause of the
confusion is in Go, not outside of Go - unlike some Go authors seem be
thinking.

Please put me on the "wrong" list. There is nothing said in the Go specs how maps, channels, ... are to be implemented. One specific implementation of Go can have this easily as a pointer, another as a struct (i.e. a value) and it doesn't matter at all. The semantics of those two differing implementation are identical.

Therefore, IMO you're wrong.

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm pretty sure that:

x := new(map[string]string]
x["foo"] = "bar"

Would have different effects if an implementation used a value rather
than a reference. (In reference version, it fails because you haven't
allocated the actual underlying object.)

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4VoOQACgkQJdeBCYSNAAPr4ACeP3CAzKdhQalD/v72LJDtZAi+
zy4AniOLV7DDX7lJXwX+ZNEJkr+bknhV
=LqcT
-----END PGP SIGNATURE-----

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...

>> 1) The EffectiveGo documentation recommends that you write initializers
>> as: NewMyType() *MyType
>> It explicitly uses:
>> func NewFile(fd int, name string) *File
>>
>> And *all* of these "New" functions are returning initialized types. So
>> there is at least some confusion because:
>>
>> new(MyType) is uninitialized but
>
> It [the pointed-to storage] /is/ initialised, to a zero value.
>

I'm going of Rob's explicit statement: "No: new(T) returns a *T pointing

to a zeroed T, while make(T) returns an initialized (not zeroed) T. In
other words, new allocates; make initializes."

You can argue that the 0 value is initialization or not. But the fact
that people have different ideas about what it means is why people
struggle with it. If you want the meme "new allocates, make initializes"
then it should be reflected in the Tutorial and the standard library.

I'm sure Rob doesn't speak for all developers. But it sure would be nice
if there was an official nomenclature for talking about these things.

>> NewMyType() is initialized
>
> to something as its documentation specifies, same as for
> new().
>

Saying that X does Y because it is documented to do so is a valid
statement. However, if I write this:

// Delete the file on disk
func Open(path string) os.Error {
}

If Open then deletes the file on disk, it is at least "surprising" if
not "inconsistent" with the naming of the function.

Having:

type MyType(map[string]string)

n := new(MyType)
vs
n := NewMyType()
vs
n := make(MyType)

They all do something which might be useful for some edge case. I at
least posit that the value isn't enough to warrant the confusion.

...

>> 2) 'make' can only be used for Go-defined types,
>
> In fact only for chanels, maps, and slices. That's what it's
> there for: to do things that can't otherwise be done, to be
> the base case for non-zero values for those types.

Sure. The open question is still "why can't it be done?". Why do you
have to have types which are language-special that user-defined types
cannot emulate.

>
>> But it does mean that I can't define some-special-sauce and get:
>> make(MyType)
>> return an initialized version of MyType.
>
> You'll note that the language has /no/ built-in methods; which
> is why:
>
>> This shows up in other areas, like "range" not working on things that
>> aren't built-in types.
>

I'm not sure what you mean by "/no/" built-in methods. range, len,
append, copy, cap are functions that come with the language (and cannot
be easily copied, if only because of type restrictions). Maps and slices
have [] syntax which also cannot be copied. Are you saying [] isn't a
method? I guess the point is that if you define new methods on a type
derived from map/slice you can't collide/override their built-in
functionality.

...

>> If the language wants to hide whether something is on the stack or on
>> the heap, then why do we need "new" at all? (What use cases of 'new'
>> aren't covered by &MyType{}.
>
> ints, floats, complexes, strings.

You could certainly have:
x := &1
or
y := 1
x := &1
or
x := &(1)

Strings... I really don't know what "new(string)" gives you that is
actually something interesting. If strings are immutable, then it is
just a pointer to the empty string, right?

>
>> 4) new/make gets more confusing after type declarations. It is pretty
>> obvious if you see:
>>
>> x := make(map[string]string)
>>
>> x := new(map[string]string) # is "obviously" wrong
>
> Not in the least. It allocates a pointer to a zero-valued map.
> That's a perfectly sensible thing to do.

What can you do with a "zero-valued map"? I'm genuinely curious, because
it seems that to do any actual functionality with it, you have to point
it somewhere.

>
>> It is no longer obvious after a type declaration, especially at a distance:
>> type Dict map[string]string
>>
>> x := Dict{}
>> x := make(Dict)
>> x := new(Dict)
>
> I don't see the problem. All of those things make sense.
> Knowing which one does what you want is a matter of what
> you're trying to do.

So far, I don't yet understand when new(map...) is useful. If it *can*
be useful, but is rarely so, it seems like something that falls under
the "we should consider pruning this for clarity".

>
> Rather than trying to reconstruct this corner of the language,
> I wonder if we can find a form of words for Effective Go and
> the tutorial which is more effective at avoiding people
> garden-pathing down misunderstandings. I'm probably
> to close to the problem to see why it's a problem ...
>
> Chris
>

As a semi-outsider, the new/make distinction just sounds like something
"you just have to learn". And when it gets discussed, people seem to
justify it based on grounds of "that is how it works". Which is
something you can live with, but always feels like a bump. The fact that
it comes up often also hints at that. There may be no better way to do it...

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4VoQsACgkQJdeBCYSNAANMTgCePhOZQq8WgUWrlxHq7y22TVZu
MI8AniuU5syKOVwIdBEFZYsmo8N2/Y8Y
=gqxZ
-----END PGP SIGNATURE-----

[chris dollin]

Oh. Well, working on under-defined types that are renames of
types range already works on isn't what I was thinking of; I
was thinking of /any// user-defined type with "the right methods".
The Go team seem to be set against magic built-in methods,
and I can sort of see why, but it is a bit of a tripwire IMAO.

[bflm]

On Thursday, July 7, 2011 2:04:52 PM UTC+2, John Arbash Meinel wrote:

I'm pretty sure that:

x := new(map[string]string]
x["foo"] = "bar"

Would have different effects if an implementation used a value rather
than a reference. (In reference version, it fails because you haven't
allocated the actual underlying object.)

Wrong in several ways:

1) 'x["foo"] = "bar" // error "invalid operation: x["foo"] (index of type *map[string] string)". That's because x is a pointer to map, not the map itself.

2) The line "x := new(map[string]string])" *does* allocate the zero value of the map.

3) If 1) is corrected to `(*map[string]string)x["foo"] = "bar"` it will compile OK, but fail due to the map *value" (*x) is not initialized. Therefore it's the same for any implementation (pointer based vs value based).

4) if 3 is corrected by adding `*x = make(map[string]string)`, the program will work. Again, no difference for pointer based vs value based implementation of Go.

It seems to me that you're maybe confusing yourself with assuming same special/artificial/non-existent special properties of maps/channels etc. The rules/semantics are very actually very simple - as stated in the specs.

[chris dollin]

On 7 July 2011 12:58, bflm <befeleme...@gmail.com> wrote:

>
> Please put me on the "wrong" list. There is nothing said in the Go specs how
> maps, channels, ... are to be implemented. One specific implementation of Go
> can have this easily as a pointer, another as a struct (i.e. a value) and it
> doesn't matter at all. The semantics of those two differing implementation
> are identical.
> Therefore, IMO you're wrong.

I don't think you can have the sharing properties of maps and slices
without a pointer or equivalent [1] being involved. This
isn't a property of the implementation, its a requirement placed
on the implementation by the spec.

a := make(map[string]string)
b := a
b["hello"] = "world"

... a["hello"] is now "world".

Chris

[1] eg an index into an implied array

--
Chris "allusive" Dollin

[bflm]

You're right that some pointers are involved in any reasonable/valid implementation of a Go map. You're wrong in the assumption that therefore a map must be a pointer per se. It is enough if a map is a value type, where one of it's bookkeeping fields is e.g. a pointer to a bucket list of the map. (Simplified a lot - but compare this to the implementation of a Go slice, which is also not a pointer and still has [partial] reference semantics).

[chris dollin]

On 7 July 2011 13:46, bflm <befeleme...@gmail.com> wrote:
> On Thursday, July 7, 2011 2:29:27 PM UTC+2, chris dollin wrote:

>> I don't think you can have the sharing properties of maps and slices
>> without a pointer or equivalent [1] being involved. This
>> isn't a property of the implementation, its a requirement placed
>> on the implementation by the spec.
>>
>> a := make(map[string]string)
>> b := a
>> b["hello"] = "world"
>>
>> ... a["hello"] is now "world".
>>
>> Chris
>>
>> [1] eg an index into an implied array
>
> You're right that some pointers are involved in any reasonable/valid
> implementation of a Go map. You're wrong in the assumption that therefore a
> map must be a pointer per se.

I did not make that assumption.

> It is enough if a map is a value type, where
> one of it's bookkeeping fields is e.g. a pointer to a bucket list of the
> map.

Yes. "... without a pointer or equivalent [1] being involved".

Maps share; HENCE the implementation requires pointers.
Your "value type" has to wrap a pointer to the sharable bit.

(It's not clear to me that there could be something useful in
the non-pointer part of the value object. Maybe a cache?)

[Namegduf]

Some types are "reference types", meaning copying the type's value with
an assign results in a second reference to the same value, and
changing either is reflected in both. This usually means the
implementation contains a pointer to (most of) the type's value, which
is allocated elsewhere.

If a type is as such, you need to know about it- although not, as you
claim, about how or why it is as such internally.

Some types require initialisation to be usable. If a type is like that,
you need to know about it, and know how to set it up yourself or what
functions are available for the task.

Some types are cheap to copy by value. Some aren't. It's good to know
about that, too, if trying to write performant code.

In almost all languages, user-defined types have all of the above
concerns. There's partial ways to reduce the last, like removing copying
by value entirely, with corresponding downsides. The second supposedly
can be fixed by constructors, but this simply replaces it with needing
to know what constructors are available taking what parameters, and what
the instance they produce provides.

The built-ins are not any more "inconsistent", than types in general
are in almost every other language. Could they avoid sharing the
concerns of the user-defined types? Maybe, with magic. But it wouldn't
make programming in Go easier; the built-ins are the easiest ones to
know about.

Go's types are no more inconsistent than types in many other languages.
It just doesn't try to hide it from you. This isn't a problem with Go,
this is a problem with people giving it a shallow look, seeing
differences between types, and failing to realise that they were
everywhere else, too, and they just hadn't known they needed to be
aware of it.

[bflm]

On Thursday, July 7, 2011 2:57:20 PM UTC+2, chris dollin wrote:

I did not make that assumption.

You're rigth. I mixed in my mind replies from you and from another poster. My apologies. 

> It is enough if a map is a value type, where
> one of it's bookkeeping fields is e.g. a pointer to a bucket list of the
> map.

Yes. "... without a pointer or equivalent [1] being involved".

Maps share; HENCE the implementation requires pointers.
Your "value type" has to wrap a pointer to the sharable bit.

(It's not clear to me that there could be something useful in
the non-pointer part of the value object. Maybe a cache?)

I don't know. It's the job of the implementor, what he/she may possibly see as an advantage to put in the "map" struct. Maybe there's no such thing. That doesn't invalidate the possibility of implementing a Go map as a value type, it would make it only silly ;-)

But the main idea here is IMO that regardless of the implementation, the semantics are in the specs and any valid implementation will behave the same way. Stated differently, it would be wrong, if the different implementation could be somehow observed/detected by code in a trivial way (sans unsafe etc.).

[chris dollin]

On 7 July 2011 14:14, bflm <befeleme...@gmail.com> wrote:
> On Thursday, July 7, 2011 2:57:20 PM UTC+2, chris dollin wrote:
>>
>> I did not make that assumption.
>
> You're rigth. I mixed in my mind replies from you and from another poster.
> My apologies.

No problem. It's a big hot thread.

> But the main idea here is IMO that regardless of the implementation, the
> semantics are in the specs and any valid implementation will behave the same
> way. Stated differently, it would be wrong, if the different implementation
> could be somehow observed/detected by code in a trivial way (sans unsafe
> etc.).

Concur.

[⚛]

On Jul 7, 3:00 pm, Namegduf <nameg...@gmail.com> wrote:
> On Thu, 7 Jul 2011 04:51:41 -0700 (PDT)
> ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
> > On Jul 7, 9:34 am, "Rob 'Commander' Pike" <r...@google.com> wrote:
> > > On Jul 7, 2011, at 5:20 PM, ⚛ wrote:
>
> > > > Simply: new(T) returns *T, while make(T) returns T. That is all
> > > > that is to know here.
>
> > > No: new(T) returns a *T pointing to a zeroed T, while make(T)
> > > returns an initialized (not zeroed) T. In other words, new
> > > allocates; make initializes.
>
> > Indeed, as I already wrote: this forum thread is amazing.
>
> > > If you come from a language where new initializes, this can confuse
> > > you. If you come from a language where everything is an object,
> > > this can confuse you. If you come from a language where everything
> > > is a pointer but you pretend it's not, this can confuse you.
>
> > If you come from a language where "functions, interfaces, slices,
> > channels, and maps" are behind the curtain implemented as pointers

> > [seehttp://golang.org/doc/go_spec.html#The_zero_value] and you are at

> > the same time using * to distinguish pointer-types from value-types,
> > it will confuse people.
>
> > The confusion really has its roots in the fact that Go channels and
> > maps are not implemented as values. They are implemented as pointers.
> > So "x := new(map[int]int)" will in fact return a pointer to (a
> > structure containing) a pointer, and therefore (x != nil) and (*x ==
> > nil).
>
> > Go uses * to indicate a pointer. But there is nothing in "map[int]int"
> > which indicates the hidden pointer used to implement the map. AND
> > *THAT* IS CONFUSING.
>
> > Anybody denying that the above paragraphs are the accurate description
> > of the primary root of the confusion is wrong. The cause of the
> > confusion is in Go, not outside of Go - unlike some Go authors seem be
> > thinking.
>
> Some types are "reference types", meaning copying the type's value with
> an assign results in a second reference to the same value, and
> changing either is reflected in both. This usually means the
> implementation contains a pointer to (most of) the type's value, which
> is allocated elsewhere.
>
> If a type is as such, you need to know about it- although not, as you
> claim, about how or why it is as such internally.

Well, but what if knowledge of the compiler's internal implementation
details of a Go map exposes the essence of "What a Go map is, and what
'new' is" with utmost clarity? A clarity that cannot be obtained by
merely reading the specification.

[bflm]

On Thursday, July 7, 2011 4:20:41 PM UTC+2, ⚛ wrote:

Well, but what if knowledge of the compiler's internal implementation
details of a Go map exposes the essence of "What a Go map is, and what
'new' is" with utmost clarity? A clarity that cannot be obtained by
merely reading the specification.

I disagree with 'cannot' in the last sentence above. Several people here are showing their IMO good understanding of this part of the Go specs.

[Steven Blenkinsop]

It's hard to get a word in edgewise though, with all the hullabaloo :).

One thing I keep seeing is people comparing `new(T)` and `make(T)` and saying "I have to know which one to use based on the type!".

Okay, the problem is, new(T) returns a *T, make(T) returns a T. The two aren't simply interchangeable based on what T is, they produce different types. The question is, is the zero value of T ready to use? For many types in the Go libraries, it is. In many other instances, it isn't. This determines whether `var t T` or `t := new(T)` is sufficient. Can you work with the zero value of T? In the case of maps, slices and channels, you can't. So you need to call a function to initialize that value (not the pointer). In the Go libraries, in this case, the type in question has a function or method that does this. For map, slice, and chan, you need a built-in function to do the initialization, and that's what make is for.

As for map being a pointer but not looking like a pointer, it isn't a pointer. You can't dereference it. You can't reach through it with a selection to select on its elements. It's implemented as a C pointer, but we can't access that C pointer. We can only access the map value, which has its own semantics that are different from the semantics of a Go pointer. A Go pointer is more than just a memory address. Its a set of interactions as well, and a map doesn't support those interactions.

Pointers, maps, chans, and slices are all reference types implemented using C pointers (and other bits as necessary). They are on equal standing as distinct kinds of types. All you need to know is that the zero value isn't usable, and once initialized, all copies of the same value refer to the same shared data. This could also be said of a library type, and isn't unique to the built-ins. Pointer types use `new` for this initialization, and because of its semantics, you have to pass the element type to it in order to get an initialized value of the pointer type. The others use make, which accepts the type itself and returns a value of that same type. 

Because pointers and slices (and to a certain degree, maps) have elements, you have to know whether the zero values of those element types can be used. If not, you also have to initialize those, or else call a function that initializes both the reference type and the element(s). This is what NewT functions normally do. They initialize a pointer, and initialize its element as necessary. It hides this extra step from you so you don't have to worry about it. If you don't want the user to care about your implementation, use a function to hide it.

[Steven Blenkinsop]

On Thursday, July 7, 2011, Rob 'Commander' Pike <r...@google.com> wrote:
> instead Go reflects the nature of the underlying machine model.

Where else does this hold? It doesn't seem to be generally true. It
seems that in this case, the decision was made based on the
implementation rather than the language abstractions, but I don't see
that as being prevalent elsewhere.

[bflm]

I think Rob was talking about the Go memory layout control and I think he is correct. In Go basic/primitive types (int, arrays, ...) are directly mapped to their memory image, same for struct types (with padding applied) - no hidden VMT pointers added and no other/similar tricks. Not applicable to non-primitive types (e.g. slices, interfaces, ...), where the memory layout is intentionally hidden from the programmer.

There probably may be something in the Go specs based/inspired by it's implementation, but I'm not aware of anything like that.

[Steven Blenkinsop]

On Fri, Jul 8, 2011 at 4:37 AM, bflm <befeleme...@gmail.com> wrote:

I think Rob was talking about the Go memory layout control and I think he is correct. In Go basic/primitive types (int, arrays, ...) are directly mapped to their memory image, same for struct types (with padding applied) - no hidden VMT pointers added and no other/similar tricks.

Fair enough. I don't see how this is the same meaning as "reflecting the nature of the underlying machine model" as could possibly be used with reference to `new` though. `new` seems to better reflect the underlying C model, rather than the machine model.

 

Not applicable to non-primitive types (e.g. slices, interfaces, ...), where the memory layout is intentionally hidden from the programmer.

And here we get to the exceptions. The thing is, slices, maps and channels are exactly the types we're talking about here. We're hiding their internals, and yet the argument for them using `make` while pointers use `new` is based on those very same internals.

 

There probably may be something in the Go specs based/inspired by it's implementation, but I'm not aware of anything like that.

And that's why `new` is such an oddity. `new` allocates zeroed memory and returns a reference to it. `make` (for slices) allocates zeroed memory and returns a reference to it. The only way to distinguish the two is that `new` works like C allocation (take info about the element type and return a pointer to the memory allocated for it), while `make` works like a C factory function (plus generics), which is necessary because the implementation of slices requires some extra initialization.

At the end of the day, it seems kind of like painting the bikeshed a different colour. Whether you type `new(T)` or `make(*T)`, you get the same result (well, `make(*T)` would work better for named pointer types, but those aren't common). But because of the slight difference in how they operate (`make` returns a `T`, `new` returns a `*T`) newcomers get confused, especially if they haven't been exposed to pointers much before.

So it's more like having a shed where one of the doors opens in and the other opens out, and there's no apparent reason for it except that that's how the contractor originally walked through the shed. If you use the shed everyday, you come to accept that this is how the doors work. But someone new will have trouble figuring out when they're supposed to pull and when they're supposed to push. If you make them both open the same way, the newcomer only has to care about which way they want to go, rather than worrying about how the door opens.

[bflm]

On Friday, July 8, 2011 7:46:54 PM UTC+2, Steven Blenkinsop wrote:

On Fri, Jul 8, 2011 at 4:37 AM, bflm <befeleme...@gmail.com> wrote:

I think Rob was talking about the Go memory layout control and I think he is correct. In Go basic/primitive types (int, arrays, ...) are directly mapped to their memory image, same for struct types (with padding applied) - no hidden VMT pointers added and no other/similar tricks.

Fair enough. I don't see how this is the same meaning as "reflecting the nature of the underlying machine model" as could possibly be used with reference to `new` though. `new` seems to better reflect the underlying C model, rather than the machine model.

In the case of simple/scalar and struct types I see the CPU memory model reflected 1:1 with the content of such entities. In C it's the same, i.e. both C and Go are "close to metal" in this.

 

Not applicable to non-primitive types (e.g. slices, interfaces, ...), where the memory layout is intentionally hidden from the programmer.

And here we get to the exceptions.

Intentional hiding of data structure (of non simplistic types) is a (IMHO good) design choice and not an exception. Even compared to other languages it's not an exception. Consider e.g. very popular (though not for me) Javsacript, which AFAIK has no control on memory layout at all - everything is opaque, every implementation can choose how to implement Javascript all entities.

 

The thing is, slices, maps and channels are exactly the types we're talking about here. We're hiding their internals, and yet the argument for them using `make` while pointers use `new` is based on those very same internals.

Agreed that both new and make are 'magic' builtins, mainly because they take a type name as their first parameter. This magic must be supported by the compiler. In contrast e.g. C has AFAIK almost no magic builtins ('sizeof' is magic). Even malloc and free are library functions.

There probably may be something in the Go specs based/inspired by it's implementation, but I'm not aware of anything like that.

And that's why `new` is such an oddity.

Well, I can say only that I see nothing odd about 'new'.

 

`new` allocates zeroed memory and returns a reference to it. `make` (for slices) allocates zeroed memory and returns a reference to it. The only way to distinguish the two is that `new` works like C allocation (take info about the element type and return a pointer to the memory allocated for it), while `make` works like a C factory function (plus generics), which is necessary because the implementation of slices requires some extra initialization.

There is a huge difference between new and make which some tend to overlook. 'v = make(T)' initializes an already existing entity 'v', but 'v = new(T)'' guarantees to (if succeeds at all) to return a pointer to - not surprisingly - a *new*, unique zero value of T. Analogically for ':=',  but let me ignore some details for now.

The just mentioned difference between make and new make them so distinct for me, that I fail to understand, how those two builtins can be confused at all. They heave IMO mostly nothing in common, except for the detail of the magic understanding of a type name as a parameter.

At the end of the day, it seems kind of like painting the bikeshed a different colour. Whether you type `new(T)` or `make(*T)`, you get the same result (well, `make(*T)` would work better for named pointer types, but those aren't common).

It's true, and it was discussed several times before, that 'make(*T)' can be considered as an (syntax sugar) replacement for 'new(T)'. What's left is the design choice made by the Go language designers (I'm not one of them). After diving myself into Go, I think they've made a good choice. Because aliasing the semantics of new(T) and make(*T) would/could - I guess - cause even more confusion for those, who already have trouble to get a grip on new vs make.

 

But because of the slight difference in how they operate (`make` returns a `T`, `new` returns a `*T`) newcomers get confused, especially if they haven't been exposed to pointers much before.

Reiterated: IMO - the difference is big, what they have in common is almost irrelevant.

 

So it's more like having a shed where one of the doors opens in and the other opens out, and there's no apparent reason for it except that that's how the contractor originally walked through the shed. If you use the shed everyday, you come to accept that this is how the doors work. But someone new will have trouble figuring out when they're supposed to pull and when they're supposed to push. If you make them both open the same way, the newcomer only has to care about which way they want to go, rather than worrying about how the door opens.

 

I do understand the words in this paragraph, I can parse the sentences, but I fail to get the message. My fault, granted.

[Steven Blenkinsop]

On Jul 8, 2011, at 2:57 PM, bflm <befeleme...@gmail.com> wrote:

On Friday, July 8, 2011 7:46:54 PM UTC+2, Steven Blenkinsop wrote:

On Fri, Jul 8, 2011 at 4:37 AM, bflm <befeleme...@gmail.com> wrote:

I think Rob was talking about the Go memory layout control and I think he is correct. In Go basic/primitive types (int, arrays, ...) are directly mapped to their memory image, same for struct types (with padding applied) - no hidden VMT pointers added and no other/similar tricks.

Fair enough. I don't see how this is the same meaning as "reflecting the nature of the underlying machine model" as could possibly be used with reference to `new` though. `new` seems to better reflect the underlying C model, rather than the machine model.

In the case of simple/scalar and struct types I see the CPU memory model reflected 1:1 with the content of such entities. In C it's the same, i.e. both C and Go are "close to metal" in this.

In terms of memory layout, Go and C are both close to the metal. I agree. In terms of new, the chosen semantics have nothing to do with the metal.

 

Not applicable to non-primitive types (e.g. slices, interfaces, ...), where the memory layout is intentionally hidden from the programmer.

And here we get to the exceptions.

Intentional hiding of data structure (of non simplistic types) is a (IMHO good) design choice and not an exception. Even compared to other languages it's not an exception. Consider e.g. very popular (though not for me) Javsacript, which AFAIK has no control on memory layout at all - everything is opaque, every implementation can choose how to implement Javascript all entities.

 

I agree. :) My point was that they aren't close to the metal, so it isn't generally true that the constructs in Go are close to the metal, and that these particular constructs are relevant here.

Well, I can say only that I see nothing odd about 'new'.

Not on it's own. IMO, it's a bit odd in the context of the language, because it treats pointers just as addresses rather than as distinct types by putting the emphasis on the element type. If we had distinct make functions for each kind of type that took element types as parameters, rather than the type itself, it wouldn't be odd.

`new` allocates zeroed memory and returns a reference to it. `make` (for slices) allocates zeroed memory and returns a reference to it. The only way to distinguish the two is that `new` works like C allocation (take info about the element type and return a pointer to the memory allocated for it), while `make` works like a C factory function (plus generics), which is necessary because the implementation of slices requires some extra initialization.

There is a huge difference between new and make which some tend to overlook. 'v = make(T)' initializes an already existing entity 'v', but 'v = new(T)'' guarantees to (if succeeds at all) to return a pointer to - not surprisingly - a *new*, unique zero value of T. Analogically for ':=',  but let me ignore some details for now.

v = make(T) initializes an already existing entity v based on an allocation it performs. v = new(T) initializes an already existing entity v based on an allocation it performs. Where T sits is different between the two cases (in the first, T is what is returned, in the second, T is what is allocated), but the difference has nothing to do with the types involved and everything to do with the chosen semantics of the functions. So yes, there is a distinction, but it's self generating, there's no external reason for it. You could also have newslice(T, 5) if you wanted to, but why would you?

The just mentioned difference between make and new make them so distinct for me, that I fail to understand, how those two builtins can be confused at all. They heave IMO mostly nothing in common, except for the detail of the magic understanding of a type name as a parameter.

Yes, there is a distinction, and once you know it, it's hard to get it confused. The question is, *why* is there a distinction?

At the end of the day, it seems kind of like painting the bikeshed a different colour. Whether you type `new(T)` or `make(*T)`, you get the same result (well, `make(*T)` would work better for named pointer types, but those aren't common).

It's true, and it was discussed several times before, that 'make(*T)' can be considered as an (syntax sugar) replacement for 'new(T)'. What's left is the design choice made by the Go language designers (I'm not one of them). After diving myself into Go, I think they've made a good choice. Because aliasing the semantics of new(T) and make(*T) would/could - I guess - cause even more confusion for those, who already have trouble to get a grip on new vs make.

If make(*T) replaced new(T), there'd be no new(T) for people to get confused about.

But because of the slight difference in how they operate (`make` returns a `T`, `new` returns a `*T`) newcomers get confused, especially if they haven't been exposed to pointers much before.

Reiterated: IMO - the difference is big, what they have in common is almost irrelevant.

The difference is artificial, while the similarities are intrinsic. That, to me, makes the similarities more important.

So it's more like having a shed where one of the doors opens in and the other opens out, and there's no apparent reason for it except that that's how the contractor originally walked through the shed. If you use the shed everyday, you come to accept that this is how the doors work. But someone new will have trouble figuring out when they're supposed to pull and when they're supposed to push. If you make them both open the same way, the newcomer only has to care about which way they want to go, rather than worrying about how the door opens.

 

I do understand the words in this paragraph, I can parse the sentences, but I fail to get the message. My fault, granted.

New is a door that opens inward (analog: the type argument is the thing allocated). Make is a door that opens outward (analog: the type argument is the thing returned). There's no reason for the difference, except for something subjective about how the contractor (analog: language developer) chose to install them on that particular day. It would probably be less confusing for people *using* the bikeshed (at least at first) if the doors opened thee same way (analog: if the functions did the same thing with the type parameter, which would make them identical). Worrying about which way the door opens (see before) distracts from the important decision of which way you want to go (analog: if you want to initialize a pointer, a slice, a map, or a channel).

[Paul Borman]

On Fri, Jul 8, 2011 at 11:57 AM, bflm <befeleme...@gmail.com> wrote:

There is a huge difference between new and make which some tend to overlook. 'v = make(T)' initializes an already existing entity 'v'

I believe this is wrong:

package main

import (

        "fmt"

)

func main() {

        v := make(chan int)

        fmt.Printf("%p\n", v)

        v = make(chan int)

        fmt.Printf("%p\n", v)

}

When I run this I see: 

0xf840014460

0xf8400149b0

So I really see make as making a new channel, not initializing an existing channel.  If the compiler was going to do what you said it would be: make(v, T) and then the name would not make sense.

    -Paul

[Jessta]

On Sat, Jul 9, 2011 at 5:34 AM, Steven Blenkinsop <stev...@gmail.com> wrote:
> If make(*T) replaced new(T), there'd be no new(T) for people to get confused
> about.

But the confusion would be,
Can I make(T)? What does that give me?
Can I make(*map[string]string)? What does that give me?
Why does make(map[string]string) return something initialized while
make(*T) gives me something that still requires initialization.
The same confusion is there, you just moved it a bit and the same
level of understanding is required to remove the confusion.

New users eventually have to understand the difference between the
functions of new() and make() even if we call them something else and
it's better that they get confused early on and learn than get too far
in with an incorrect understanding.

- jessta
--
=====================
http://jessta.id.au

[bflm]

On Friday, July 8, 2011 9:34:35 PM UTC+2, Steven Blenkinsop wrote:

I'm picking just one item to be short.

v = make(T) initializes an already existing entity v based on an allocation it performs. v = new(T) initializes an already existing entity v based on an allocation it performs. Where T sits is different between the two cases (in the first, T is what is returned, in the second, T is what is allocated), but the difference has nothing to do with the types involved and everything to do with the chosen semantics of the functions. So yes, there is a distinction, but it's self generating, there's no external reason for it. You could also have newslice(T, 5) if you wanted to, but why would you?

I still can't agree with any (important) similarity. Maybe this code illustrates what I tried to say (about the differences) in a better way. Would there be an easy way to count the live/reachable instances of type Map and Int, then I believe the program will print 1 for N1 and 10 for N2.

[bflm]

var m map[int]int // s is an zero value instance here

m = make(map[int]int) // m is initialized here, i.e. we assign an initialized value to the same m instance

and the same on one line:

m := make(map[int]int)

HTH

[Alexey Gokhberg]

> I believe this is wrong:

I believe this is implementation dependent. At present channels are
implemented as references to opaque objects, and each call to 'make'
allocates a new object and returns a reference to it.

In some more conventional language the equivalent functionality would
be implemented using the 'new' statement:

class Chan {

Chan() {
/* Initialize a channel object here */
}

}

...
Chan v = new Chan(); // allocate a reference, initialize an object
...
v = new Chan(); // one more time allocate a reference,
initialize an object

However, in principle another implementation of channels may be
chosen, that is, a channel may be represented not as a reference but
as a "value type" structure, and then repeated calls to 'make' would
just reinitialize particular fields in this structure.


On 8 Jul., 21:55, Paul Borman <bor...@google.com> wrote:

[Alexey Gokhberg]

P.S. Well, I just realised that in C# the value type structure would
be also initialized with 'new'.

This means that C# would likely use 'new' to emulate the 'make'
functionality of Go, no matter how channels are represented, as
references or as values.


On 8 Jul., 22:19, Alexey Gokhberg <express...@unicorn-enterprises.com>
wrote:

> >     -Paul- Zitierten Text ausblenden -
>
> - Zitierten Text anzeigen -

[⚛]

On Jul 8, 9:34 pm, Steven Blenkinsop <steven...@gmail.com> wrote:

> On Jul 8, 2011, at 2:57 PM, bflm <befelemepesev...@gmail.com> wrote:
> > Intentional hiding of data structure (of non simplistic types) is a (IMHO good) design choice and not an exception. Even compared to other languages it's not an exception. Consider e.g. very popular (though not for me) Javsacript, which AFAIK has no control on memory layout at all - everything is opaque, every implementation can choose how to implement Javascript all entities.
>
> I agree. :) My point was that they aren't close to the metal, so it isn't generally true that the constructs in Go are close to the metal, and that these particular constructs are relevant here.

I might have a slightly different interpretation.

Javascript and dynamic languages in general are approaching the
problem from a different perspective than languages such as Go or C.
In dynamic languages, the *hope* is that the most optimal memory
layout depends on what the program is doing at run-time. This is
contrary to Go or C, where the most optimal memory layout is *hoped*
to be the one that is explicitly specified in the source code.

A failure in dynamic languages so far has been that in order to
optimize the memory layout based on the actual run-time program
behavior, the programmer would need to somehow specify the abstract
mathematical operations concerning the objects used in the program.
For example, to unlock a certain type of optimizations, a compiler
needs to know whether the program wants to be able to compare the
objects for identity. Unfortunately, it is very hard to determine
whether the mathematical operation 'identity' is used in the program
or not. Impossible to determine if code can be loaded during run-time.
Also impossible if run-time reflection is allowed. Also impossible if
the user is allowed to debug the running program.

There is one very strange phenomenon here: a lot of people are using
dynamic languages in contexts where a lot of static information is
available.That seems to go against the purpose of dynamic languages,
because the purpose is that they are useful in contexts where you have
a lot of run-time (dynamic) information and little static information.

My point is that the distinction between Go and Javascript is not in
"being close to metal". The distinction, in my opinion, is in
predictability of future events. If you are unable to predict the
future, use a scripting (=dynamic) language. That is, if you are
unable to predict (you don't have enough static information), close-to-
metal is non-sense.

What I don't understand: why is there such a big gap between scripting
languages and C-like languages.

[Steven Blenkinsop]

On Jul 8, 2011, at 4:00 PM, Jessta <jes...@jessta.id.au> wrote:

On Sat, Jul 9, 2011 at 5:34 AM, Steven Blenkinsop <stev...@gmail.com> wrote:

If make(*T) replaced new(T), there'd be no new(T) for people to get confused

about.

But the confusion would be,
Can I make(T)?

The answer: "only if it is a pointer, slice, map, or chan". Same as now, just adding pointer.

What does that give me?

"It gives you an initialized T. Any elements of T, however, aren't initialized". Same as now, just adding pointer. And you don't have the `new` semantics, where it gives you a *T instead, to create confusion. 

Why does make(map[string]string) return something initialized while
make(*T) gives me something that still requires initialization.

"make(*T) makes an initialized pointer. Whether you still need to initialize its element depends on the element type".  The same answer as you already have for make([]T, 5), you just don't have a different answer depending on whether its new or make).

Why does make(map[string]string) return something initialized while

make([]map[string]string)/make(*[]map[string]string)/make([][]map[int]*[]map[string]string)...

It's all the same question, a fact that gets disguised by the new/make distinction.

The same confusion is there, you just moved it a bit and the same
level of understanding is required to remove the confusion.

The level of understanding requires acknowledging pointers as distinct types, rather than an imperfect "that's just the way it is" kind of understanding people often develop to handle new, without truly appreciating the pointer aspect.

New users eventually have to understand the difference between the
functions of new() and make() even if we call them something else and
it's better that they get confused early on and learn than get too far
in with an incorrect understanding.

If you call them something else, there is no difference between them. The same answers and understanding applies to each new type you use with make. People seem to get very far using make/new with the wrong understanding, because it hides the real problem — understanding pointers — from you.

[Steven Blenkinsop]

On Jul 8, 2011, at 4:13 PM, bflm <befeleme...@gmail.com> wrote:

I still can't agree with any (important) similarity. Maybe this code illustrates what I tried to say (about the differences) in a better way. Would there be an easy way to count the live/reachable instances of type Map and Int, then I believe the program will print 1 for N1 and 10 for N2.

Equally valid code: http://goo.gl/q6IsR

Or: http://goo.gl/7iLtk

The difference between the two uses isn't make and new, it's the choice to assign to a pointer vs its element.

I don't see how you can acknowledge that new(T) could be replaced by make(*T), and yet claim that there is no similarity between what make and new do.

[bflm]

On Friday, July 8, 2011 11:12:16 PM UTC+2, Steven Blenkinsop wrote:

Equally valid code: http://goo.gl/q6IsR

Or: http://goo.gl/7iLtk

IMHO you're - I don't know why - confusing an entity and what an entity internally may point to. The later was nothing I was talking about. I'm sorry I failed to explain the problem in a better way.

 

The difference between the two uses isn't make and new, it's the choice to assign to a pointer vs its element.

I must disagree, but don't know how to not just reiterate: make returns an initialized *value*, new returns a *pointer* to a *new* zero value. Completely different semantics. For me not even similar. Ponter vs value, zeroed vs initialized (and behind the scene malloc vs no malloc - on the entity level [internal details are out of specs semantics]).

 

I don't see how you can acknowledge that new(T) could be replaced by make(*T), and yet claim that there is no similarity between what make and new do.

I said "as syntactic sugar"* and I said it's IMO a good choice to not sweeten Go in this way ;-) 

(*) e.g. a channel close can in theory also be provided via something like 'c <- _', if a language designer likes it that way. But why to mix send with close if they are so distinct?

[Steven Blenkinsop]

On Jul 8, 2011, at 5:34 PM, bflm <befeleme...@gmail.com> wrote:

On Friday, July 8, 2011 11:12:16 PM UTC+2, Steven Blenkinsop wrote:

Equally valid code: http://goo.gl/q6IsR

Or: http://goo.gl/7iLtk

IMHO you're - I don't know why - confusing an entity and what an entity internally may point to. The later was nothing I was talking about. I'm sorry I failed to explain the problem in a better way.

Unfortunately not. I am quite clear on the difference between a pointer and its element. However, I also acknowledge that a pointer is a value in its own right, something you seem to be having trouble bringing yourself to acknowledge.

[bflm]

On Friday, July 8, 2011 11:50:26 PM UTC+2, Steven Blenkinsop wrote:

IMHO you're - I don't know why - confusing an entity and what an entity internally may point to. The later was nothing I was talking about. I'm sorry I failed to explain the problem in a better way.

Unfortunately not. I am quite clear on the difference between a pointer and its element. However, I also acknowledge that a pointer is a value in its own right, something you seem to be having trouble bringing yourself to acknowledge.

Although I don't know what an element of a pointer is, I disagree again. It seems to me that I'm just repeatedly saying the very same as the language authors do about this topic, while that's not the case on the other side.

Anyway, the decision about new vs make was already done long ago by the Go authors and my opinion in this is admittedly completely irrelevant (even though I agree with them), no problem ;-)

[Steven Blenkinsop]

On Fri, Jul 8, 2011 at 6:25 PM, bflm <befeleme...@gmail.com> wrote:

On Friday, July 8, 2011 11:50:26 PM UTC+2, Steven Blenkinsop wrote:

IMHO you're - I don't know why - confusing an entity and what an entity internally may point to. The later was nothing I was talking about. I'm sorry I failed to explain the problem in a better way.

Unfortunately not. I am quite clear on the difference between a pointer and its element. However, I also acknowledge that a pointer is a value in its own right, something you seem to be having trouble bringing yourself to acknowledge.

Although I don't know what an element of a pointer is, I disagree again. It seems to me that I'm just repeatedly saying the very same as the language authors do about this topic, while that's not the case on the other side.

The element of a pointer is the thing returned by, given a var p *T,

reflect.ValueOf(p).Elem().Interface().(T)

Or, more simply:

*p

Another way to say it would be "pointee", or "the value that p points to".

You aren't quite saying the same thing as what the language authors are saying. I'm sure the language authors realize that the choice of semantics is exactly that: a choice. You seem to be saying that the alternative semantics are illogical. You then go on to attempt to demonstrate the current semantics in a way that just shows you don't actually understand them. I'm sure the language designers understand their own language, most of the time (there have been a few instances where even they have been surprised).

[bflm]

On Saturday, July 9, 2011 12:36:32 AM UTC+2, Steven Blenkinsop wrote:

The element of a pointer is the thing returned by, given a var p *T,

reflect.ValueOf(p).Elem().Interface().(T)

Or, more simply:

*p

Another way to say it would be "pointee", or "the value that p points to".

I know what a pointee is, but I can't recall ever seeing it being called 'a pointer's element' before.

 

You aren't quite saying the same thing as what the language authors are saying. I'm sure the language authors realize that the choice of semantics is exactly that: a choice. You seem to be saying that the alternative semantics are illogical.

Well, I don't think I've said that, but to make it clear - I do think it's illogical.

 

You then go on to attempt to demonstrate the current semantics in a way that just shows you don't actually understand them.

 

 We share that opinion ;-)

 

I'm sure the language designers understand their own language, most of the time (there have been a few instances where even they have been surprised).

Yes, some of the posts in this thread could even make them sad, I guess.

[Kyle Lemons]

I know what a pointee is, but I can't recall ever seeing it being called 'a pointer's element' before.

At risk of making this really long thread even longer, in C any pointer is functionally identical to an array, thus p[0] is the same as *p, and so whatever p points to is element 0 of p.

~K 

[Steven Blenkinsop]

Yes. A more language-relevant source is in the reflect package, where the word element is used in this way. IIRC, there used to also be similar usages in the spec, but they became unnecessary as things were streamlined. Element is a nice general term for when you want to talk about similar things with similar language, as is the case in reflect.

It's interesting to point out that C's pointer and dynamic array aren't only similar, but the same thing. Go makes a distinction, which is safer, but that doesn't mean they're strangers.

Here's an illustrative example of what I'm talking about:

x := make([]map[int]*[]chan int, 1)

x[0] = make(map[int]*[]chan int)

x[0][0] = new([]chan int)

*x[0][0] = make([]chan int, 1)

(*x[0][0])[0] = make(chan int, 1)

(*x[0][0])[0] <- 5

fmt.Println(<-(*x[0][0])[0])

Obviously, there is way too much composition going on here, but I think it helps the illustration. I'm not doing anything all that different on line 3 than on lines 1, 2, 4 and 5. The composite reference type I'm initializing just happens to be a pointer, so I have to use a different function with different semantics to initialize it. I didn't have trouble writing this code, but I sympathize with anybody trying to figure out why line 3 needs to break the pattern.

[crazy 2be]

This is the most concise and relevant post in this entire thread. I completely agree with the distinction here, and found it only marginally confusing at first. Now, I appreciate the difference because I know that new(T) will not do anything unexpected behind the scenes. Being able to guarantee what code will do is important not only in building a reliable system, but also in basic code comprehension. In C++, creating an object could create hundreds more. You don't even have to call a function, but execution control jumps to another function. This is inconsistent.

That being said, the issue seems pretty clear to me: Newcomers don't understand the difference. In order to solve this, we should not attempt to hide said difference, but to educate on said difference. Perhaps newcomers can't find this in the documentation. Perhaps they do not wish to read documentation, and instead want to just start writing code. Perhaps they find the existing documentation boring and uninteresting, and so instead immediately jump to attempting to write code.

In any case, this is not an issue in the language spec nor an inconsistency in the implementation. It is an inconsistency with other programming languages, and one that has lead to significant confusion because of the unfortunate choice of naming. Programmers coming from other languages expect new in go to be magic, and it's not. Perhaps alloc() would have been a better choice, since it better describes what it does.

Really, however, this entire discussion is irrelevant. This is such a fundamental language construct that it is unlikely to change- it would frustrate existing developers, and force pretty much all existing code to be gofix'd. Unless there is a really, really, really compelling reason to change this, it's not going to happen.

To reiterate- our decision has to be not to hide the differences from newcomers, but rather make sure there is good and accessible documentation in place to educate them on the differences and why they exist. Changing the name or the semantics of the function is pretty much impossible at this point, and I have not yet seen a good justification.

-crazy2be

[Steven Blenkinsop]

On Jul 8, 9:39 pm, crazy 2be <crazy...@gmail.com> wrote:
> This is the most concise and relevant post in this entire thread.

There's no way to know what post you're talking about... though I
suspect it's by ⚛ based on the CC.

[crazy 2be]

Google groups indentation levels make it clear for me, but for those who are using more traditional email clients, it may not be clear. My apologies. I was responding to r (is that Russ?). The CC for ⚛ was generated automatically by google groups, not sure why. I just left it as-is.

-crazy2be

[Martin Capitanio]

On Saturday, July 9, 2011 4:02:44 AM UTC+2, crazy 2be wrote:

Google groups indentation levels make it clear for me, but for those who are using more traditional email clients, it may not be clear. My apologies. I was responding to r (is that Russ?). The CC for ⚛ was generated automatically by google groups, not sure why. I just left it as-is.

No hat's Rob Pike. And yes, he is right.

Martin

[bflm]

On Saturday, July 9, 2011 12:56:10 AM UTC+2, Kyle Lemons wrote:

At risk of making this really long thread even longer, in C any pointer is functionally identical to an array, thus p[0] is the same as *p, and so whatever p points to is element 0 of p.

 

I'm not a C programmer but I know this tricky (but powerful) property of C. Even in C I would personally prefer to call 'p[0] /* *(p+0) */' the 'zeroth element of the array p points to' and not an 'element of p'. The reason is the same as why Go compiler will reject this program when uncommenting the last println.

Additionally the discussion went about Go, not C, and not specifically about arrays, but more generally about pointers/values/new/make, so the 'pointer's element' was somehow context less or at least a bit surprising ;-) 

[⚛]

Well, he (Rob Pike) is right. No doubts about that. Many of us in this
discussion are right.

The issue here is not "being right", but how to transfer the knowledge
into the minds of people who don't know the difference between "new"
and "make".

My favorite approach is to tell people about the most general
operational semantics of "new" and "make". Basically, this means to
tell them how "new" and "make" are implemented. The main point is that
the implementations are different. A consequence is that their results
have different types (T vs *T).

[bflm]

On Saturday, July 9, 2011 2:27:55 AM UTC+2, Steven Blenkinsop wrote:

Yes. A more language-relevant source is in the reflect package, where the word element is used in this way.

The specs are more/solely relevant compared to any identifiers used in any package, IMO.

IIRC, there used to also be similar usages in the spec, but they became unnecessary as things were streamlined.

I can't deny it (too much time to inspect all versions), but I doubt it.

 

Element is a nice general term for when you want to talk about similar things with similar language, as is the case in reflect.

How has reflect suddenly joined the topic? And if it's specific to some C-only semantics than it hardly can be a "general term". Moreover, discussion was about Go maps/channels/slices, not about C arrays.

 

It's interesting to point out that C's pointer and dynamic array aren't only similar, but the same thing. Go makes a distinction, which is safer, but that doesn't mean they're strangers.

Here's an illustrative example of what I'm talking about:

x := make([]map[int]*[]chan int, 1)

x[0] = make(map[int]*[]chan int)

x[0][0] = new([]chan int)

*x[0][0] = make([]chan int, 1)

(*x[0][0])[0] = make(chan int, 1)

(*x[0][0])[0] <- 5

fmt.Println(<-(*x[0][0])[0])

Obviously, there is way too much composition going on here, but I think it helps the illustration. I'm not doing anything all that different on line 3 than on lines 1, 2, 4 and 5. The composite reference type I'm initializing just happens to be a pointer, so I have to use a different function with different semantics to initialize it. I didn't have trouble writing this code, but I sympathize with anybody trying to figure out why line 3 needs to break the pattern.

 

Except for being a bit hard to read code I have no idea what the example tries to illustrate. It is valid Go code, it does what it should do. What/where should be a/the surprise? Creating a *pointer* to a new *zero* *value* of a slice (*not* "initializing a composite reference type") at line 3 using new is 'breaking the pattern'? Why? That's just exactly what new is for.

Also note that slice is not strictly a reference type, altougth it is called like that in the specs:

http://golang.org/doc/go_spec.html#Making_slices_maps_and_channels

Actually slice is a value type with mixed reference and value semantics. Example. That's why I usually don't use the term "reference type" in Go's context.

[Martin Capitanio]

On Saturday, July 9, 2011 11:00:15 AM UTC+2, ⚛ wrote:

... 

My favorite approach is to tell people about the most general
operational semantics of "new" and "make". Basically, this means to
tell them how "new" and "make" are implemented. The main point is that
the implementations are different. A consequence is that their results
have different types (T vs *T).

Absolutely. And if you additionally tell them about a particular map, slice and the

interface implementation, they are ready to go.

Martin

[⚛]

This is sarcasm, or something like that?

[Martin Capitanio]

On Saturday, July 9, 2011 3:25:42 PM UTC+2, ⚛ wrote:

Ah, no sorry :-) 

s/to go/for Go/

 

i meant to say, if you know how it's implemented, you can better

understand how it works.

[⚛]

Ok. I wasn't sure what you meant.

I wasn't sure because a good counter-argument to the "simply tell
people how it's implemented" approach is that people might then start
to assume that it is the only possible implementation, and base their
programming decisions on this assumption. It might cause problems if
the operational semantics they have been told about is correct, but
totally sub-optimal.

A related example: Sorting has multiple implementations: bubble sort,
quick sort, merge sort, bitonic sort, etc. The most simple
implementation is bubble sort. Unfortunately, bubble sort has the
worst performance. If a person only knows about bubble sort, it is
effectively preventing the possibility to use sorting to solve medium-
to-large problems.

[Honza]

CS 101? ;-)

Dne 9.7.2011 16:30 "⚛" <0xe2.0x...@gmail.com> napsal/a:

Ok. I wasn't sure what you meant.

I wasn't sure because a good counter-argument to the "simply tell
people how it's implemented" approach is that people might then start
to assume that it is the only possible implementation, and base their
programming decisions on this assumption. It might cause problems if
the operational semantics they have been told about is correct, but
totally sub-optimal.

A related example: Sorting has multiple implementations: bubble sort,
quick sort, merge sort, bitonic sort, etc. The most simple
implementation is bubble sort. Unfortunately, bubble sort has the
worst performance. If a person only knows about bubble sort, it is
effectively preventing the possibility to use sorting to solve medium-
to-large problems.

On Jul 9, 3:59 pm, Martin Capitanio <m...@capitanio.org> wrote:

> On Saturday, July 9, 2011 3:25:42...