Re: [go-nuts] pointer vs object in return statement

[John Cowan]

Alex Kaushansky scripsit:

> OK, let it be allocated on heap, and somebody says:
> return &array[15]
> How will garbage collector work? Will it keep the whole array in memory?

Inevitably so. I do not know of any garbage collector that can collect part
of an array and leave the rest intact. But it would be more Go-style to
return a slice anyway.

> The pointers certainly introduce confusion (as it is evident from many other
> discussions here - e.g. make vs new),
> but what (apart from existential love to pointers) justifies it, in
> measurable technical terms?

If you want efficiency, you have to have both boxed and unboxed varieties
of things. C does boxing through pointers, so does Go.

--
Is not a patron, my Lord [Chesterfield], John Cowan
one who looks with unconcern on a man http://www.ccil.org/~cowan
struggling for life in the water, and when co...@ccil.org
he has reached ground encumbers him with help?
--Samuel Johnson

[Seer]

This view (lack of partial release of an aggregate as failure/leak)
seems inconsistent with the natural "sovereignty" of an array (or set/
bag/hash table/...) under any definition in any programming language).
If you allocate an array it is fully alive so long as any of it is
reachable. Do you really mean what you have written?

On Nov 12, 11:51 am, Alex Kaushansky <kaushan...@gmail.com> wrote:
> If garbage collector fails to clean up the entire array just because someone
> keeps pointer to one
> element - it constitutes memory leak. It will be very difficult to detect
> it, and there's nothing in language
> description that suggests even the possibility of leaks like this. I think
> it's a serious issue.

>
>
>
> On Thu, Nov 12, 2009 at 2:03 PM, John Cowan <co...@ccil.org> wrote:
> > Alex Kaushansky scripsit:
>
> > > OK, let it be allocated on heap, and somebody says:
> > >   return &array[15]
> > > How will garbage collector work? Will it keep the whole array in memory?
>
> > Inevitably so.  I do not know of any garbage collector that can collect
> > part
> > of an array and leave the rest intact.  But it would be more Go-style to
> > return a slice anyway.
>
> > > The pointers certainly introduce confusion (as it is evident from many
> > other
> > > discussions here - e.g. make vs new),
> > > but what (apart from existential love to pointers) justifies it, in
> > > measurable technical terms?
>
> > If you want efficiency, you have to have both boxed and unboxed varieties
> > of things.  C does boxing through pointers, so does Go.
>
> > --
> > Is not a patron, my Lord [Chesterfield],        John Cowan

> > one who looks with unconcern on a man          http://www.ccil.org/~cowan<http://www.ccil.org/%7Ecowan>

[atomly]

On Thu, Nov 12, 2009 at 2:51 PM, Alex Kaushansky <kaush...@gmail.com> wrote:
>
> If garbage collector fails to clean up the entire array just because someone keeps pointer to one
> element - it constitutes memory leak. It will be very difficult to detect it, and there's nothing in language
> description that suggests even the possibility of leaks like this. I think it's a serious issue.

So, say you're programming in a language that's not garbage
collected... C, for example.  Would you really write code like:

char foo[100] = malloc(100 * sizeof(char));
char *p = foo + 20;
free(foo);
bar(p);

That's essentially what you're asking the GC to do, it seems.

--
:: atomly ::

[ ato...@atomly.com : www.atomly.com  : http://blog.atomly.com/ ...
[ atomiq records : new york city : +1.917.442.9450 ...
[ e-mail atomly-new...@atomly.com for atomly info and updates ...

[Ian Lance Taylor]

inspector_jouve <kaush...@gmail.com> writes:

> In error.go I found the function:
> func NewSyscallError(syscall string, errno int) Error {
> if errno == 0 {
> return nil
> }
> return &SyscallError{syscall, Errno(errno)};
> }
> The function was supposed to return Error (which is an interface with
> the only method: String()), but instead returns the pointer(!) to
> object implementing this interface, not the object itself.
> It seems that we could as well write return SyscallError{syscall,
> Errno(errno)};
> Is it correct?
> but in this case, how will caller know whether we return object, or
> pointer to object?

The caller will get an interface value, and doesn't need to know.
Yes, this function could return an object. It doesn't make too much
difference in this case, because nobody expects to change the values.


> This also begs the question: why doesn't language just create all
> objects in a heap, like java does?

Because that would be less efficient. Many objects don't need to be
on the heap, and can simply live efficiently on the stack.


> How the use of pointers is justified in this language? There's no
> pointer arithmetic anyway. So what is the rationale?

There are many uses of pointers other than arithmetic. For example,
sometimes it's nice to pass a pointer to a function to tell it which
of some set of variables to modify. Or to pass a pointer to a
function so that it can modify a local variable.

Ian

[Ian Lance Taylor]

Alex Kaushansky <kaush...@gmail.com> writes:

> Your example of var array [256]struct{...} exactly illustrates the point I
> want to make.
> Imagine someone creates a pointer to one of this structures, e.g.
> return &array[0]
> In another discussion, people explained to me that in this case, compiler is
> smart enough to figure out that the whole array(!)
> should not be allocated in a stack. Alternatively (as it was suggested)
> compiler can copy this struct to heap before return.
> Second option is bad (it may have pointers to itself, they need to be
> adjusted - too complicated), first one is too involved: address can be taken
> for attribute of attribute etc...

No pointers need to be adjusted when copying the array to the heap.
Remember that arrays are passed by value; if an array contains
pointers to itself (what would be the type of the elements of the
array?) then it will already fail when calling the function, long
before the value is copied to the heap.


> But how much is really gained by not allocating these structs in the heap,
> and creating array of references, like in java?
> Is there any statistical measure of this? My bet is that the gains are
> negligible, but complexity introduced here is not.
> And most importantly, when I write var array [256]struct{...} - I have no
> idea of where this array in fact allocated, because it depends
> on whether somebody takes address of something inside it. It can drive
> programmer crazy. For what end?

I don't see understand it is going to drive the programmer crazy at
all. In fact, the compiler can simply take care of everything. Why
do you care where the array is allocated?

Ian

[Ian Lance Taylor]

Alex Kaushansky <kaush...@gmail.com> writes:

> If garbage collector fails to clean up the entire array just because someone
> keeps pointer to one
> element - it constitutes memory leak. It will be very difficult to detect
> it, and there's nothing in language
> description that suggests even the possibility of leaks like this. I think
> it's a serious issue.

I think it's too early to know whether this sort of issue is serious
or not. I also think that all garbage collected languages have this
issue in one form or another.

Ian

[John Cowan]

Alex Kaushansky scripsit:

> If garbage collector fails to clean up the entire array just because
> someone keeps pointer to one element - it constitutes memory leak. It
> will be very difficult to detect it, and there's nothing in language
> description that suggests even the possibility of leaks like this. I
> think it's a serious issue.

No GCed language, as far as anyone knows, has ever had such a facility
for partial reclamation. It's no use crying because the moon is not in
your soup.

--
We pledge allegiance to the penguin John Cowan
and to the intellectual property regime co...@ccil.org
for which he stands, one world under http://www.ccil.org/~cowan
Linux, with free music and open source
software for all. --Julian Dibbell on Brazil, edited

[John Cowan]

inspector_jouve scripsit:

> Consider a function find(persons []person, name string) *person which
> returns a person by name.
> Someone allocated array of persons, calls this function, it returns a
> pointer to one element.

Much better just to return the person object itself (which will copy it)
or, if you want to update it, to return an index into the []person array.

--
Clear? Huh! Why a four-year-old child John Cowan
could understand this report. Run out co...@ccil.org
and find me a four-year-old child. I http://www.ccil.org/~cowan
can't make head or tail out of it.
--Rufus T. Firefly on government reports

[Matt Godbolt]

On Thu, Nov 12, 2009 at 8:52 PM, Alex Kaushansky <kaush...@gmail.com> wrote:

Java doesn't have this problem at all - because in java, array of objects is internally represented as array of pointers
to individual elements to begin with. Elements of array are not supposed to occupy adjacent locations in memory, and each object
lives by itself. There's nothing remotely close to this kind of memory leak in java

 

Java has this issue for substrings of strings: if you write (excuse my pidgin java, it's been a while)...

String getFirstLine(String filename) {

  String wholeFile = readEntireFile(filename);

  return wholeFile.Substring(0, wholeFile.findFirst("\n"));

}

The substring is implemented as a slice of the existing string, and so references to the substring keep the entire file's contents "alive" in memory.

Additionally, from a performance point of view, having an array of sequential value types can be a huge win compared to an array of pointers to objects, saving  both a level of indirection and making much better use of processor caches. As a systems language, Go needs to be able to support this kind of layout, something which can't be controlled as tightly in Java.

[Matt Godbolt]

On Thu, Nov 12, 2009 at 9:32 PM, Alex Kaushansky <kaush...@gmail.com> wrote:

What you describe is a known java bug. Everybody agrees that it's bug, not intentional feature.

I don't want to steer this off topic; but I guess I wouldn't think of this as a bug myself, more of a design choice made by the Java guys to optimise for the case where substrings share the memory of their parent string, and for the GC to not do partial object reclamation. These seems reasonable to me, but as you can tell from my code, I'm clearly no Java expert! :)

[Russ Cox]

On Thu, Nov 12, 2009 at 13:32, Alex Kaushansky <kaush...@gmail.com> wrote:
> What you describe is a known java bug. Everybody agrees that it's bug, not
> intentional feature.

Whether it is an intentional feature or a bug is not a
matter of opinion: either the Java designers intended
that behavior or not, and I can't speak to that.

I can speak to the situation in Go. We originally
started with slicing a string making a copy, to avoid
exactly the Java behavior that people think is a bug.
It turned out that all the code ended up passing around
(string, int, int) triples to make up for the inefficient
slicing. So we changed the slicing behavior to do
for us what we were already doing by hand.

If you want to make a true copy of a string (or slice),
it is easy to do explicitly.

I think the most important flaw here is that we need
good tools to analyze and debug memory usage, so
that it is easy to see inside the running program.

Russ

[Rob 'Commander' Pike]

On Nov 12, 2009, at 9:43 AM, inspector_jouve wrote:

> In error.go I found the function:
> func NewSyscallError(syscall string, errno int) Error {
> if errno == 0 {
> return nil
> }
> return &SyscallError{syscall, Errno(errno)};
> }
> The function was supposed to return Error (which is an interface with
> the only method: String()), but instead returns the pointer(!) to
> object implementing this interface, not the object itself.
> It seems that we could as well write return SyscallError{syscall,
> Errno(errno)};
> Is it correct?
> but in this case, how will caller know whether we return object, or
> pointer to object?
>

> This also begs the question: why doesn't language just create all
> objects in a heap, like java does?

> How the use of pointers is justified in this language? There's no
> pointer arithmetic anyway. So what is the rationale?

Whatever is returned by NewSyscallError is of type Error, an
interface. Therefore the return statement must supply a value that
implements the Error interface, which is { String() string }. Now
the question is: does SyscallError's String() method take a value
receiver or a pointer receiver? You'll see it takes a pointer
receiver, so it makes sense to return a pointer from NewSyscallError.
If the String() method had a value receiver, NewSyscallError would
return a value.

Effective Go has a discussion about pointers and values as receivers.
It's not quite as simple as I make it sound here, but pointer-receiver
methods only work for pointer values.

-rob

[Ian Lance Taylor]

Alex Kaushansky <kaush...@gmail.com> writes:

>>There are many uses of pointers other than arithmetic. For example,
>>sometimes it's nice to pass a pointer to a function to tell it which
>>of some set of variables to modify. Or to pass a pointer to a
>>function so that it can modify a local variable.
>

> Don't you think this practice goes against the spirit of the language?
> Looking at the libraries, I can see you prefer returning multiple values,
> which makes
> things more explicit, and is certainly a "nicer" solution? I thought you
> added this multiple return values
> specifically to avoid functions with side effects, right?

In general, yes, but on the other hand if you have a large struct, and
a function which modifies a single field, a pointer is appropriate.

Ian

[Marcin 'Qrczak' Kowalczyk]

2009/11/14 Alex Kaushansky <kaush...@gmail.com>:

> However, if you decide to implement your own kind of map, say SortedMap, and
> use it instead of default map,
> situation will be completely different - you will have to declare it as
> pointer. Isn't it strange?

Yes, I don't like this either.

You can wrap a pointer to your map in your own "handle" type and use
it exclusively in public interfaces. Then it will be like map in this
respect. But unlike File which uses explicit pointers. In any case
it's inconsistent in practice, although the formal language rules are
consistent in this respect, they are only inconsistently applied: with
or without a public handle type.

--
Marcin Kowalczyk

[Russ Cox]

On Sat, Nov 14, 2009 at 05:41, Alex Kaushansky <kaush...@gmail.com> wrote:
> There's another interesting phenomenon.
> As I gather from the above discussion , the only justification for NOT
> always allocating struct values in heap, java-style, is that
> sometimes you want to save nanoseconds and allocate it in stack. But it
> occurred to me that you can't do it anyway in most of typical cases.
> ... We drive java out through the door - it comes back through the window.

While it's true that many local variables get allocated on the
heap instead of the stack because they are addressed, this
is not the same as allocating every struct as a separate
heap object.

In Go, when I define

type Point struct {
X, Y int;
}

type Rect struct {
Min, Max Point;
}

type RectPtr struct {
Min, Max *Point;
}

Rect and RectPtr have different memory layouts, regardless
of where a particular instance gets allocated: Rect is four ints
in a row, and RectPtr is two pointers. Java only lets me have
the latter. In the code I've optimized in the past few years,
memory locality has been very important to performance, so
it's nice that Go gives me explicit control over what is and is
not a pointer.

Russ

[Ian Lance Taylor]

Alex Kaushansky <kaush...@gmail.com> writes:

>>In general, yes, but on the other hand if you have a large struct, and
>>a function which modifies a single field, a pointer is appropriate.
>

> That's why I believe java made a right choice: all objects have reference
> semantics.
> The only "disadvantage" of it is that you can't pass object by value. But I
> bet you will
> almost never want to do it anyway.

Well, for example, I think it is quite normal to pass ints and floats
by value. That is, I think it would be odd if this printed 1:

func p(i int) { i++; }
func main() { v := 0; p(v); fmt.Print(v); }


> In go language, some objects have reference semantics (e,g. maps), others do
> not, which is confusing.

It is inconsistent, but my experience is that it soon makes sense.
Originally in Go everything was a value type, and you could not
declare a value of type map, only type of type *map. After a while
the requirement to always use a pointer seemed sort of pointless, so
we made it a reference type instead. Same for channel.


> For example, when you have to pass map as parameter (obviously, by
> reference), do you declare this parameter as pointer or not?
> In other words, which is correct:
> func foo(mymap map[string]int)
> or
> func foo(mymap *map[string]int) ?
> My guess is that the second passes the pointer to location where reference
> to map is stored, right? (clearly, that's not what we want)

Correct.

> However, if you decide to implement your own kind of map, say SortedMap, and
> use it instead of default map,
> situation will be completely different - you will have to declare it as
> pointer. Isn't it strange?

It is inconsistent, yes. However, it doesn't show up in normal use,
because in practice you would write SortedMap as an interface, and you
would provide a SortedMap.New() function which returned a pointer
which met the interface. So code which uses SortedMap would look like
code which uses map: it wouldn't use a pointer.

Ian

[Marcin 'Qrczak' Kowalczyk]

2009/11/14 Ian Lance Taylor <ia...@google.com>:

>> That's why I believe java made a right choice: all objects have reference
>> semantics.
>> The only "disadvantage" of it is that you can't pass object by value. But I
>> bet you will
>> almost never want to do it anyway.
>
> Well, for example, I think it is quite normal to pass ints and floats
> by value.  That is, I think it would be odd if this printed 1:
>
> func p(i int) { i++; }
> func main() { v := 0; p(v); fmt.Print(v); }

This printing 0 is compatible with the reference semantics as long as
i++ creates a new reference rather than modifying an object in place
(at least formally).

> After a while
> the requirement to always use a pointer seemed sort of pointless, so
> we made it a reference type instead.  Same for channel.

What about File?

--
Marcin Kowalczyk

[Ian Lance Taylor]

Alex Kaushansky <kaush...@gmail.com> writes:

> As I gather from the above discussion , the only justification for NOT
> always allocating struct values in heap, java-style, is that
> sometimes you want to save nanoseconds and allocate it in stack.

I think those nanoseconds might tend to add up once you factor in the
garbage collection costs.

> But it
> occurred to me that you can't do it anyway in most of typical cases.

> The problem is that whenever you attach methods to struct, you do it via
> pointer:
> func (p *person) getAge() int
>
> This style (attaching method via pointer) is the only reasonable choice -
> you certainly don't want to pass person by value for this.
> But as soon as you take an address of person, compiler figures out that
> object cannot be allocated in the stack! (That is what someone explained to
> me).

That is true today, although I do think that this is an area where the
compilers could certainly improve.


> Because most of struct types have attached methods (via pointers), none of
> them can be allocated in the stack. We drive java out through the door - it

> comes back through the window.

Fair enough, but I don't see why one need conclude that all struct
values should be allocated on the heap. Allocating them on the stack
when possible does work, and does not affect the language. The
programmer normally doesn't have to care where a struct is allocated.
The compiler is free to allocate it in the most efficient place.


> If you go to the root of the problem, it's this: garbage collection doesn't
> live very well with explicit pointers. What do you think?

This conclusion doesn't seem to follow. I don't see any difficulty in
garbage collection and explicit pointers living together.

What you may be driving at is that we might as well turn structs into
a reference type, but I don't think that follows. A small struct like
"type Point struct { x, y int}" often does not needs its address
taken, and it can be efficiently passed by value. Why not simply pass
it by value when the fields don't need to change? That is also an
example where "func (p Point) ..." rather than "func (p *Point) ..."
makes perfect sense.

Ian

[Bob Cunningham]

On 11/14/2009 11:18 AM, Ian Lance Taylor wrote:
> Alex Kaushansky<kaush...@gmail.com> writes:
>
>> As I gather from the above discussion , the only justification for NOT
>> always allocating struct values in heap, java-style, is that
>> sometimes you want to save nanoseconds and allocate it in stack.
>
> I think those nanoseconds might tend to add up once you factor in the
> garbage collection costs.

If speed is your god, then even stacks can be slow, since they still map all the way back to system memory. Anything not in registers runs the risk of having horrendously slow access in the event of a cache miss. Relatively speaking, of course.

For specific routines on embedded / real-time systems needing hair-on-fire speed, I'd lock the cache for the stack and prevent writebacks until the need for speed has passed. This has resulted in 100x speed-ups with no other changes to the compiled code. The writeback prevention frees bus bandwidth for use by other threads/tasks/cores.

The general theme is to ensure "locality of access". In such cases, pass-by-value can win by a huge margin, since the item is near the top of the stack (or in registers), so no dereferencing is needed, and there is no remote data to also try to keep cached. Pointer access can easily cause cache thrashing. In many cases, when multiple parallel access is not required, it is beneficial to implement pass-by-reference as a pair of pass-by-value operations, copy-in and copy-back, rather than via direct access to the referenced data.

The downside is stack size. When that becomes an issue, the program needs to be restructured to manage stack use. Since most processors are highly optimized for fast stack access (well, access to at least the top stack frame), other mechanisms can be slower, even if they are also locked in cache.

Furthermore, if you want to avoid GC in Go, the current best way is to ensure your data is either on the stack or is main-global (I'm not absolutely sure if Go allocates system globals on the heap or not). The heap can be bad when speed is everything.

snip

>> Because most of struct types have attached methods (via pointers), none of
>> them can be allocated in the stack. We drive java out through the door - it
>> comes back through the window.
>
> Fair enough, but I don't see why one need conclude that all struct
> values should be allocated on the heap. Allocating them on the stack
> when possible does work, and does not affect the language. The
> programmer normally doesn't have to care where a struct is allocated.
> The compiler is free to allocate it in the most efficient place.

Except when the programmer *does* care, which will happen when profiling shows compiler decisions are sub-optimal. This is precisely the reason why I hope Go will eventually provide a rich set of (possibly complex) memory and execution control features.

Once such controls exist, even the language toolchain can take advantage of them: Self-directed program optimization via automated analysis of profiler runs, AKA "autotuning", would also be a HUGE help. Recent research suggests this is by far the best known way to automatically maximize performance on multicore systems (CACM Vol 52 Number 10 Page 56).

>> If you go to the root of the problem, it's this: garbage collection doesn't
>> live very well with explicit pointers. What do you think?
>
> This conclusion doesn't seem to follow. I don't see any difficulty in
> garbage collection and explicit pointers living together.
>
> What you may be driving at is that we might as well turn structs into
> a reference type, but I don't think that follows. A small struct like
> "type Point struct { x, y int}" often does not needs its address
> taken, and it can be efficiently passed by value. Why not simply pass
> it by value when the fields don't need to change? That is also an
> example where "func (p Point) ..." rather than "func (p *Point) ..."
> makes perfect sense.

Clearly, there is no "one best way" that is optimal for all situations. Sometimes, when the need arises, only the programmer will know which way is optimal for a given situation, and therefore the language and its toolchain must provide the ability for the programmer to force specific things onto the stack or the heap, or even into registers.

If all you need is "good enough" speed, and are willing to give up the ability to occasionally obtain truly massive speedups, then leaving all decisions to the compiler will simplify the situation. But that could make Go unusable or inefficient for performance-critical applications.

At the bottom of the problem is the issue of keeping all cores fed while both instructions and data are accessed via a single shared system memory interface. While this is typically something the OS tries to manage through various forms of load balancing, it can only do so much if the application is actively thrashing the caches. In this case, one bad app on one core can limit memory access for all other cores. There is an inherent performance limit to conventional multicore architectures when the shared single memory interface exists.

Modern parallel programming will, by necessity, move many code execution decisions away from the OS and into the language runtime. Go already does this by how it maps goroutines to threads. It is possible, perhaps likely, that Go's future thread-level load balancing optimizations will conflict with the OS's thread/task/core management optimizations. Should this become a chronic situation, it will be well beyond the ability of the programmer to manage it correctly for each application, at which point the distinction between the OS and the language runtime on multicore systems will need to be reassessed from a higher perspective.

Until this day comes, both the OS and Go will need to let the programmer directly manage the situation. Fortunately, all modern OSs and RTOSs I've used provide such features, often through a rich set of several overlapping mechanisms. Go will need to provide similar programmer control of its memory and scheduling behavior.

Architecturally, Go may want to expose its mechanisms via something that looks like an OS extension, rather than as a Go-specific feature. Then a patch submitted to Linus could permit Linux to quickly become "multicore language aware", much as Linux has recently become virtualization-aware.


-BobC

[Robert Griesemer]

On Sat, Nov 14, 2009 at 3:15 PM, Alex Kaushansky <kaush...@gmail.com> wrote:

>It is inconsistent, yes.  However, it doesn't show up in normal use,
>because in practice you would write SortedMap as an interface, and you
>would provide a SortedMap.New() function which returned a pointer
>which met the interface.  So code which uses SortedMap would look like
>code which uses map: it wouldn't use a pointer.

Oh I see. This explains a lot.

>A small struct like
>"type Point struct { x, y int}" often does not needs its address
>taken, and it can be efficiently passed by value.  Why not simply pass
>it by value when the fields don't need to change?  That is also an
>example where "func (p Point) ..." rather than "func (p *Point) ..."
>makes perfect sense.

I agree with that; big disadvantage of java  is that they don't allow user-defined value types.
Things make much more sense now (especially due to your SortedMap comment),
but it's difficult to extract this sense from language documentation.

I'd like you to clarify one more point - return type of make() function.
It seems that this type is difficult to characterize. Most likely word is "handle", but for some reason
this word is not a part of a language. Clearly, it's a kind of pointer, but not exactly.
Why didn't you make it a pointer? In case of map or channel, it's just a pointer, is it not?
In any case, if there's some problem addressed by this decision, same problem will arise
in user-defined types, too, but there's no way for user to define his own "handles".
Please describe some motivations behind this solution.

The return type of make is easy: make(T) returns a value of type T (whereas new(T) returns a value of type *T). At some point make and new were not separated which caused all kinds of problems (there was another mail about this, a few days ago).

make() is used to create new slice, map, and channel values. If you think about these values as descriptors for a data structure (rather than the entire data structure itself)  the reference semantic may become clearer. For instance, given a slice type []int, make([]int, 5, 10) creates a new slice (descriptor) s that might (in the implementation) look like this:

struct {

  data *[10]int;  // a pointer to the underlying array with max 10 elements

  start *int;  // a pointer to data

  len int;  // the value 5

}

when you assign a slice value (the descriptor), only the slice descriptor (value) is assigned, the underlying array is unchanged. It's like a shallow copy of the slice data structure; as a consequence you get the reference semantic. Similar things happen for maps and channel assignment; it's just a descriptor that is copied, not the underlying data, thus the reference semantics. The reason why the word "handle" or "descriptor" is not part of the language spec is that it doesn't need to be there (but perhaps it would make it clearer when explaining it).

Thus, make() "makes" a new descriptor value (it may allocate data underneath, such as the underlying slice array), but the slice (descriptor) is a fully initialized value. In contrast, new(T) only allocated storage for a T and returns the pointer to it, and that storage is initialized with the respective zero value.

It is possible to write:

slicePtr := new([]int);

*slicePtr = make([]int, 10);

and then you get a pointer to a slice (descriptor) as opposed to just the slice (descriptor) itself.

Hope that helps.

- gri

[Sverre Rabbelier]

Heya,

On Sun, Nov 15, 2009 at 01:09, Robert Griesemer <g...@golang.org> wrote:
> struct {
>   data *[10]int;  // a pointer to the underlying array with max 10 elements
>   start *int;  // a pointer to data
>   len int;  // the value 5
> }

Am I right in that:

> slicePtr := new([]int);

Would create the above struct with data = nil, start = nil, and len = 0?

> *slicePtr = make([]int, 10);

And this would set data to wherever the array is allocated, start to
it's first element, and len to 10?

--
Cheers,

Sverre Rabbelier

[Robert Griesemer]

On Sat, Nov 14, 2009 at 4:16 PM, Sverre Rabbelier <srabb...@gmail.com> wrote:

Heya,

On Sun, Nov 15, 2009 at 01:09, Robert Griesemer <g...@golang.org> wrote:
> struct {
>   data *[10]int;  // a pointer to the underlying array with max 10 elements
>   start *int;  // a pointer to data
>   len int;  // the value 5
> }

Am I right in that:

> slicePtr := new([]int);

Would create the above struct with data = nil, start = nil, and len = 0?

Yes, but from a Go program this will be presented as a slice of zero value, i.e. nil (you cannot "see" the descriptor directly from within Go). An extremely useful property of nil slices is that their length is 0. So len(*slicePtr) == 0 in this case.

 

> *slicePtr = make([]int, 10);

And this would set data to wherever the array is allocated, start to
it's first element, and len to 10?

yes. 

--
Cheers,

Sverre Rabbelier

[Sverre Rabbelier]

Heya,

On Sun, Nov 15, 2009 at 01:32, Robert Griesemer <g...@golang.org> wrote:
> On Sat, Nov 14, 2009 at 4:16 PM, Sverre Rabbelier <srabb...@gmail.com>

>> Am I right in that:
>>
>> > slicePtr := new([]int);
>>
>> Would create the above struct with data = nil, start = nil, and len = 0?
>
> Yes, but from a Go program this will be presented as a slice of zero value,
> i.e. nil (you cannot "see" the descriptor directly from within Go). An
> extremely useful property of nil slices is that their length is 0. So
> len(*slicePtr) == 0 in this case.

Ah, indeed if I dereference slicePtr and compare it to nil, the
comparison returns true.
However, if I do this, I get a segfault:

var slicePtr *[]int;
*slicePtr = nil;
equal := (*slicePtr == nil);
fmt.Printf("%t\n", equal);

What gives?

--
Cheers,

Sverre Rabbelier

[Russ Cox]

On Sat, Nov 14, 2009 at 17:34, Alex Kaushansky <kaush...@gmail.com> wrote:
> Speaking of slices, I don't fully understand the motivation. In this thread,
> someone explained that slices were supposed to address the problem
> similar to java bug where substring prevents the parent string from being
> garbage collected.

I said the opposite. We explicitly changed the substring
operations to be the *same* as what Java does, because
the alternative, we discovered, is worse: so much copying
that you end up passing around string+index+length
and avoid the substring operation entirely.

> But based on your definition of slice, I see same thing

You're right: it's the same as the Java behavior. We just
don't think the behavior is a bug.

Russ

[Robert Griesemer]

You declared a pointer (slicePtr) to a heap-allocated slice variable of type []int, but you didn't allocate that variable:

var slicePtr *[]int;

slicePtr = new([]int);  // or just slicePtr := new([]int); and remove the var decl above

*slicePtr = nil;

equal := (*slicePtr == nil);

fmt.Printf("%t\n", equal);

- gri

--
Cheers,

Sverre Rabbelier

[Rob 'Commander' Pike]

On Nov 15, 2009, at 1:44 PM, Alex Kaushansky wrote:

> Apparently, there's a typo in the following example in "Effective Go".
> "[]slice" should be "[]byte" , or ByteSlice (probably the latter)
>
> type ByteSlice []byte
>
> func (slice ByteSlice) Append(data []byte) []slice {
> // Body exactly the same as above
> }

Yup, a typo. Will fix.

-rob

[Ian Lance Taylor]

"Marcin 'Qrczak' Kowalczyk" <qrcz...@gmail.com> writes:

>> After a while
>> the requirement to always use a pointer seemed sort of pointless, so
>> we made it a reference type instead.  Same for channel.
>
> What about File?

Well, what about it? We could implement File as a pointer type easily
enough. So far we haven't done so.

Ian

[Ian Lance Taylor]

Alex Kaushansky <kaush...@gmail.com> writes:

> OK, I see your point, thanks.
> Just for curiosity: descriptor for map returned by make() function contains
> just a pointer, or something else? Same question for channel.

Yes, at present it's just a pointer.

Ian

[german diago]

> > However, if you decide to implement your own kind of map, say SortedMap, and
> > use it instead of default map,
> > situation will be completely different - you will have to declare it as
> >pointer. Isn't it strange?
>
> Yes, I don't like this either.

Neither do I. I think the semantics should be equal for every type.
But I sure there
was a reason to do this. My question is. Was this reason strong enough
to introduce
reference types? Because the language has pointers as well.

[Sverre Rabbelier]

Heya,

On Sun, Nov 15, 2009 at 22:29, Robert Griesemer <g...@golang.org> wrote:
> You declared a pointer (slicePtr) to a heap-allocated slice variable of type
> []int, but you didn't allocate that variable:

Ah, of course, so the "*slicePtr" part of the assignment is going to
fail, since the pointer doesn't point to a valid address. Thanks for
explaining :).

--
Cheers,

Sverre Rabbelier