GolangRT Docs?

[josvazg]

Golang runtime has been fully translated to Go for a while now. I know I could just read the source code directly but...

Is there any known or recommended documentation (talk, slides, article) about the runtime Go code?

I am specially interested in anything that describes the Golang runtime language subset restrictions:

• How does it avoid the GC? (while it's implementing it)
• Uses the GC primitives it provides at a lower level? A kind of its own malloc/free?
• Ensures all is in the heap?
• Does it avoid goroutines as well? how? Uses plain threads directly?
• What other restrictions apply compared to "userland" Golang?
• Why wouldn't the runtime subset be suitable for OS internals or real-time?
• Why wouldn't it be interesting to define a GolangRT subset language formally?

Thanks in advance,

Jose

[Ian Lance Taylor]

On Tue, Jan 24, 2017 at 12:46 PM, josvazg <jos...@gmail.com> wrote:
>
> Golang runtime has been fully translated to Go for a while now. I know I
> could just read the source code directly but...
>
> Is there any known or recommended documentation (talk, slides, article)
> about the runtime Go code?

Not really. It changes pretty quickly so any such docs would be more
or less out of date.

> I am specially interested in anything that describes the Golang runtime
> language subset restrictions:

Mostly the code is written carefully by people who understand the compiler.

> How does it avoid the GC? (while it's implementing it)

By not allocating memory. It is helped by a "secret" compiler option,
-+. When that compiler option is used, the compiler gives an error
for any implicit memory allocation. Note that this is not a change to
the language, it is, as you say, a subset of the language: operations
that implicitly allocate memory are forbidden.

> Uses the GC primitives it provides at a lower level? A kind of its own
> malloc/free?

There is a kind of simple malloc (but not free) in the
`persistentalloc` function.

> Ensures all is in the heap?

Not sure what you mean there.

> Does it avoid goroutines as well? how? Uses plain threads directly?

The runtime avoids goroutines (except in a few places) by simply not
using the `go` statement. The runtime uses plain threads to implement
goroutines, but otherwise does not use threads.

> What other restrictions apply compared to "userland" Golang?

The code that handles stacks can not itself use unlimited stack space.
The linker checks that it lives within the limited bounds available.

> Why wouldn't the runtime subset be suitable for OS internals or real-time?

It could be.

> Why wouldn't it be interesting to define a GolangRT subset language
> formally?

It's a fairly limited language. I'm not sure why anybody would want
to use it if they weren't already using Go.

Ian

[josvazg]

Thanks a lot Ian, that gives me a big picture of it!

To clarify my question about the heap, it was wrongly written; I meant to say if all memory was kept in the stack (instead to the heap). You already it answered anyway.

Here are my takeaways (with a few questions):

1. There is not a clearly defined GolangRT subset (at least for now), it is a moving target requiring very low level knowledge on the compiler inner workings.
   
2. For the runtime, implicit memory allocations are flagged by the compiler with a secret option.
• Why not make this available to normal users? won't it be useful on some use cases?
3. When memory is actually needed by the runtime, either use the stack if possible or use 'persistentalloc' but no free:
• Can't that mean that the runtime may leak memory if golang userland repeatedly requests work so that the runtime calls persistentalloc repeatedly?
• Can in go runtime a stack allocated memory be used by callees of the function that allocated that memory on its frame? Or is it restricted to only live in its own function?
4. The runtime must be using it's own memory space, managed separately to the one it provides to Golang userland. I infer this as userland memory can be requested, freed (by the GC) and even given back to the OS.
5. The runtime seems to be:
• A library when userland requests a service from it, so it runs in the same thread as the user code?
  
• A set of active tasks, like the GC, that are basically single threaded.
• When a go program is single threaded as well, does the runtime share the thread or does it live on a separate OS thread anyway?
6. The stack size for the runtime is static.
7. GolangRT subset language not only is fuzzy and changing, it is also very limited to me generally useful for anything else.

Thanks,

Jose

[murali]

IIRC, even interface conversions would potentially allocate memory. (Correct me if I am wrong. It might have been changed.)

And I wonder if the 'restrictions' can be checked through an external tool like 'go vet' or another rather than deep corners of the compiler?

If so, that would be useful for some applications which need strict zero-allocation guarantee, or at least, giving hints for performance tuning.

Considering that Go can do almost what C can do except maybe union in terms of the granularity of low-level control, I wouldn't say it's that useless.

[Ian Lance Taylor]

On Wed, Jan 25, 2017 at 12:40 AM, josvazg <jos...@gmail.com> wrote:
>
> Here are my takeaways (with a few questions):
>
> There is not a clearly defined GolangRT subset (at least for now), it is a
> moving target requiring very low level knowledge on the compiler inner
> workings.
> For the runtime, implicit memory allocations are flagged by the compiler
> with a secret option.
>
> Why not make this available to normal users? won't it be useful on some use
> cases?

Because then we would have to define it formally, as you suggested.
Right now the option just means "whatever we need to help ensure that
the Go runtime is correct."

> When memory is actually needed by the runtime, either use the stack if
> possible or use 'persistentalloc' but no free:
>
> Can't that mean that the runtime may leak memory if golang userland
> repeatedly requests work so that the runtime calls persistentalloc
> repeatedly?

The runtime doesn't do that. It only uses persistentalloc for things
that wil remain live for the rest of the program.

> Can in go runtime a stack allocated memory be used by callees of the
> function that allocated that memory on its frame? Or is it restricted to
> only live in its own function?

Memory allocated on the stack in function F can not be used by callees
of F. After F returns, the memory is gone.

> The runtime must be using it's own memory space, managed separately to the
> one it provides to Golang userland. I infer this as userland memory can be
> requested, freed (by the GC) and even given back to the OS.

Only for the special case of persistentalloc.

> The runtime seems to be:
>
> A library when userland requests a service from it, so it runs in the same
> thread as the user code?

Yes, although speaking of "threads" can be misleading when talking about Go.

> A set of active tasks, like the GC, that are basically single threaded.

Yes, although the GC is not single threaded.

> When a go program is single threaded as well, does the runtime share the
> thread or does it live on a separate OS thread anyway?

It's misleading to speak of "threads" in a Go program. A Go program
is never single-threaded. But if GOMAXPROCS == 1, then in general
only one thread will be executing at a time. The runtime code doesn't
execute in separate threads, but it does execute in separate
goroutines.

> The stack size for the runtime is static.

For some parts of the runtime, yes, specifically the parts that handle
stack copying.

> GolangRT subset language not only is fuzzy and changing, it is also very
> limited to me generally useful for anything else.

Yes.

Ian

[josvazg]

Thanks a lot Ian!

My mind picture of the runtime is more detailed now. I guess for more I should now read the code itself.

Just one side issue. You said:
"
Memory allocated on the stack in function F can not be used by callees
of F. After F returns, the memory is gone.
"

I don't quite get it.

As I understand it, a caller is earlier in the stack that any of the functions it calls (callees), so those callees will always return before the caller does, their scope is a subset of the caller's.

As such, the fact that the callers locals are gone when it returns does not matter to the callees, which have already been long gone by then.

That is, of course, if those callees are using that callers variables locally only, not passing then elsewhere (other Goroutines, effectively other stacks). That is, there is no variable escape from the stack.

What did I miss here?

[josvazg]

Also, any recommended entry point or guide to read the runtime source code?

[josvazg]

Replying to myself. Here is a must read to understand the go runtime better and its source code:
https://github.com/golang/go/blob/master/src/runtime/HACKING.md

[Ian Lance Taylor]

You're right, sorry, I mixed callee and caller. Functions that F
calls can freely refer to memory allocated by F on the stack.

Ian

[T L]

On Wednesday, January 25, 2017 at 5:16:07 AM UTC+8, Ian Lance Taylor wrote:

On Tue, Jan 24, 2017 at 12:46 PM, josvazg <jos...@gmail.com> wrote:
>
> Golang runtime has been fully translated to Go for a while now. I know I
> could just read the source code directly but...
>
> Is there any known or recommended documentation (talk, slides, article)
> about the runtime Go code?

Not really.  It changes pretty quickly so any such docs would be more
or less out of date.


> I am specially interested in anything that describes the Golang runtime
> language subset restrictions:

Mostly the code is written carefully by people who understand the compiler.


> How does it avoid the GC? (while it's implementing it)

By not allocating memory.  It is helped by a "secret" compiler option,
-+.  When that compiler option is used, the compiler gives an error
for any implicit memory allocation.  Note that this is not a change to
the language, it is, as you say, a subset of the language: operations
that implicitly allocate memory are forbidden.

What is the difference between -+ and -m for checking whether or not a variable "escapes to heap"?
It looks the -m reports much more escapes than -+.
 

[Ian Lance Taylor]

On Thu, Jan 26, 2017 at 9:18 AM, T L <tapi...@gmail.com> wrote:
>
> On Wednesday, January 25, 2017 at 5:16:07 AM UTC+8, Ian Lance Taylor wrote:
>
>> By not allocating memory. It is helped by a "secret" compiler option,
>> -+. When that compiler option is used, the compiler gives an error
>> for any implicit memory allocation. Note that this is not a change to
>> the language, it is, as you say, a subset of the language: operations
>> that implicitly allocate memory are forbidden.
>>
>
> What is the difference between -+ and -m for checking whether or not a
> variable "escapes to heap"?
> It looks the -m reports much more escapes than -+.

-m is a debugging option for analyzing the compiler's escape analysis
pass. -+ is a code generation option that causes the compiler to
reject implicit memory allocations. They don't have anything in
common except that they are both related to memory allocation. For
example, if an apparent memory allocation (such as a call to new)
does not escape (as could be reported by -m) then it is accepted by -+
(because it does not allocate memory).

Ian

[T L]

It looks a call to new will not be reported by -m either.

I still don't understand what are implicit memory allocations, could you make an explanation?

[Jakob Borg]

It depends. A call to new can (and will, often) give you a pointer to a stack allocated object. If you pass that pointer to something like fmt.Println() you'll see the object escape to the heap, get flagged by -m, and presumably forbidden by -+.

//jb

[Ian Lance Taylor]

On Thu, Jan 26, 2017 at 11:10 AM, T L <tapi...@gmail.com> wrote:
>
> I still don't understand what are implicit memory allocations, could you
> make an explanation?

An example of an implicit memory allocation:

var globalVar *int
func f() {
var i int
globalVar = &i
}

Another one:

var globalVar interface{}
func f() {
globalVar = 0
}

Ian

[T L]

It looks -+ doesn't think the second one is implicit memory allocation.
"-m" really think it is an "escapes to heap".

So I am still not very clear on what is implicit memory allocation.
It looks the output of -+ is hard to predict.
For example, in the following program, f1 and f3 are reported as implicit memory allocation, but f2 and f4 are not:

package main

import "fmt"

var globalVar1 *int
func f1() {
    var i int
    globalVar1 = &i
}

var globalVar2 interface{}
func f2() {
    globalVar2 = 0
}

func f3() {
    a := 2
    fmt.Println(&a)
}

func f4() {
    c := make(chan int, 10)
    c <- 1
   
    go func() {
        <-c
    }()
}

func main() {
}

 

[T L]

On Friday, January 27, 2017 at 3:38:12 AM UTC+8, Jakob Borg wrote:

It depends. A call to new can (and will, often) give you a pointer to a stack allocated object. If you pass that pointer to something like fmt.Println() you'll see the object escape to the heap, get flagged by -m, and presumably forbidden by -+.

so

new(T)

is just a sugar of the following one?

var t T; &t

 

[Ian Lance Taylor]

On Thu, Jan 26, 2017 at 7:53 PM, T L <tapi...@gmail.com> wrote:
>
> On Friday, January 27, 2017 at 4:05:03 AM UTC+8, Ian Lance Taylor wrote:
>>
>> On Thu, Jan 26, 2017 at 11:10 AM, T L <tapi...@gmail.com> wrote:
>> >
>> > I still don't understand what are implicit memory allocations, could you
>> > make an explanation?
>>
>> An example of an implicit memory allocation:
>>
>> var globalVar *int
>> func f() {
>> var i int
>> globalVar = &i
>> }
>>
>> Another one:
>>
>> var globalVar interface{}
>> func f() {
>> globalVar = 0
>> }
>>
>> Ian
>
>
> It looks -+ doesn't think the second one is implicit memory allocation.
> "-m" really think it is an "escapes to heap".

My apologies for getting it wrong.

> So I am still not very clear on what is implicit memory allocation.
> It looks the output of -+ is hard to predict.

Yes, -+ is hard to predict. That is why it is not documented and is
only intended for use when compiling the runtime package.

Ian

[Konstantin Khomoutov]

On Thu, 26 Jan 2017 19:57:22 -0800 (PST)
T L <tapi...@gmail.com> wrote:

> > It depends. A call to new can (and will, often) give you a pointer
> > to a stack allocated object. If you pass that pointer to something
> > like fmt.Println() you'll see the object escape to the heap, get
> > flagged by -m, and presumably forbidden by -+.
> >
>
> so
>
> new(T)
>
> is just a sugar of the following one?
>
> var t T; &t

Sort of yes, but the allocated (pointed at) value will be anonymous
(not contained in any variable).

IIUC, Go compiler makes no distinction between "stack" and "heap"
allocation: you are free to return pointer to a variable which -- in
C -- would be stack-allocated, and the Go compiler will notice the
value of that variable "escapes" its scope and will make sure it will
be heap-allocated (or otherwise preserved).

Allocating a value with new(T) has the similar property: contrary to the
(not overloaded) operator new from C++, calling new() does not mean you
will get a heap-allocated value: if it can be proven to not escape, it
may well be allocated on the stack.

As you can see for yourself, [1] does not in any way mention how
exactly a value is allocated -- just that a pointer to its memory is
returned.

1. https://golang.org/ref/spec#Allocation