What steps need to be done to bring the Go Tiny runtime up to date?

[logan.nea...@gmail.com]

So, originally I had planned on attempting to write a toy kernel in Go. I wanted to do as much of it as I could in Go and the rest in Assembly and stay away from any C dependencies, if I could. However, to my knowledge, the Tiny Runtime is no longer available. Even if it were, I'm not sure that I've seen any documentation on how to use it.

I know that the Tiny runtime was removed because it was taking too much effort to keep it updated with the rest of the language. At least, this was the reason I saw a developer cite. So, what steps would need to be taken to bring the Tiny runtime back up to date? I still have dreams of writing a kernel in Go, but even if that never occurs, I'd be happy to be able to run Go code on bare metal. 

Many people have argued that some of the features that make the Go runtime so large (the GC, goroutines, etc) are not feasible in the Tiny runtime and, at that point, it's not really Go. They may be right regarding the feasibility, but I disagree that the lack of those features makes the language completely different. I would still enjoy programming in Go without some of those features. I sure hope they can be included, but it's not a deal breaker.

So, can anyone point me on the right track?

[Kyle Lemons]

I was hoping to do the same thing awhile back, but that was before the myriad runtime improvements that have come around recently.

Broadly, I think the work would fall into a few categories (hopefully someone will correct me if this is outdated or wrong):

1) make the runtime truly single threaded

2) stub out or implement the various system calls that the runtime uses

3) implement the locking primitives that the runtime uses

4) rewrite or implement the timing mechanisms that the runtime uses

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jeff R. Allen]

It went from inconvenient to really hard a long time asp, even before go 1, because go started depending on the os to keep track of memory for it. See the numerous threads about why go programs have a huge virtual footprint but a much smaller resident set size. Without an underlying vm system, you need a new memory manager, and the one in the current runtime its not built to be unplugged and simply replaced.

I agree that go on bare hardware is unselfishly interesting, but maybe something equally interesting is to combine a kernel, busybox, and a go program to make a single-tenant linux instance in a tiny image. That way you use linux for what it's good at (vm and hardware access) and go for what ours good at (everything user space)

Good luck!

Jeff

[Diddymus]

Not sure if this helps but you could pass your binary to the linux kernel using the init= parameter for a very small system just running a single Go program. I did this with some Go 1.0 code and haven't tried recently with Go 1.1.1. You can even compile your own kernel image with just the bits you need for an even smaller footprint.

[Logan Collins]

Kyle, it's exactly because it assumes things that I think I should get at least a small background before I start reading through it all. 

I see. 'runtime' is to Go as 'init' is to the Linux Kernel.

Diddymus, that is very interesting! I'll probably play around with it and maybe use it for testing, but it's not the same as running on bare hardware.

On Fri, Aug 9, 2013 at 8:51 AM, Diddymus <wol...@gmail.com> wrote:

Not sure if this helps but you could pass your binary to the linux kernel using the init= parameter for a very small system just running a single Go program. I did this with some Go 1.0 code and haven't tried recently with Go 1.1.1. You can even compile your own kernel image with just the bits you need for an even smaller footprint.

--
---Logan Collins

GPG Public Key 2AAE8775
0xfc1185c82aae8775
http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0xFC1185C82AAE8775

[Diddymus]

Logan,

If you want to experiment you can take a normal Linux box (I tried with Debian on a desktop and Raspbian on an RPi) and just reboot, tweaking the grub kernel line to add init=/path/to/go-program for that boot. The advantage here is you still have the full OS installed and everything available if needed but only the minimum is running. In fact I wrote a little shell script to load the network module and bring up networking then run the Go program as no serverices are started. Think I used dash to run the script. On the Raspbery Pi (Raspbian) I went from:

$ free -h

               total       used       free     shared    buffers     cached

  Mem:          249M        37M       212M         0B       6.4M        18M

  -/+ buffers/cache:        12M       237M

  Swap:         465M         0B       465M

To this:

# free -h

               total       used       free     shared    buffers     cached

  Mem:          249M       8.7M       241M         0B       1.7M       1.8M

  -/+ buffers/cache:       5.2M       244M

  Swap:           0B         0B         0B

Those figures are from just after booting - and I've just noticed swap does not get mounted either :)

[Ross Schulman]

Hi Logan and others.

I'm in exactly the same boat: interested in developing a hobby kernel in Go and wondering what would have to change to get a compiler pointed at something like i586-elf is for gcc. It seems obvious that much of the runtime (just like much of libc) would not be available in the new environment and that it would have to be built back up on the other side, but now I'm not even sure that's possible from what people here seem to be saying. That would be too bad as it sounds like an interesting project that I'd love to collaborate with others on.

-Ross

[Ian Lance Taylor]

On Thu, Aug 15, 2013 at 6:11 AM, Ross Schulman <rsch...@gmail.com> wrote:
> Hi Logan and others.
> I'm in exactly the same boat: interested in developing a hobby kernel in Go
> and wondering what would have to change to get a compiler pointed at
> something like i586-elf is for gcc. It seems obvious that much of the
> runtime (just like much of libc) would not be available in the new
> environment and that it would have to be built back up on the other side,
> but now I'm not even sure that's possible from what people here seem to be
> saying. That would be too bad as it sounds like an interesting project that
> I'd love to collaborate with others on.

The Go compiler/linker generates code that assumes thread local
variables, and those have to live somewhere. I think that is the only
system dependency in the compiler/linker; everything else is in the
runtime. So to get a compiler/linker like i586-elf, pick a TLS
scheme, and you may as well use the one that most ELF systems use. So
in fact the default Go compiler is already pretty much a i586-elf
compiler.

Then you'll need to write key functions like runtime·newosproc and
runtime·SysAlloc for your OS. In general you need to get the runtime
package fully up and running, or you won't be able to do anything.
Once you have runtime you can at least run a hello world program using
the builtin print function. Then you can think about the syscall and
other packages.

Ian

> On Thursday, August 8, 2013 6:45:10 PM UTC-4, logan.nea...@gmail.com wrote:
>>
>> So, originally I had planned on attempting to write a toy kernel in Go. I
>> wanted to do as much of it as I could in Go and the rest in Assembly and
>> stay away from any C dependencies, if I could. However, to my knowledge, the
>> Tiny Runtime is no longer available. Even if it were, I'm not sure that I've
>> seen any documentation on how to use it.
>>
>> I know that the Tiny runtime was removed because it was taking too much
>> effort to keep it updated with the rest of the language. At least, this was
>> the reason I saw a developer cite. So, what steps would need to be taken to
>> bring the Tiny runtime back up to date? I still have dreams of writing a
>> kernel in Go, but even if that never occurs, I'd be happy to be able to run
>> Go code on bare metal.
>>
>> Many people have argued that some of the features that make the Go runtime
>> so large (the GC, goroutines, etc) are not feasible in the Tiny runtime and,
>> at that point, it's not really Go. They may be right regarding the
>> feasibility, but I disagree that the lack of those features makes the
>> language completely different. I would still enjoy programming in Go without
>> some of those features. I sure hope they can be included, but it's not a
>> deal breaker.
>>
>> So, can anyone point me on the right track?
>

[Andrew Jackman]

I have been pursing this recently, also. 

I have checked out the historical, last-known version of the tiny runtime from the original tree. In addition, I've gotten Jeff Allen's code and GOFY's code. All of these worked at one time, and it looks like the biggest difference between all of these and the current tree is that the Go project has moved from using makefiles to a bootstrapping system that keeps the code tree free of build instruction. Consequently, I've been re-integrating the tiny runtime into the new tree structure and updating the build tools to support 'tiny' as an OS target. Right now, the bootstrap/build partially completes.

After I can get the code to generate, I'll be tackling the problem of making sure that the build generates ELF-friendly binaries/modules/objects and I can link it to a bootloader. 

The bootloader is an interesting topic, too. I was surprised that the code for the bootloader was not in the code tree with tiny, so I hunted it down. It's from the xv6 project. I copied it into my github repo for quick reference. I've also been using the barebones bootloader from wiki.osdev.org, which is easier to get your head around if you're interested in understanding what the bootloader does for you. The osdev site also provides some hints on how the linking process works and what needs to be done to get everything l linked up.

My interest in kernel development is beyond simply writing it in Go. So, regardless of whether I can get Go into a bare metal environment, I'll still move forward. 

Please let me know what progress you've made. Thanks!