Adapting go for embedded/real-time use

[Mikey]

I have read a few posts discussing the use of go in embedded/real-time systems and it seems that it is unsuitable for the following reasons:

• Garbage collection would interfere with real-time behavior unpredictably.
• Scheduling of go-routines isn't sufficiently tied into the OS scheduler.
• Segmented stacks could potentially cause problems for OS exception handling if the OS uses the process' stack during exception processing.
• Perhaps there are other issues I haven't identified yet.

Although these features are important to go's design, I'm wondering if it makes sense to fork the golang project and strip them out so that the remaining go features could be used in an embedded/real-time context.  Perhaps it could be called embedded-golang (ego? ;) or go-lang-rt (gort? ;)

I am contemplating petitioning my company to sponsor such an effort (i.e. my time) so I'm looking for a discussion on whether others are interested in using go this way and what technical hurdles need to be investigated before going forward.

Thoughts?

[Hotei]

There was a discussion about this about a year and a half ago - can't see that much has changed since ...

groups.google.com/forum/?fromgroups=#!topic/golang-nuts/95BJqJvb7I0

[Mikey]

I read that thread, but as far as I could tell, nothing ever came of it.  Maybe we can make more progress this time around.

[Kevin Gillette]

Ironically, segmented stacks are likely _critical_ for idiomatic go programs to function well in low memory environments. If this is true, "embedded golang" might not be a proper name.

On Monday, April 22, 2013 10:15:59 AM UTC-6, Mikey wrote: 

• Scheduling of go-routines isn't sufficiently tied into the OS scheduler.

The reason Go does its own scheduling is that the supported OS's don't have (sufficient) interfaces for a userspace process to indicate how its threads relate and under what conditions they should run. Even though good OS schedulers are more sophisticated than Go's scheduler, they don't have the information that the Go runtime has, and without that information, the Go scheduler can particularly handle internal synchronization and signalling (such as with the use of channels) much more efficiently than a userspace process possibly could if scheduling were the exclusive domain of a contemporary OS' scheduler. Goroutines aren't the same as threads, and OSs generally only know about threads, so there's no way to usefully tie goroutine scheduling into kernel scheduling without modifying the kernel to have a concept of "schedulable entity" that is finer grained than "thread".

If the intent is to run a Go program on an RTOS with, at best, process-granular realtime guarantees, then that's already handled transparently: a non-cooperatively scheduled RTOS is certainly able to (and will) preempt some or all of the threads in a running Go program, and the Go runtime will not fail in those cases. If the intent is to give individual goroutines independent realtime guarantees, that can be achieved using runtime.LockOSThread and then making syscalls to arrange realtime requirements. In that case, "realtime goroutines" should not use blocking channels, and lockless structures should be preferred to the stdlib mutexes whenever data sharing is required.

• Segmented stacks could potentially cause problems for OS exception handling if the OS uses the process' stack during exception processing.

By "exception processing", do you mean preempting a process to meet some realtime requirement? If so, it seems that all we'd need is a dummy stack that can reinitialize the runtime when and if the process is switched back to. If you meant some OS facility for providing extended core dump output, then surely the OS wouldn't assume the presence of a traditional stack (to do otherwise would be ludicrously non-robust), and the benefits of OS tracing of process exceptions would likely not begin to make up for the shortcomings of not having segmented stacks.

[Mikey]

Those are good points so let me establish some additional real-time OS context from which to discuss them.  My background is primarily with VxWorks so I'm trying to envision how a golang program can be made to run as a VxWorks real-time process (RTP), which is roughly analogous to a standard process in unix.  RTPs consist of one or more tasks at varying priorities that can be scheduled and pre-empted by outside sources such as other RTPs, kernel tasks, or interrupts.

On Monday, April 22, 2013 3:03:06 PM UTC-4, Kevin Gillette wrote:

Ironically, segmented stacks are likely _critical_ for idiomatic go programs to function well in low memory environments. If this is true, "embedded golang" might not be a proper name.

Agreed.  My goal is to try to adapt golang to operate in a real-time environment, not a memory-constrained environment.

 
On Monday, April 22, 2013 3:03:06 PM UTC-4, Kevin Gillette wrote:

On Monday, April 22, 2013 10:15:59 AM UTC-6, Mikey wrote: 

• Scheduling of go-routines isn't sufficiently tied into the OS scheduler.

The reason Go does its own scheduling is that the supported OS's don't have (sufficient) interfaces for a userspace process to indicate how its threads relate and under what conditions they should run. Even though good OS schedulers are more sophisticated than Go's scheduler, they don't have the information that the Go runtime has, and without that information, the Go scheduler can particularly handle internal synchronization and signalling (such as with the use of channels) much more efficiently than a userspace process possibly could if scheduling were the exclusive domain of a contemporary OS' scheduler. Goroutines aren't the same as threads, and OSs generally only know about threads, so there's no way to usefully tie goroutine scheduling into kernel scheduling without modifying the kernel to have a concept of "schedulable entity" that is finer grained than "thread".

If the intent is to run a Go program on an RTOS with, at best, process-granular realtime guarantees, then that's already handled transparently: a non-cooperatively scheduled RTOS is certainly able to (and will) preempt some or all of the threads in a running Go program, and the Go runtime will not fail in those cases. If the intent is to give individual goroutines independent realtime guarantees, that can be achieved using runtime.LockOSThread and then making syscalls to arrange realtime requirements. In that case, "realtime goroutines" should not use blocking channels, and lockless structures should be preferred to the stdlib mutexes whenever data sharing is required.

I like where you went with the "process-granular realtime guarantees" and I think that would be the way to go so that it would fit reasonably well with existing code and practices.  There would probably need to be some work in the golang runtime to ensure that everything behaved deterministically.

While thinking more about what this would look like in VxWorks, I think that if you were to have golang code running in multiple RTOS tasks (say two tasks for now) and both those tasks were within the same RTP, then each task would probably need to have its own golang runtime environment where it could kick off go-routines within the context of its own task.  The two tasks would essentially behave independently of each other and wouldn't really share any golang context between each other even though they would be running out of the same RTP address space.

Note that without garbage collection, memory management between go-routines within each task would probably be error-prone.

• Segmented stacks could potentially cause problems for OS exception handling if the OS uses the process' stack during exception processing.

By "exception processing", do you mean preempting a process to meet some realtime requirement? If so, it seems that all we'd need is a dummy stack that can reinitialize the runtime when and if the process is switched back to. If you meant some OS facility for providing extended core dump output, then surely the OS wouldn't assume the presence of a traditional stack (to do otherwise would be ludicrously non-robust), and the benefits of OS tracing of process exceptions would likely not begin to make up for the shortcomings of not having segmented stacks.

I think VxWorks uses a separate exception stack for CPU exceptions, but that may actually be configurable.

As I was thinking about this more, I don't think the segmented stacks are a problem for golang code because that would have to be hidden from whatever OS spawned the golang executable.

[Paulo Pinto]

Have a look at Oberon-7 for embedded development.

http://www.astrobe.com/default.htm (A company selling Oberon
toolchains for embedded development)
http://www.oberonday2011.ethz.ch/talks/ ("ARM Embedded Development
Using Oberon-07" )

http://www.inf.ethz.ch/personal/wirth/Articles/Oberon.html ("A
Computer System for Model Helicopter Flight Control, Technical Reports
285, ETH Zürich, Institute of Computer Systems, 03 1999." )

This can help you see how Go could achieve similar functionality.

In case you don't know it, Oberon is a GC enabled systems programming
language, used for
developing two desktop operating systems at Zurich's Technical
University during the mid 90's,
similar to the Smalltalk environment in user interface. Those systems
were used as desktop operating
systems by ETHZ researchers and students for operating system classes.

--
Paulo

On Apr 22, 6:15 pm, Mikey <mbsul...@mac.com> wrote:
> I have read a few posts discussing the use of go in embedded/real-time
> systems and it seems that it is unsuitable for the following reasons:
>

>    - Garbage collection would interfere with real-time behavior
>    unpredictably.
>    - Scheduling of go-routines isn't sufficiently tied into the OS
>    scheduler.
>    - Segmented stacks could potentially cause problems for OS exception

>    handling if the OS uses the process' stack during exception processing.

>    - Perhaps there are other issues I haven't identified yet.