SDL2 and runtime.LockOSThread

[Jared Anderson]

I have been working on a binding for SDL2, and I have found out that in order for SDL2 to work on OSX a goroutine needs to be locked to the main OS thread.  As I understand it this is done by calling runtime.LockOSThread in an Init function.

So my question is should the user be required to call runtime.LockOSThread in the main package Init function or is it fine for the sdl package to call runtime.LockOSThread in its Init function?  My gut is telling me that it should be done in the main package, but I am not sure.

Here is a link to my binding: http://bitbucket.org/dooots/go-sdl2

[Steve McCoy]

The SDL docs say not to expect window, rendering, and event functions to work if they aren't on the main thread. I believe things will also fail on Windows in this case, at least for events. Since this subject came up on the list twice this week, I'm starting to think that it should be done by the sdl packages or people will continue to write non-portable programs, because who reads the docs?

[André Moraes]

On Thu, Nov 29, 2012 at 5:34 AM, Jared Anderson <redd...@gmail.com> wrote:
> I have been working on a binding for SDL2, and I have found out that in
> order for SDL2 to work on OSX a goroutine needs to be locked to the main OS
> thread. As I understand it this is done by calling runtime.LockOSThread in
> an Init function.
>
> So my question is should the user be required to call runtime.LockOSThread
> in the main package Init function or is it fine for the sdl package to call
> runtime.LockOSThread in its Init function? My gut is telling me that it
> should be done in the main package, but I am not sure.

The user shouldn't be concerned with that, but you could write
something like this:

commChan := make(channel interface{}, 0)
go runOutisideMainThread(commChan)
sdl2.Loop(commChan) // this block everything in the main thread.

This isn't pretty, but since it's a limitation on the SDL, the user of
your binding are accepting this.

Another alternative would be to expose something like:

sdl.OneIteration()

This allow the user to have control over the sdl loop if he want's

With something like that, you allow the default usage (let SDL do
everything), or a more controlled use (I control the SDL).


--
André Moraes
http://amoraes.info

[Ian Lance Taylor]

On Wed, Nov 28, 2012 at 11:34 PM, Jared Anderson <redd...@gmail.com> wrote:
>
> So my question is should the user be required to call runtime.LockOSThread
> in the main package Init function or is it fine for the sdl package to call
> runtime.LockOSThread in its Init function? My gut is telling me that it
> should be done in the main package, but I am not sure.

It's fine to call runtime.LockOSThread in an init function. But it
may not solve your problem, because you don't know whether the main
program will call your functions in the main goroutine. You will
probably have to start your own goroutine, have it call
runtime.LockOSThread, and use that goroutine to make the C calls.

Ian

[Steve McCoy]

On Thursday, November 29, 2012 12:04:52 PM UTC-5, Ian Lance Taylor wrote:

It's fine to call runtime.LockOSThread in an init function.  But it
may not solve your problem, because you don't know whether the main
program will call your functions in the main goroutine.  You will
probably have to start your own goroutine, have it call
runtime.LockOSThread, and use that goroutine to make the C calls.

Ian

Unfortunately, the GUI systems with this limitation really do insist that the function in question be called from *the* main thread. I don't think there's a way to force a new goroutine onto a certain thread, but you'd know better than me. If it were possible, that would let some of my stuff become so much nicer.

[bryanturley]

I have used opengl from a dedicated goroutine, it suffers similar design flaws requiring LockOSThread.
The key is to not make a single call to anything that might make use of thread local storage (sdl/opengl) until after LockOSThread is run in the goroutine.
I think the gl context is the only thing stored thread local (in opengl) not sure for sdl.


 

[bryanturley]

 
I should add this was only in linux/x11

[Russ Cox]

It's too bad about these graphics libraries and their main thread
restrictions. Here is the pattern I would use.

package sdl

// Arrange that main.main runs on main thread.
func init() {
runtime.LockOSThread()
}

// Main runs the main SDL service loop.
// The binary's main.main must call sdl.Main() to run this loop.
// Main does not return. If the binary needs to do other work, it
// must do it in separate goroutines.
func Main() {
for f = range mainfunc {
f()
}
}

// queue of work to run in main thread.
var mainfunc = make(chan func())

// do runs f on the main thread.
func do(f func()) {
done := make(chan bool, 1)
ch <- func() {
f()
done <- true
}
<-done
}

And then other functions you write in package sdl can be like

func Beep() {
do(func() {
// whatever must run in main thread
})
}

Russ

[Sebastien Binet]

On Thu, Nov 29, 2012 at 9:42 PM, Russ Cox <r...@golang.org> wrote:
> It's too bad about these graphics libraries and their main thread
> restrictions. Here is the pattern I would use.

nice. this should be put into one of those wiki pages...
simple and elegant.

-s

> --
>
>

[Carlos Castillo]

I would prefer that if the suggested package (an SDL2 binding) was in fact a light weight wrapper around the C-library, that this thread-control mechanism be left out. If I wanted to use another C-library with similar restrictions (only run from main thread), it is now impossible for it to work.

I suggest that the functionality to run code on the "main" thread be implemented in it's own package, with an exported Do function, so that a programmer can use multiple libraries at once.

If the suggested package is instead meant to be a high-level solution like PyGame or the like, than this solution is fine.

I would prefer slightly more support in the runtime, something like runtime.SetThreadAffinity(NAME) which told the runtime to only run a goroutine on a matching thread. Such a feature would allow multiple goroutines to run on the same thread, instead of now where LockOSThread() forces a 1-1 association of thread to goroutine. It also would prevent the case of some package "grabbing" the main thread before your package could. NAME wouldn't necessarily be a system or thread specific identifier, but could be a class such as "main", "sound", "network". The "main" one could be created by the runtime, and force execution to the main thread, while others could be created (by using a non-existant name) for other tasks which needs or would benefit from sharing the same OS thread.

[minux]

On Fri, Nov 30, 2012 at 4:48 AM, Sebastien Binet <seb....@gmail.com> wrote:

On Thu, Nov 29, 2012 at 9:42 PM, Russ Cox <r...@golang.org> wrote:
> It's too bad about these graphics libraries and their main thread
> restrictions. Here is the pattern I would use.

nice. this should be put into one of those wiki pages...
simple and elegant.

I've put it into http://code.google.com/p/go-wiki/wiki/LockOSThread

feel free to elaborate or correct errors.

[Mortdeus]

Dont worry people, im working on a goroutine friendly solution for all your cross platform gl woes. As ive been saying for a long time in #go-nuts; EGL is the way FORWARD for golang. Stop working on crap desktop GL related projects. For example, http://go-gl.github.com/. This project will only mislead people into making false assumptions regarding go shortcomings that do not even exist because it presents itself like its the "official" gl solution for golang. 

If people really want to move forward with using go to make video games, then get on IRC and ping me so I can update you on what needs to be worked on. If we stay fragmented and dont collaborate then nothing is ever going to get done.  

https://github.com/mortdeus/egles

[Patrick Mylund Nielsen]

Looks like it's very related to what you're posting. Those posts aren't awesome; they're just self-promotion, which would be fine, except your project still doesn't do anything, AFAICT. (It's odd that you're calling the other packages crap when they're very accomplished in comparison.)

You were initially filtered because you kept posting links to gocos2d.org before it even had any content on it or the navigation links worked, and the source code for the project was a file consisting of 20 lines that did nothing. Make something that does something, then call for collaborators and link to it. If you link to other interesting things in the meantime, you'll be unfiltered eventually. This is not special to r/golang; it's how reddit works.

On Sat, Dec 1, 2012 at 10:14 PM, Mortdeus <mort...@gocos2d.org> wrote:

Wow, so I just found out that Uriel must have flagged me to have all my posts to /r/golang filtered out for reasons completely unrelated to the content that I posted in /r/golang. Some of my most recent posts were awesome, yet dont even show up on the subreddit. 

25 steps to wrap long repetitive C headers for cgo in under an hour using Sublime Text 2.
Gordon the Gopher???

Including my most recent post.
Stop using/making Opengl/SDL/GLFW wrappers. Help with egles, instead.

Im also the author of gocos2d, which is turning out to be a well polished golang sdk. 
http://blog.gocos2d.org/

Can anybody remediate this issue for me considering uriel isnt here with us anymore? :( 

--
 
 

[Mortdeus]

Which is why I havent posted anything further. I honestly only started using reddit after joining the go community earlier this year and wasnt even aware that it was frowned upon. I literally got banned for my first posts ever...

Lets put it this way, I made the first post. Realized the name wasnt well thought of like the other posts. So I created a new post with a better name that would have looked nicer if the project was upvoted to the front page. Then I posted my powered by gopher buttons, twice because I had finished the pink gopher button after popular request awhile after the first post fell 2 -3 pages back.

So it seems like I got banned because I invested alot of time doing something nice for everyone, and posted an update. 
You may criticize my lack of code submitted to my projects, however be aware that is unjust criticism considering Ive had to study the language src and all to gain a thorough understanding of how it works. I have had to study DRI, mesa, and wayland to understand the best way to make a game sdk with golang that is actually innovative. Ive had to thoroughly study cocos2d-iphone's sdk so I can figure out how to model this 
http://www.cocos2d-iphone.org/api-ref/latest-stable/inherit_graph_1.png
into something like this 
http://blog.gocos2d.org/2012/12/gocos2d-type-heirachy.html

Now, you want to know why those packages are crap compared to mine? The people who wrote them didnt take the time to study the problem thoroughly enough to understand the best way to solve them. I can tell everyone here exactly why SDL depends on executing on the mainthread for OS X/Windows, and I can also explain why desktop GL contexts depend on a single thread. I am working on a solution now and my next commit will have a work egles solution for Xorg on linux. What I cant do is test, windows and linux. That is why I asked for help. Criticize me all you want, but I take great offense to your accusations considering I work uncompensated towards Go's usefulness full time. I am also always helping new gophers in #go-nuts.

Now if you want to keep me banned, then fine. I stated my case so I can let it go. (pun intended)  

[Jared Anderson]

Russ

--

That do function is a nice little trick.  I wanted to avoid going this route but it does seem like a usability win.

For those interested in go-sdl2, my plan moving forward is to get the last few missing features added and get the API stable.  It will remain a light weight wrapper, so it will be up to the user to call runtime.LockOSThread.  After that I think I will start work on a package built on top of go-sdl2 that tries to make it goroutine friendly.

[Mortdeus]

It completely defeats the purpose of goroutines to implement a split stack friendly abstraction layer ontop of SDL. This is because SDL enables hardware acceleration using opengl. You would have to implement a RPC like opengl scheduler using a buffered channel to communicate with the gl context that is locked to the main os thread to get the best performance when you scale into the hundreds of gl calls that can be made / second, which is still a severe bottle neck.

Not only that but goroutines are highly unpredictable in regards to which opengl commands will be sent into the scheduler's channel when using a channel like this. If you directly call the SDL cgo functions from Goroutines, the context switching is going to completely defeat the purpose of using hardware acceleration in the first place.

Like I said earlier, EGL is the ONLY way forward for Go because it was specifically designed to be a thread safe embedded alternative to desktop opengl. Trying to use go concurrency with desktop opengl expecting to get quality performance is like trying to build a monolithic kernel with python hoping it will replace linux one day.

Keep dreaming...  

[Mortdeus]

I dont mean to discourage your project or ambition by saying this. Its just everyone is working on GL related projects to use with go and there is absolutely nothing anybody can do to address this issue of goroutines and opengl without implementing the EGL interface and GLES apis. The power of goroutines for 3d graphics is far too valuable in composition imho to even consider sticking with a strictly sequential programming model.   

[Jared Anderson]

--
 
 

Can you please stop assuming everyone has the same requirements you do?  For my needs a layer on top of SDL2/go-sdl2 will work perfectly fine.  Also I never said anything about making GL goroutine friendly, just SDL2.  Simply giving the user a way to do batch calls to SDL2's renderer API should be enough to accomplish this goal without suffering massive slowdowns due to context switching.  Yes this will not work for everyone and that is why I am going to do it as a layer on top of go-sdl2.

In short not everyone looking to use SDL2 is looking to make high performance 3D graphics.  If you are, I agree that it is a poor choice if you want to take advantage of goroutines.

[André Moraes]

On Sat, Dec 1, 2012 at 6:37 PM, Mortdeus <mort...@gocos2d.org> wrote:
> Dont worry people, im working on a goroutine friendly solution for all your
> cross platform gl woes. As ive been saying for a long time in #go-nuts; EGL
> is the way FORWARD for golang. Stop working on crap desktop GL related
> projects. For example, http://go-gl.github.com/. This project will only
> mislead people into making false assumptions regarding go shortcomings that
> do not even exist because it presents itself like its the "official" gl
> solution for golang.

I could be wrong, that's why I am asking instead of making statements:

The go-gl/gl package and go-gl/glh package work with OpenGL and not
directly with the window system. In this case the only difference I
see is that your project work's with OpenGL ES. Does OpenGL ES is that
much different from OpenGL?

The go-gl/glfw is a wrapper around the window system (render surface)
and also around the input handling (mouse/kbd/joystick). The EGL, from
what I know, only solve the render surface problem, is this right?

Another point, this one isn't a question.

Most render engines, use one thread to do the rendering and, at least,
another one to do the calculations (those that can't be moved to
shader pipeline). In that case the main-thread limitation isn't enough
to call the software "crap".