[ANN] Write Android apps in Go

[Elias Naur]

Hi,

Continuing where my shared library proposal left, I managed to add support to the go tools and runtime for creating go shared libraries suitable for loading from an Android app with System.loadLibrary. I've uploaded the required patches along with a simple demo here:

https://github.com/eliasnaur/goandroid

If everything goes well, you'll end up with a simple coloranimated triangle that you can drag around the screen.

Other than the CLs mentioned in my proposal, two Android specific patches are included in the goandroid repo. One patch works around a TLS limitation in the Android linker, and the other disables various features not available in the Android libc (bionic).

 - elias

[Stephen Gutekanst]

Very nice work!

I am excited to take a look at this more in depth once I have time.

Stephen

[Max]

Great. But what applications for go in android? 

Network scanner that pings thousands of hosts from android could be one.

What else requires such concurrency and will run on android?

[Bob Hemington]

Any.

If you look at the example in the repo it uses JNI, which gives you access to all Java API's.. Go could be used to write any android application currently (or not) available right now on the Play Store.

Stephen

[Brad Fitzpatrick]

Nice!  :-)

 - elias

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[paulo....@gmail.com]

Nice work,

Even with the language design issues I voiced a few times here, I would actually
consider using Go to write Android applications, if it would be an official supported language.

This is why I created this issue,

http://code.google.com/p/android/issues/detail?id=39482

But from the looks of it, nothing much has happened and to be honest from this year's
Google IO talks, I got the idea that the NDK is a kind of stepchild, considering that the
new Game APIs are Java only and the emphasis on Renderscript.

--
Paulo

[Elias Naur]

As Bob pointed out, you can write more or less all apps in Go, given enough JNI glue. However, I see a particular good case for mobiles games in Go.

Games have the unusual property that they are loosely tied to the particular platform (basically only need OpenGL for graphics and access to input) so a large portion of a game code base can be made portable. However, the current highest level common language between Android and iOS is C/C++, but because Go output native executables/libraries I see a great opportunity to make it a great crossplatform language to write games in.

 - elias

[Elias Naur]

On Sunday, June 30, 2013 10:09:48 AM UTC+2, paulo....@gmail.com wrote:

Nice work,

Even with the language design issues I voiced a few times here, I would actually
consider using Go to write Android applications, if it would be an official supported language.

This is why I created this issue,

http://code.google.com/p/android/issues/detail?id=39482

But from the looks of it, nothing much has happened and to be honest from this year's
Google IO talks, I got the idea that the NDK is a kind of stepchild, considering that the
new Game APIs are Java only and the emphasis on Renderscript.

Besides, Google has to keep support for Java/Dalvik/NDK basically forever, so adding yet another language is a big decision not taken lightly. Make great Go apps/games and help influence the decision. Sometimes we have to show the way forward instead of waiting for the big guys :)

 - elias

[Max]

Go has async file io. I think OpenGL ES is single threaded. In case of Go I guess OpenGL ES wrapper will work with single go routine.

        glGenRenderbuffers(1, &_msaaColorBuffer);

        glBindRenderbuffer(GL_RENDERBUFFER, _msaaColorBuffer);

        glRenderbufferStorageMultisampleAPPLE(GL_RENDERBUFFER, 4, GL_RGBA4, width, height);

        glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, _msaaColorBuffer);

        

        // Multisample Depth Render Buffer

        glGenRenderbuffers(1, &_msaaDepthBuffer);

        glBindRenderbuffer(GL_RENDERBUFFER, _msaaDepthBuffer);

        glRenderbufferStorageMultisampleAPPLE(GL_RENDERBUFFER, 4, GL_DEPTH_COMPONENT16, width, height);

        glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, _msaaDepthBuffer);

OpenGL ES can't be called concurrently as there is single context with state.

I think it is senseless to use Go without its strengths. 

"Go for games" requires special async graphics API that will load graphics recourses asynchronously.

I am sure there is lack of such API

Modern phones share video and main memory. So lacking features are limitations of Open GLES API. 

In theory CPUs and GPU may blend graphics streams (textures, vertexes) in much more efficient way.

[Nigel Tao]

On Mon, Jul 1, 2013 at 7:15 AM, Max <max.se...@gmail.com> wrote:
> I think it is senseless to use Go without its strengths.

Go's strengths aren't just goroutines. There's at least the built-in
slice and map types, unsigned integer types, light-weight interfaces
and standard library. And even if you can't do graphics from multiple
goroutines, you can still do AI, simulation, image decoding, and other
computation.

[Stephen Gutekanst]

Hi,

I'd like to add that this is also not an problem with OpenGL itself. Even if OpenGL API's supported being called from multiple threads, graphics drivers would just have to serialize the calls into uniform ones order to execute them anyway, so there would be little to none performance benefit. Even DirectX is not an multi-threaded rendering API.

Also, most (all?) games on Android and the iPhone are written using OpenGL ES, some Android ones are even purely using Java, which Go can surely meet or exceed performance wise.

Also note that while an single OpenGL context may only exist in one thread, multiple threads in an application can use different OpenGL contexts at once, and they may share memory and resources (Google: OpenGL shared contexts). The driver will still serialize render calls, obviously, so there is no multi-threaded rendering performance benifit. But with this you can upload textures or models asynchronously.

Go is an perfect candidate for making great interactive games and applications using OpenGL.

Stephen

[Elias Naur]

Hi,

I've updated the Go patches included in goandroid, so you should now be able to build Go Android apps on darwin too.

 - elias

[Alexei Sholik]

Sufficiently graphically advanced games require deterministic frame performance. With Go on Android, I see this requirement challenged twice: by unrestricted background apps and non-deterministic GC pauses. Any ideas on how those issues could be mitigated, if only partially?

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Best regards
Alexei Sholik

[Stephen Gutekanst]

Hi,

My main usage of Go has been creating games. While I cannot say for certain that the GC will be an problem for me; thus far in my development I have seen no issue using Go as-is for game development.

I do, however, have an backup plan, in the event that the GC does impose noticeable pauses (which I suppose could be more obvious on slower mobile devices).

With cgo you can make calls to malloc, free, calloc, etc. this will give you manual memory management for the portions of your game where you need it (I.e. those portions of code who impose noticeable GC pauses).

That being said; maybe you should try it before you knock it? I haven't had any issues.

Stephen

[Elias Naur]

On Friday, July 5, 2013 6:29:32 PM UTC+2, alco wrote:

Sufficiently graphically advanced games require deterministic frame performance. With Go on Android, I see this requirement challenged twice: by unrestricted background apps and non-deterministic GC pauses. Any ideas on how those issues could be mitigated, if only partially?

Unrestricted background apps is not a Go specific problem and doesn't seem to get in the way for other Android games. As for the GC pauses, minimize or eliminate per-frame garbage after level load is key. No garbage, no GC.

 - elias

[André Moraes]

Congrats!

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--

André Moraes
http://amoraes.info

[Kevin Gillette]

On Friday, July 5, 2013 11:12:25 AM UTC-6, Elias Naur wrote:

As for the GC pauses, minimize or eliminate per-frame garbage after level load is key. No garbage, no GC.

Precisely, and at the end of level load you can call runtime.GC to force any garbage collection prior to gameplay, and if need be call SetGCPercent from runtime/debug to tune the collection to your needs. Besides which, it's entirely feasible to design a game engine that produces no garbage.

[mortdeus]

Sweet. Ill experiment with trying to get egles to work with this. To everyone saying that egl cant be used in a multithreaded fashion, that isnt entirely true. Section 2.5 in the egl spec discusses multithreading. While there are challenges associated with getting egl to work with goroutines, its definitely possible. We may end up having to use something like the chromium model to implement thread safety. (not the browser, google it).  

[Elias Naur]

Nice. If you do this, take a look at NativeActivity (but skip android_native_app_glue which is a C/C++ centric library on top of NA) if you haven't done so already. It's a bit more work to implement, but in return you get less Java code and direct access to EGL and with LockOSThread/UnlockOSThread you can completely control the render loop much like on other platforms. The goandroid sample use GLSurfaceView only for simplicity since it is only a demo of Android/Go interaction. In fact, I already have NativeActivity working with my own ad-hoc gl es wrapper, and would consider adding a sample to goandroid if there's enough interest.

 - elias

[Alexei Sholik]

On Friday, July 5, 2013 11:12:25 AM UTC-6, Elias Naur wrote:

As for the GC pauses, minimize or eliminate per-frame garbage after level load is key. No garbage, no GC.

Precisely, and at the end of level load you can call runtime.GC to force any garbage collection prior to gameplay, and if need be call SetGCPercent from runtime/debug to tune the collection to your needs. Besides which, it's entirely feasible to design a game engine that produces no garbage.

This is only theoretically feasible. In practice, preallocating all your buffers might not be feasible, especially on mobile. Besides, in a garbage-collected language, the standard library itself relies on garbage collection. So to be sure you're not generating excessive garbage, you'd need to either inspect the code of the libraries you're using or not use them at all (which would defeat the point of using Go in the first place).

Say, you want to display frames per second on each frame. Do you use a library function to convert the number to string or do manual digit juggling to save yourself an allocation? This reminds me of a chapter in Real World Haskell[1] in which they show how to optimize a simple piece of code. It turns out you can write fast programs in Haskell, but it won't be idiomatic Haskell (definitely not like that "quicksort" you've seen) and you'd need to spend time profiling many parts of your program or adopting a very different coding style from the start, to not end up with unrunnable code after a month's time (remember, I'm talking in the context of mobile; every CPU cycle and memory byte is precious there).

Of course, this whole issue is theoretical. Go would work for 90% of the games out there. Besides, you could take an engine in C++ and write a wrapper for it so that only game logic would be written in Go.

My point is that for any language there is a certain cost you pay in terms of performance overhead vs high-levelness. I'd love to see Go outrunning Java on Android! So good luck!

[1]: http://book.realworldhaskell.org/read/profiling-and-optimization.html

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Best regards
Alexei Sholik

[paulo....@gmail.com]

The funny thing about this discussion is how it brings me back memories how
I used to write Assembly, because Turbo Pascal, C and C++ were too slow and
produced horrible big slow executables.

[Kevin Gillette]

On Saturday, July 6, 2013 4:36:40 AM UTC-6, alco wrote:

Besides, in a garbage-collected language, the standard library itself relies on garbage collection.

The Go standard library is actually fairly practical about this, and tries to be garbage-free wherever practical to do so. Many other garbage-collected languages are ref-counted, using garbage collection for cycles, and those languages' stdlibs may rely on automatic collection much more than Go's since they're already paying at least an additional per-frame-unwind cost (thus overuse of strings and such are free).

 

So to be sure you're not generating excessive garbage, you'd need to either inspect the code of the libraries you're using or not use them at all (which would defeat the point of using Go in the first place).

That's reasonable anyway for both general familiarity and peer review. Profiling is critical for mobile development anyway, and if you avoid premature optimization *and* profile, then you can put the minimum required effort into making the software reliably usable within the given operational constraints.

 

Say, you want to display frames per second on each frame. Do you use a library function to convert the number to string or do manual digit juggling to save yourself an allocation?

I know you're using this as a theoretical example, though there is strconv.AppendInt, which doesn't generate it's own garbage. That said, manual digit juggling, especially if you're limiting yourself to decimal, is not all that troublesome -- I've done it numerous times in a dozen or so lines of assembly, so it'd take at most that many lines of Go code. There would of course be other cases in which the already-available approach may not be ideal, though Java and C++ also have weak spots in this category too.

[Elias Naur]

I've now added the "nativeactivity" sample to demonstrate the NativeActivity API through Go. The sample contains no Java code, and allow full control of the EGL context creation and input/render loop from Go.

 - elias

[Anssi Porttikivi]

In game development I see threading as an answer to flexible background loading of all kinds of resources, to shorten load times, to eliminate a synchronous "load" experience from the user.