[ANN] Darwin/ARM (aka. iOS) port of Go is READY

[minux]

Hi all,

     I'm glad to announce that the Darwin/ARM port has just passed *ALL* tests. It's ready to be
merged. I'm here to welcome anyone interested to test/use it.

     I've got an iOS toolchain on Linux built, so you don't have to build it on iDevice or Mac OS X
any more. You can build the full set of packages and Go compiler for iOS on your Linux box.
Only testing requires a jailbroken iDevice.

     Code is hosted at: https://bitbucket.org/minux/goios/ . You might want to read the wiki for building
and testing instructions.


Best regards,
minux

[Jan Mercl]

On Friday, January 6, 2012 10:05:34 PM UTC+1, minux wrote:

Hi all,

     I'm glad to announce that the Darwin/ARM port has just passed *ALL* tests. It's ready to be
merged.

I doubt it is advisable for Google to officially host code (within its own maintained project like Go) which resulting use is possible only on a jailbroken Apple product. Just a personal opinion, I'm not affiliated with Google (nor Apple) in any way and no, IANAL.

[robintim...@googlemail.com]

Jailbreaking isn't illegal

http://www.wired.com/threatlevel/2010/07/feds-ok-iphone-jailbreaking/

[Jan Mercl]

I'm aware jailbreaking is legal (under DMCA in US), that's not why I think what I wrote earlier, really.

[Russ Cox]

Let's keep the list for technical, not legal, discussion.

Can I run the tests using the iOS SDK instead of an
actual device?

Russ

[minux]

No.... AFAIK, no usable emulator exists for iOS. iEmu port do exists, but
no usable now.
Though you can build the entire suite (5g/5c/5l, all packages) for iOS on
Mac OS X. It has to be tested on real hardware now.

I currently are looking for ways to test this without iDevice. Because this
port doesn't use any unique features of iOS kernel, theoretically it's possible
to build a qemu-darwin-user to run the test suite. But it will require a lot
of work (more than porting Go IMO, also, it would be hard to verify the
correctness of the emulator).

Russ, is setting up a real (jailbroken) iDevice for testing this port prohibitive?

[Paul Borman]

Is it possible to also cross-compile for x86 iOS?  This would then potentially let you do an iOS test using the iOS emulator that is part of Xcode as a smoke test.

    -Paul

[minux]

On Tue, Jan 10, 2012 at 2:36 AM, Paul Borman <bor...@google.com> wrote:

Is it possible to also cross-compile for x86 iOS?  This would then potentially let you do an iOS test using the iOS emulator that is part of Xcode as a smoke test.

It might be possible. But I don't think it is necessary. Because the port is more about ARM than about Darwin.
(e.g., the syscall interface is almost the same for iOS and Mac OS X, the difference is mainly in the ARM
part.) Testing on x86 iOS won't test any ARM related stuff in this port.

Also, even testing is passed on x86 iOS emulator, we still can't be sure that it will work on real iOS.

[Russ Cox]

I just know that I have what appears for all the world to
be a software simulator of an iPad installed on my machine
as part of Xcode. That's probably an ARM simulator rather
than x86. It seems like it should be possible to use that
to test the Go binaries. How do people test binaries they
compile using Objective C?

Russ

[Andrew Gerrand]

IIUC the iphone simulator is not an ARM simulator like the one that comes with the Android sdk. It is merely an environment that runs the iOS userland and system services. All the binaries it runs are x86.

Andrew

[Paul Borman]

Andrew is correct.  You cross compile to x86.  In the past, and maybe still, if you don't pay Apple the $99 you can only produce x86 binaries for iOS.

    -Paul

[minux]

I'm wondering if the only obstacle for merging back is the lack of Darwin/ARM builder.
If it is, I think I could set up my iPad as a builder for this port.
(The Darwin/ARM port is pretty stable now, I sync daily with upstream repo and most
of the time, ./all.bash will pass all applicable tests. Refer: https://bitbucket.org/minux/goios/wiki/CompleteTestLog )

[Russ Cox]

Sorry, I meant to reply about this before. We have decided
not to include the darwin/arm code in the main tree until there
is a way to run it on an ordinary iDevice without any kind of
"jailbreaking". If the port advances to that point, please let
us know, but until then I'm afraid you'll need to maintain your
own copy. Of course, if you have bug fixes that are not specific
to darwin/arm, we're always happy to accept those, and thanks
for the many you've already sent in.

Russ

[Mikkel Krautz]

Here's a little status update on the darwin/arm port.
(And a picture! https://dl.dropbox.com/s/nrcadpfrl9yurw1/igopher_small.jpeg)

Minux and I have spent the last week trying to get the port in a shape
that allows us to run without jailbreaking.

The first hurdle was getting the Mach-O binaries produced by 5l to
code sign correctly. After some small rearrangements in 5l and working
around an issue with Apple's code sign utility and non-cgo binaries,
we were able to run a hello world program successfully. With one
caveat. iOS expects apps launched through the home screen to
initialize "correctly", as in set up a UI using UIKit, and what not
(we think!).

Turns out, however, that it's possible to run programs for a longer
time if they're run through a debugger. The easiest way we've found to
do this is using fruitstrap: https://github.com/ghughes/fruitstrap.
It's a small command line utility that uses Apple's private
MobileDevice.framework to install a binary on the device and launch it
through GDB.
(Sidenote: an open source implementation of many of the protocols used
by MobileDevice.framework is available through the libimobiledevice
project at http://www.libimobiledevice.org)

So far so good.

There was still the issue of iOS not allowing runtime code generation,
for use in closures (you can't mmap memory using PROT_EXEC -- it's
filtered out). We discussed a few different options for solving this
in a low-impact manner (we didn't want to change the compiler, at
least not to begin with, if we could come up with a quick workaround).

Here's what we ended up doing:

Instead of generating code in runtime.closure, we simply put a magic
number there, along with the closure function and its arguments.

When the program tries to jump to the chunk of memory allocated in
runtime.closure, it traps (because the page isn't executable). Then,
in the signal handler, we add a special case for SIGBUS: If
info->si_addr == r->pc, we use runtime.mlookup to check whether the
memory was allocated by us. If that's the case, we can safely check
whether the closure magic is there or not. If it is, we call
runtime.handclosure to push the arguments to the stack and change the
program counter to point to the closure fn. We also make sure the
closure fn returns to a special closret function that cleans up the
stack (gets rid of the closure args), and jumps back to where we
should be.

The code's available here:
https://bitbucket.org/mkrautz/goios/changeset/e718a5e86a1c

Here are some benchmarks of the approach (tested on an iPod Touch 3rd gen)
The tests are the ones from runtime/closure_test.go:

Codegen
-------

runtime_test.BenchmarkCallClosure 20000000 78.6 ns/op
runtime_test.BenchmarkCallClosure1 20000000 82.2 ns/op
runtime_test.BenchmarkCallClosure2 500000 4979 ns/op
runtime_test.BenchmarkCallClosure3 1000000 4797 ns/op
runtime_test.BenchmarkCallClosure4 1000000 4778 ns/op

Trap-based
----------

runtime_test.BenchmarkCallClosure 50000000 73.7 ns/op
runtime_test.BenchmarkCallClosure1 20000000 85.3 ns/op
runtime_test.BenchmarkCallClosure2 500000 4971 ns/op
runtime_test.BenchmarkCallClosure3 1000000 4771 ns/op
runtime_test.BenchmarkCallClosure4 1000000 4778 ns/op

benchcmp
--------

benchmark old ns/op new ns/op delta
runtime_test.BenchmarkCallClosure 78 73 -6.23%
runtime_test.BenchmarkCallClosure1 82 85 +3.77%
runtime_test.BenchmarkCallClosure2 4979 4971 -0.16%
runtime_test.BenchmarkCallClosure3 4797 4771 -0.54%
runtime_test.BenchmarkCallClosure4 4778 4778 +0.00%

I was pleased to see the results of the benchmarks, but I remain skeptical.
It seems to good to be true. :)

Since we can't, for now at least, run for more than 10 secs without
GDB on non-jailbroken devices, Minux is working on brining cgo on
linux/arm up to speed, and when that's done, the plan is to do the
same for darwin/arm. With some work, that should allow us to call
into the ObjC runtime, and be able to run GUI apps as well.

I suppose we also need a way to hook up gotest to be able to run tests
properly on non-jailbroken devices. For that we'd need a way to let
gotest execute the tests through an intermediary program that pushes
the binary up to the device and executes it, while redirecting the
program's stdout/stderr and return value.

Hopefully, we can now get this merged once Go 1's out the door.

Mikkel

[Andrew Gerrand]

Wow. Very cool!

Do Apple accept non-Objective C apps in the app store?

Andrew

[minux]

On Mon, Jan 30, 2012 at 7:03 AM, Andrew Gerrand <a...@golang.org> wrote:

Wow. Very cool!

Do Apple accept non-Objective C apps in the app store?

It seems Apple only accept programs compiled by its official iOS SDK. So Apps written
in Go might not be available in the near future.

[Mikkel Krautz]

There are examples of tools that allow you to write non-ObjC apps,
such as Unity (which uses Mono's ahead-of-time compiling feature to
output ARM assembly language, which is then compiled along with a
small shim of ObjC code to produce the final app), and Adobe AIR
(which I don't know how works).

The above tools don't, however, do their own runtime initialization
before hitting main(), and perhaps Apple will not like that.

I intend to find out once cgo for darwin/arm is in a better shape.

Mikkel

[Mikkel Krautz]

On Mon, Jan 30, 2012 at 2:39 PM, Andrea Fazzi <andrea...@alcacoop.it> wrote:
> 2012/1/28 Mikkel Krautz <mik...@krautz.dk>:

>
>> Since we can't, for now at least, run for more than 10 secs without
>> GDB on non-jailbroken devices, Minux is working on brining cgo on
>> linux/arm up to speed, and when that's done, the plan is to do the
>> same for darwin/arm.  With some work, that should allow us to call
>> into the ObjC runtime, and be able to run GUI apps as well.
>

> Am I missing something or there isn't support for cgo on linux/arm yet?
>
> Andrea

Minux is working on linux/arm cgo, and the work is progressing nicely.

Quoting Minux from another thread:
> Linux/ARM cgo is *almost* ready now.
> With CGO_ENABLED=1, the whole pkg tests and tests in /test
> directory has passed.
> But, misc/cgo/stdio and misc/cgo/test still can't passed. Other
> tests in misc/cgo has passed.
>
> PS: The misc/cgo/stdio build failure is a minor one, cgo failed to
> generate the right dynimports. I'm sure that once this is fixed,
> this test will pass. The misc/cgo/test test failure is a fatal one;
> It seems g/m is out of sync. I'm working on it now.

I believe he intends to fire off a CL soon.

Mikkel

[Andrea Fazzi]

2012/1/30 Mikkel Krautz <mik...@krautz.dk>:

This is a very cool news! Finally I can port my speccy emulator on ARM :)!

Thanks for all your hard work!

Andrea

[Andrea Fazzi]

2012/1/28 Mikkel Krautz <mik...@krautz.dk>:

> Since we can't, for now at least, run for more than 10 secs without
> GDB on non-jailbroken devices, Minux is working on brining cgo on
> linux/arm up to speed, and when that's done, the plan is to do the
> same for darwin/arm.  With some work, that should allow us to call
> into the ObjC runtime, and be able to run GUI apps as well.

Am I missing something or there isn't support for cgo on linux/arm yet?

Andrea

[mhh...@gmail.com]

Hi,

It look awesome!

I d like very much to give it a try, can you clarify the procedure please ?
I was tempted to run this https://bitbucket.org/minux/goios/wiki/Home
through vagrant and post the result. But seems outdated ?

[fastf...@gmail.com]

Hi Minx 

It is go 1.8 now. how to build arm/darwin on macos ?  Is the wiki still workable since go 1.5 has a great change .

How to go build with cgo arm/darwin static/dynamic c lib for ios and android ?

Yes I need c lib , not gomobile framwork/aar file.    

Could u give a tutorial ?

There is little doc on google. And this cross compile always fail for me for weeks.

在 2012年1月7日星期六 UTC+8上午5:05:34，minux写道：