naclports doesn't build cleanly

[C. Scott Ananian]

Just checked out naclports:

  cd ~/Projects/NaCl

  mkdir naclports ; cd naclports

  gclient config http://naclports.googlecode.com/svn/trunk/src

  gclient sync

  cd src/packages

  export NACL_SDK_ROOT=~/3rdParty/native_client_sdk_0_1_721_0

  ./nacl-install-all.sh 

and the build "failed":

------

  ...

  a - f77_wrap4.o

make[1]: Leaving directory `/home/cananian/Projects/NaCl/naclports/src/packages/repository/cfitsio/cfitsio-build'

naclports nacl-install-all: Install SUCCEEDED cfitsio (32)

@@@BUILD_STEP 32-bit Summary@@@

@@@STEP_FAILURE@@@

naclports nacl-install-all: Install FAILED for ImageMagick-6.5.4-10 (32)

Error building for 32-bits.

-------

This failure looks very much like success.  Anyone have some insight?

(I might suggest that a "how to build" page might be added to the naclports wiki, or that a top-level naclports/README file be added.  But I'm pretty sure I'm doing the right thing.)

  --scott

--
                         ( http://cscott.net/ )

[Bradley Nelson]

Hi Scott,

Yeah naclports needs some work.

We track it on a continuous build here: http://build.chromium.org/p/client.nacl.ports/waterfall

And well, it's been broken for a good bit. More on some platforms than others.

The success, followed by failure thing is an artifact of our build waterfall.

It tries to build everything and then reports failure at the end.

On the link above it gets chopped up into nice little boxes.

I'll defer commentary on when we'll get it sorted out to others.

-BradN

--
You received this message because you are subscribed to the Google Groups "Native-Client-Discuss" group.
To post to this group, send email to native-cli...@googlegroups.com.
To unsubscribe from this group, send email to native-client-di...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/native-client-discuss?hl=en.

[Nicholas Fullagar]

Hi Scott,

It has probably built most of the libraries for 32-bit NaCl, look in your

~/3rdParty/native_client_sdk_0_1_721_0/src/toolchain/linux_x86/nacl/usr/include
~/3rdParty/native_client_sdk_0_1_721_0/src/toolchain/linux_x86/nacl/usr/lib

(replace linux_x86 with your platform)

The nacl-install-all.sh script adds these directories to nacl-gcc / nacl-g++ search paths, so using #include <zlib.h> in your source and -lz at link time should work.

If you need to generate 64-bit libraries, normally nacl-install-all.sh would do this, but since it is bailing early, you can explicitly build just the 64-bit libraries with

./nacl-install-all-bitsize.sh 64

If you blow away or update to a different SDK, you'll need to run the install scripts again.

On Tue, Apr 12, 2011 at 3:50 PM, C. Scott Ananian <csc...@laptop.org> wrote:

--

[C. Scott Ananian]

On Tue, Apr 12, 2011 at 7:08 PM, Bradley Nelson <bradn...@google.com> wrote:
> Hi Scott,
> Yeah naclports needs some work.
> We track it on a continuous build
> here: http://build.chromium.org/p/client.nacl.ports/waterfall
> And well, it's been broken for a good bit. More on some platforms than
> others.
> The success, followed by failure thing is an artifact of our build
> waterfall.
> It tries to build everything and then reports failure at the end.
> On the link above it gets chopped up into nice little boxes.
> I'll defer commentary on when we'll get it sorted out to others.

I'm willing to put in some work to fix it. Any initial advice?

[I've got ambitious goals for nativeclient (
http://cananian.livejournal.com/62667.html ) so a robust ports system
is definitely in my best interest.]
--scott

[C. Scott Ananian]

Here's a starter question: can someone explain how
http://code.google.com/p/naclports/source/browse/trunk/src/packages/URLS_TO_MIRROR
is supposed to work?

If I'm adding a package to naclports, I assume I'd add the actual
upstream URL to the package script for now, but it looks like all of
the existing package have had their tarballs mirrored at
http://commondatastorage.googleapis.com/nativeclient-mirror/nacl/ URLs
via some magic mechanism.
--scott

[Bradley Nelson]

Hi Scott,

The 'magic' mechanism is by hand :-)

The intention was to document all the upstream locations needed. We mirror, as historically they've been flaky / disappear.

We can add to that set, and going forward, obviously something more general and robust would be good.

-BradN

[David Springer]

You have correctly discovered that this is a magic mechanism.  The idea here is to take a snapshot of source repos that then get patched.  This is not necessary, since you can do an svn co of the repo as a build step in naclports.  If you have an account on Google's data storage service, you can add tarballs to that, then access them from your scripts.

 --scott

[C. Scott Ananian]

OK, well then I'll use the upstream URLs in my ports, and one of you
googlers can do the magic mirror. ;-)

Hopefully I'll get some patches to you tomorrow-ish -- first to fix
ImageMagick (I'm not seeing the SDL failure on my machine; it looks
like your buildbot is missing autoconf?), and then probably I'll
tackle pango next, via:
libiconv.a
libgettextpo.a
libglib-2.0.a
libpixman-1.a (already done!)
libfreetype.a (already done!)
libcairo.a (already done!)
libxml2.a (looks like you're using tinyxml instead for fontconfig?)
libfontconfig.a (already done!)
libpango-1.0.a

Ultimately I'm probably going to shoot for integrating nacl and
gobject-introspection to do a proof-of-concept where javascript calls
into cairo/pango via g-o-i to render into a canvas.
--scott

[Evgeniy Stepanov]

Hi Scott,

we've got a partially-working port of pango + glib outside of naclports:

https://github.com/eugenis/pango-nacl

https://github.com/eugenis/glib-nacl

We will put it in naclports eventually, but for now it is much easier to do incremental development in a git repo then in a tarball+patch format. Our port is capable of running pango-view and getting a reasonable-looking image out of it; the fonts look terrible though, there is something wrong with fontconfig setup.

This is a part of our (me and Evgeny Eltsin) effort to port Wesnoth to NaCl, and we have a couple of other libs ported as well. You can find the full list at

https://github.com/eugenis/wesnoth-nacl-build/blob/master/howto

It is worth to note that we are building all this stuff with the GLibC-based toolchain. Porting large libraries to newlib is a lot of extra work.

 --scott

[C. Scott Ananian]

On Wed, Apr 13, 2011 at 9:15 AM, Evgeniy Stepanov <eug...@google.com> wrote:
> https://github.com/eugenis/wesnoth-nacl-build/blob/master/howto
> It is worth to note that we are building all this stuff with the GLibC-based
> toolchain. Porting large libraries to newlib is a lot of extra work.

Could you explain this further? I dug through your git repository and
I couldn't find anything relating to glibc.
--scott

[Evgeniy Stepanov]

You can grab a toolchain from the buildbot at

http://build.chromium.org/p/client.nacl.toolchain/waterfall?builder=lucid64-glibc

(look for the "archive build" step).

It is based on GLibC instead of newlib and supports dynamic linking. Binaries built with this toolchain do not work in the browser at the moment, but it seems like we are very close to this.

All build scripts in my repo's are expecting a GLibC toolchain. Porting stuff to newlib will require more work and I have no plans of doing that.

[C. Scott Ananian]

On Wed, Apr 13, 2011 at 10:32 AM, Evgeniy Stepanov <eug...@google.com> wrote:
> You can grab a toolchain from the buildbot at
> http://build.chromium.org/p/client.nacl.toolchain/waterfall?builder=lucid64-glibc
> (look for the "archive build" step).
>
> It is based on GLibC instead of newlib and supports dynamic linking.
> Binaries built with this toolchain do not work in the browser at the moment,
> but it seems like we are very close to this.
> All build scripts in my repo's are expecting a GLibC toolchain. Porting
> stuff to newlib will require more work and I have no plans of doing that.

Can the google folks comment as to whether the glibc toolchain is
expected to be the default in the future? Will patches to naclports
to make them work with glibc (probably removing unnecessary newlib
workarounds) be accepted?

Related question: the wiki notes that open(2) is not supported for
security reasons. Has this been reconsidered? It would make porting
much easier if open were changed to do something sandbox-safe, rather
than completely omitted from the API. For example, it could interact
with HTML5 web storage, or load HTTP resources via relative paths.

Many legacy bits of code (like fontconfig) expect to be able to
read/parse configuration files. I haven't dug into the code yet to
find out what the current "best practice" is for supporting this.
--scott

[Victor Khimenko]

On Wed, Apr 13, 2011 at 7:32 PM, C. Scott Ananian <csc...@laptop.org> wrote:

Can the google folks comment as to whether the glibc toolchain is

expected to be the default in the future?

No comments. We DO want to support glibc, but we are not sure if we'll be ready to drop newlib any time soon.

 

Will patches to naclports to make them work with glibc (probably removing unnecessary newlib workarounds) be accepted?

Situation with naclports is very much undecided at this stage. You see message "needs lots of love" for a reason: we don't yet have a solid long-term plan for naclports.

 

Related question: the wiki notes that open(2) is not supported for security reasons.

Right.

 

Has this been reconsidered?  It would make porting much easier if open were changed to do something sandbox-safe, rather
than completely omitted from the API.

Right. But this requires some kind of temporary filesystem in browser. This is VERY common request so we are pretty sure eventually some kind of solution will be found. Note that with glibc you can just provide open(...) function in your program and it'll replace default implementation so you can cook up some kind of emulation using WebSQL or whatever without any changes on NaCl side.

 

For example, it could interact with HTML5 web storage, or load HTTP resources via relative paths.

Contributions are welcome and as I've said with glibc it can be done purely in untrusted code. With newlib it can be done too with some linker magic: http://sourceware.org/binutils/docs-2.21/ld/Options.html#index-g_t_002d_002dwrap_003d_0040var_007bsymbol_007d-261

 

Many legacy bits of code (like fontconfig) expect to be able to
read/parse configuration files.  I haven't dug into the code yet to
find out what the current "best practice" is for supporting this.

"Current best practice" is --wrap open. Take a look on nethack for example.

[Nicholas Fullagar]

The latest example for open, read, lseek, close etc. is in native_client/tests/ppapi_geturl/nacl_file_main.cc. This is experimental.

[C. Scott Ananian]

On Wed, Apr 13, 2011 at 2:14 PM, Victor Khimenko <kh...@google.com> wrote:
>
> On Wed, Apr 13, 2011 at 7:32 PM, C. Scott Ananian <csc...@laptop.org> wrote:
>>
>> Can the google folks comment as to whether the glibc toolchain is
>> expected to be the default in the future?
>
> No comments. We DO want to support glibc, but we are not sure if we'll be
> ready to drop newlib any time soon.

OK. I understand there are licensing issues. Luckily, autoconf and
friends were designed to handle library differences like this, so it
shouldn't be too much trouble to support both. But it might be nice
if the target triple was different for the two toolchains, and/or if
naclports could more clearly distinguish between newlib patches, glibc
patches, and/or nacl patches. (Ultimately I would hope that the
changes to autoconf to support the nacl target could go upstream, and
then the various bits of naclports could also be upstreamed.)

Mark Seaborn has written about mechanisms to stub out unsupported bits of glibc:
http://plash.beasts.org/wiki/NativeClient/Changes
Is this the approach that has been taken with the Google glibc toolchain?

He also mentioned mechanisms to run nativeclient-compiled code
directly, which obviously helps cross-compilation a lot. I couldn't
tell if this has been upstreamed, or is glibc-specific. I'll be
looking at porting gobject-introspection, which unfortunately does a
lot of unkosher execution and introspection of target-compiled code,
so the ability to execute nacl binaries during the build process could
definitely push me towards one toolchain or the other.

> "Current best practice" is --wrap open. Take a look on nethack for example.

Thanks for the pointers. You might also consider creating a
'naclsupport' library in naclports which contains some good wrappers,
so that clients didn't have to keep reinventing them.
--scott

[Victor Khimenko]

On Wed, Apr 13, 2011 at 11:00 PM, C. Scott Ananian <csc...@laptop.org> wrote:

On Wed, Apr 13, 2011 at 2:14 PM, Victor Khimenko <kh...@google.com> wrote:
>
> On Wed, Apr 13, 2011 at 7:32 PM, C. Scott Ananian <csc...@laptop.org> wrote:
>>
>> Can the google folks comment as to whether the glibc toolchain is
>> expected to be the default in the future?
>
> No comments. We DO want to support glibc, but we are not sure if we'll be
> ready to drop newlib any time soon.

OK.  I understand there are licensing issues.  Luckily, autoconf and friends were designed to handle library differences like this, so it shouldn't be too much trouble to support both.

Yup.

 

But it might be nice if the target triple was different for the two toolchains, and/or if naclports could more clearly distinguish between newlib patches, glibc patches, and/or nacl patches.

This is what we still need to decide.

 

 (Ultimately I would hope that the changes to autoconf to support the nacl target could go upstream, and then the various bits of naclports could also be upstreamed.)

Support for nacl is in upstream autoconf for a year or two (it only needed one-liner in config.guess, actually). Individual packages are different story.

 

Mark Seaborn has written about mechanisms to stub out unsupported bits of glibc:
 http://plash.beasts.org/wiki/NativeClient/Changes
Is this the approach that has been taken with the Google glibc toolchain?

Well, it's kinda hard to compare Mark's port and Google's port if you'll recall that Mark joined Google and participated in Google's port, but well, they were done in slightly different way. Mark's approach relied on NX bit and was non-crossplatform so it was not really usable without changes. We were forced to introduce significantly more intrusive changes to make it usable on Windows and MacOS so currently it's not possible to run this port natively.

 

He also mentioned mechanisms to run nativeclient-compiled code directly, which obviously helps cross-compilation a lot.  I couldn't tell if this has been upstreamed, or is glibc-specific.  

It was abandoned. It was just too hard to support both NaCl-way and linux-way in a single libc. But, more importantly, x86-64 NaCl is different enough that it's impossible to run it's code directly.

 

I'll be looking at porting gobject-introspection, which unfortunately does a lot of unkosher execution and introspection of target-compiled code, so the ability to execute nacl binaries during the build process could
definitely push me towards one toolchain or the other.

They are identical in this regard: both don't support native execution, both can fake it using fake-ld script:

http://code.google.com/p/naclports/source/browse/trunk/src/packages/scripts/ld

But this approach creates problem with Windows (where version of NaCl is tied to version of OS: only i386 NaCl on 32bit Windows and only x86-64 NaCl on 64bit Windows) and is totally unsuitable for Mac (where you can build 64bit binaries but can not run them).

> "Current best practice" is --wrap open. Take a look on nethack for example.

Thanks for the pointers.  You might also consider creating a
'naclsupport' library in naclports which contains some good wrappers, so that clients didn't have to keep reinventing them.

Yeah, that too. And I don't know how many times fake-ld script was independently invented :-)

[C. Scott Ananian]

On Wed, Apr 13, 2011 at 4:58 PM, Victor Khimenko <kh...@google.com> wrote:
>>  (Ultimately I would hope that the changes to autoconf to support the nacl
>> target could go upstream, and then the various bits of naclports could also
>> be upstreamed.)
>>
> Support for nacl is in upstream autoconf for a year or two (it only needed
> one-liner in config.guess, actually). Individual packages are different
> story.

Would you accept a patch to naclports which added autoconf? Rather
than bloat the patches with autoconfiscated files, it would be much
more convenient just to generate/regenerate all the autoconf stuff
with a known-good version of autoconf which supported nacl.
--scott

[David Springer]

Yes.

 

 --scott

[Victor Khimenko]

In my experience the best way to create A LOT OF problems for autoconf-based package is to regenerate autoconf files using latest-and-greatest autoconf. When you apply 5-lines patch you know exactly what you are doing, when you call autoreconf - you are playing Russian roulette. If we'll go this way we'll need to keep around dozen of versions of autotools - I'm not sure it's good enough replacement for the "huge patch" like this one:

http://code.google.com/p/naclports/source/browse/trunk/src/packages/scripts/gsl-1.9/nacl-gsl-1.9.patch

--- gsl-1.9/config.sub	2006-03-21 10:21:43.000000000 -0500
+++ gsl-1.9.new/config.sub	2008-12-04 20:21:54.157675000 -0500
@@ -715,6 +715,10 @@
 		basic_machine=i370-ibm
 		os=-mvs
 		;;
+	nacl)
+		basic_machine=i686-pc
+		os=-nacl
+		;;
 	ncr3000)
 		basic_machine=i486-ncr
 		os=-sysv4
@@ -1208,7 +1212,7 @@
 	      | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \
 	      | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \
 	      | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \
-	      | -skyos* | -haiku* | -rdos*)
+	      | -skyos* | -haiku* | -rdos* | -nacl*)
 	# Remember, each alternative MUST END IN *, to match a version number.
 		;;
 	-qnx*)

[C. Scott Ananian]

Some of the upstreams don't ship autoconf-generated files. For those
the naclports patches *are* huge, and are including all the generated
stuff anyway, and it's become very hard to see that the "real" patch
is just to the 3 or 4 lines which add nacl.

Don't worry, I'll be sensitive, I won't regenerate autoconf stuff if
it's unnecessary or if the original stuff required some oddball
autoconf version. But for *some* ports, I think generating autoconf
stuff with a known-good autoconf version will lead to more concise
build scripts. I think there's a reasonable solution that doesn't
involve sticking a dozen different autoconf versions in naclports.
--scott

[Victor Khimenko]

On Thu, Apr 14, 2011 at 2:28 AM, C. Scott Ananian <csc...@laptop.org> wrote:

Some of the upstreams don't ship autoconf-generated files.  For those
the naclports patches *are* huge, and are including all the generated
stuff anyway, and it's become very hard to see that the "real" patch
is just to the 3 or 4 lines which add nacl.

This is mostly done to handle Windows case where we have no way to run binaries (32bit/64bit OS issue) and no native compiler (only nacl-gcc is provided by SDK, "native" gcc is not included). It worked for some time but right now a lot of packages are broken on Windows anyway so I'm not sure it's big deal now.

Don't worry, I'll be sensitive, I won't regenerate autoconf stuff if
it's unnecessary or if the original stuff required some oddball
autoconf version.   But for *some* ports, I think generating autoconf
stuff with a known-good autoconf version will lead to more concise
build scripts.  I think there's a reasonable solution that doesn't
involve sticking a dozen different autoconf versions in naclports.

We can try to do that - it's always possible to rollback. I'm just not too sure if it'll be more stable or not. I've seen this story many, many times, it's always the same: people start with latest version of autoconf, then they add another one, then selector and in the end you have the whole autozoo in your package system. On the other hand autotools are not too large so perhaps it's not a big deal...

[Mark Seaborn]

On 13 April 2011 12:00, C. Scott Ananian <csc...@laptop.org> wrote:

Mark Seaborn has written about mechanisms to stub out unsupported bits of glibc:
 http://plash.beasts.org/wiki/NativeClient/Changes
Is this the approach that has been taken with the Google glibc toolchain?

He also mentioned mechanisms to run nativeclient-compiled code
directly, which obviously helps cross-compilation a lot.  I couldn't
tell if this has been upstreamed, or is glibc-specific.

[You're referring to the "ncrewrite" trick described in:
http://plash.beasts.org/wiki/NativeClient/Ncrewrite]

Yeah, this was a handy trick in 2009 when NaCl only supported x86-32.  However, I'm not pursuing this trick any more as a way for building NaCl programs.  There are two reasons why it does not work so well any more:

 1) The trick doesn't apply to x86-64.  Sandboxed x86-64 NaCl code is not ABI-compatible with native Linux code.  (Having said that, the x32 ABI being implemented for binutils/gcc/glibc by H J Lu could be compatible with NaCl sandboxing.)

 2) The trick doesn't work with the Big Segment Gap scheme that we're now using for dynamic linking under NaCl.  In order to segregate code and data in NaCl's address space, we link NaCl dynamic ELF objects with a big gap between code and data segments (see http://code.google.com/p/nativeclient/wiki/DynamicLoadingOptions).  But the native Linux glibc does not know how to load these ELF objects without running out of address space.

I'll be looking at porting gobject-introspection, which unfortunately does a
lot of unkosher execution and introspection of target-compiled code,
so the ability to execute nacl binaries during the build process could
definitely push me towards one toolchain or the other.

My plan for supporting build systems that run target-compiled code is to support more Unix interfaces (especially filesystem interfaces) under NaCl, along the lines of these issues:

http://code.google.com/p/nativeclient/issues/detail?id=634
http://code.google.com/p/nativeclient/issues/detail?id=1229
http://code.google.com/p/nativeclient/issues/detail?id=1239

There are some steps towards this in native_client/src/tools/posix_over_imc.  This currently only supports open(), but I need at least getcwd() to support Python's build process.

Cheers,

Mark

[C. Scott Ananian]

On Wed, Apr 13, 2011 at 6:49 PM, Mark Seaborn <msea...@chromium.org> wrote:
> My plan for supporting build systems that run target-compiled code is to
> support more Unix interfaces (especially filesystem interfaces) under NaCl,
> along the lines of these issues:
>
> http://code.google.com/p/nativeclient/issues/detail?id=634
> http://code.google.com/p/nativeclient/issues/detail?id=1229
> http://code.google.com/p/nativeclient/issues/detail?id=1239
>
> There are some steps towards this in
> native_client/src/tools/posix_over_imc.  This currently only supports
> open(), but I need at least getcwd() to support Python's build process.

So is the idea here to actually invoke the NaCl-generated code in the
appropriate sandbox?
sel_ldr is the sandbox, right? So you'd be invoking the sandbox and
then implementing all the callouts with "real" UNIX implementations of
the syscalls? I'm just checking to see if I'm understanding this
correctly.
--scott

[Victor Khimenko]

On Thu, Apr 14, 2011 at 4:03 AM, C. Scott Ananian <csc...@laptop.org> wrote:

On Wed, Apr 13, 2011 at 6:49 PM, Mark Seaborn <msea...@chromium.org> wrote:
> My plan for supporting build systems that run target-compiled code is to
> support more Unix interfaces (especially filesystem interfaces) under NaCl,
> along the lines of these issues:
>
> http://code.google.com/p/nativeclient/issues/detail?id=634
> http://code.google.com/p/nativeclient/issues/detail?id=1229
> http://code.google.com/p/nativeclient/issues/detail?id=1239
>
> There are some steps towards this in
> native_client/src/tools/posix_over_imc.  This currently only supports
> open(), but I need at least getcwd() to support Python's build process.

So is the idea here to actually invoke the NaCl-generated code in the appropriate sandbox?

Yes.

 

sel_ldr is the sandbox, right?

sel_ldr is loader (as the name implies) and it includes sandbox (partially or fully). If the sandbox is disabled then program can call native syscalls if it wishes to do so (such program can not used in browser, obviously).

 

So you'd be invoking the sandbox and then implementing all the callouts with "real" UNIX implementations of
the syscalls?

Yeah, that's the plan. This way we can support all three platforms (Windows, Mac and Linux). Under Linux you can call syscalls directly (if you disable sandbox, obviously) but on Windows our security scheme makes it harder is not outright impossible.