x86_64-nacl-XXX executables will not work on 64-bit ubuntu unless ia32-libs is installed

[Damian Eads]

Hello,

I am trying to build some Native Client examples on an Amazon EC2
t1.micro instance and ran into an issue that I did not encounter on my
laptop. I was able to resolve it but I thought I would share my
solution in case anyone else runs into it. Until I diagnosed my
problem, I assumed the x86_64-nacl-XXX executables were 64-bit because
I ran "./naclsdk update" on a 64-bit machine. The executables are
actually 32-bit - the 64 in the filenames just indicate the
architecture of the nexe's generated for cross-compilation. Unless the
appropriate ELF reader is installed (package ia32-libs on Ubuntu
11.10), the NaCL tools won't run on 64-bit Linux (or at least they
didn't for me).

Here's some info about my EC2 instance:

ubuntu$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.10
Release: 11.10
Codename: oneiric
ubuntu$ uname -a
Linux domU-XX-XX-XX-XX-XX-XX 3.0.0-14-virtual #23-Ubuntu SMP Mon Nov
21 21:09:11 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

Now, onto the steps to reproduce the problem.

First, I installed necessary dependencies to build NaCL applications
from source (see https://developers.google.com/native-client/devguide/tutorial).

Second, I installed pepper_18 and then I went into the pepper_18
examples directory.

ubuntu$ cd nacl_sdk/pepper_18/examples/hello_world_newlib/

Then, I tried to build the "Hello World" applicaiton.

ubuntu$ make
/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/i686-
nacl-gcc -o hello_world_x86_32.nexe hello_world.c -m32 -O0 -g -pthread
-O0 -g -Wno-long-long -Wall -Wswitch-enum -Werror -pedantic -lppapi
/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/i686-
nacl-gcc: line 14: /home/ubuntu/nacl_sdk/pepper_18/toolchain/
linux_x86_newlib/bin/x86_64-nacl-gcc: No such file or directory
make: *** [hello_world_x86_32.nexe] Error 127

It appears that the i686-nacl-gcc is simply a bash script that calls
x86_64-nacl-gcc. To check "x86_64-nacl-gcc" exists, that it has the
right ownership, and that it has the right execute permissions, I then
did:

ubuntu$ stat /home/ubuntu/nacl_sdk/pepper_18/toolchain/
linux_x86_newlib/bin/x86_64-nacl-gcc
File: `/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/
bin/x86_64-nacl-gcc'
Size: 204012 Blocks: 400 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 422592 Links: 2
Access: (0755/-rwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1000/
ubuntu)
Access: 2012-02-21 22:55:44.488918815 +0000
Modify: 2012-01-09 23:54:22.000000000 +0000
Change: 2012-02-20 20:24:40.687131435 +0000

However, executing x86_64-nacl-gcc directly gives me a strange "No
such file or directory" error.

ubuntu$ /home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/
x86_64-nacl-gcc
-bash: /home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/
x86_64-nacl-gcc: No such file or directory

I then did a cursory look at the objdump output on x86_64-nacl-gcc to
see if it is a valid executable and it seems to be one.

ubuntu$ objdump -h /home/ubuntu/nacl_sdk/pepper_18/toolchain/
linux_x86_newlib/bin/x86_64-nacl-gcc

/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/x86_64-
nacl-gcc: file format elf32-i386
...

I then ran strace on x86_64-nacl-gcc and noticed the ENOENT error is
set after the execve() system call.

ubuntu$ strace /home/ubuntu/nacl_sdk/pepper_18/toolchain/
linux_x86_newlib/bin/x86_64-nacl-gcc
execve("/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/
x86_64-nacl-gcc", ["/home/ubuntu/nacl_sdk/pepper_18/"...], [/* 20 vars
*/]) = -1 ENOENT (No such file or directory)
dup(2) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|
O_LARGEFILE)
fstat(3, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f9349d7c000
lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(3, "strace: exec: No such file or di"..., 40strace: exec: No
such file or directory
) = 40
close(3) = 0
munmap(0x7f9349d7c000, 4096) = 0
exit_group(1) = ?

The execve manpage says ENOENT is returned when "The file filename or
a script or ELF interpreter does not exist, or a shared library needed
for file or interpreter cannot be found." Since the file exists, the
reason for the error is most likely the latter.

I then installed ia32-libs on my instance.

ubuntu$ sudo apt-get install ia32-libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
...
Setting up ia32-libs-multiarch:i386 (20090808ubuntu26) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

I then tried re-running make and it succeeded.

ubuntu$ make
/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/i686-
nacl-gcc -o hello_world_x86_32.nexe hello_world.c -m32 -O0 -g -pthread
-O0 -g -Wno-long-long -Wall -Wswitch-enum -Werror -pedantic -lppapi
/home/ubuntu/nacl_sdk/pepper_18/toolchain/linux_x86_newlib/bin/i686-
nacl-gcc -o hello_world_x86_64.nexe hello_world.c -m64 -O0 -g -pthread
-O0 -g -Wno-long-long -Wall -Wswitch-enum -Werror -pedantic -lppapi
ubuntu$

Is any way to modify the i686-nacl-gcc bash script to check whether 32-
bit executables can be properly read and print an error message "32-
bit executable cannot be read - try installing ia32-libs".
Alternatively, do this check in "./naclsdk update"? This might save
others time and trouble.

Thanks.

Cheers,

Damian Eads

[Mike Acton]

Thanks Damian,

I just ran into the exact same issue (also on Ubuntu EC2 instance) and you saved me some head scratching! :)

Mike.