Getting libgmp to compile with Emscripten

[Marcos Scriven]

I'm still trying to get CGAL working with Emscripten - and one of the dependencies is the GMP lib.

I found that Alon had already done this: https://github.com/kripken/gmp.js  (oddly the README.markdown is updated, whereas the README.md isn't)

However, it's about 7 months old, and I've not been able to repeat it. 

I did the following:

1) Download 5.1.1 GMP tarball / untarred

2) Ran vanilla ./configure

3) Ran vanilla make

That left the following in the .libs dir:

-rwxrwxr-x  1 marcosscriven marcosscriven  528404 Feb 21 22:22 libgmp.so.10.1.1

lrwxrwxrwx  1 marcosscriven marcosscriven      16 Feb 21 22:22 libgmp.so.10 -> libgmp.so.10.1.1

lrwxrwxrwx  1 marcosscriven marcosscriven      16 Feb 21 22:22 libgmp.so -> libgmp.so.10.1.1

-rw-rw-r--  1 marcosscriven marcosscriven 1276280 Feb 21 22:22 libgmp.a

-rw-rw-r--  1 marcosscriven marcosscriven     914 Feb 21 22:22 libgmp.lai

drwxr-xr-x 16 marcosscriven marcosscriven    4096 Feb 21 22:22 ..

lrwxrwxrwx  1 marcosscriven marcosscriven      12 Feb 21 22:22 libgmp.la -> ../libgmp.la

drwxrwxr-x  2 marcosscriven marcosscriven    4096 Feb 21 22:22 .

4) Ran: emconfigure ./configure --build=none --host=none

5)Seemed to work:

configure: summary of build options:

  Version:           GNU MP 5.1.1

  Host type:         none-none-none

  ABI:               standard

  Install prefix:    /usr/local

  Compiler:          /home/marcosscriven/sources/emscripten/emcc

  Static libraries:  yes

  Shared libraries:  no

6) Ignored this instruction : Edit gmp.h and disable LIKELY, UNLIKELY

Because in the 5.1.1 code the gmp.h looks like this:

#if __GMP_GNUC_PREREQ (3,0)

#define __GMP_LIKELY(cond)    __builtin_expect ((cond) != 0, 1)

#define __GMP_UNLIKELY(cond)  __builtin_expect ((cond) != 0, 0)

#else

#define __GMP_LIKELY(cond)    (cond)

#define __GMP_UNLIKELY(cond)  (cond)

#endif

7) Modified config.h thus:

Edit config.h and disable HAVE_QUAD_T, HAVE_OBSTACK_VPRINTF, HAVE_LONG_DOUBLE and HAVE_LONG_LONG

8) Ran: EMMAKEN_CFLAGS="-g" make -j 2

9) Result in .libs was updated to this:

rw-rw-r--  1 marcosscriven marcosscriven 1262790 Feb 21 22:28 libgmp.a

-rw-rw-r--  1 marcosscriven marcosscriven     863 Feb 21 22:28 libgmp.lai

drwxr-xr-x 16 marcosscriven marcosscriven    4096 Feb 21 22:28 ..

lrwxrwxrwx  1 marcosscriven marcosscriven      12 Feb 21 22:28 libgmp.la -> ../libgmp.la

10) Ran: emcc -O0 --closure 0 test.c .libs/libgmp.a -o complete.js

11) That ran without errors, and produced the complete.js file. Unfortunately, it didn't run:

marcosscriven@ubuntu-laptop:~/sources/gmp-5.1.1$ node complete.js 

/home/marcosscriven/sources/gmp-5.1.1/complete.js:2452

      ___gmpz_init(((5242892)|0));

      ^

TypeError: undefined is not a function

    at _pidigits (/home/marcosscriven/sources/gmp-5.1.1/complete.js:2452:7)

    at Object._main (/home/marcosscriven/sources/gmp-5.1.1/complete.js:2429:7)

    at Object.callMain (/home/marcosscriven/sources/gmp-5.1.1/complete.js:2644:24)

    at doRun (/home/marcosscriven/sources/gmp-5.1.1/complete.js:2679:20)

    at run (/home/marcosscriven/sources/gmp-5.1.1/complete.js:2703:12)

    at Object.<anonymous> (/home/marcosscriven/sources/gmp-5.1.1/complete.js:2725:13)

    at Module._compile (module.js:449:26)

    at Object.Module._extensions..js (module.js:467:10)

    at Module.load (module.js:356:32)

    at Function.Module._load (module.js:312:12)

Any ideas why that's failed please?

Thanks

[Marcos Scriven]

Just realised of course that the gmp.h file is for some reason not being generated properly from the gmp.h.in file

But why that is exactly I'm not sure. Works find with the vanilla ./configure

[Alon Zakai]

An error like that implies the function is missing. Likely it was not
linked in (emscripten is more tolerant of missing symbols than other
compilers). You can do llvm-nm on other object files to find where it
is, to find out why it wasn't included.

- azakai

> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to emscripten-disc...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Marcos Scriven]

It does appear to be there:

marcosscriven@ubuntu-laptop:~/sources/gmp-5.1.1/.libs$ llvm-nm libgmp.a | grep gmpz_init

00000000 T __gmpz_init

00000000 T __gmpz_init2

         U __gmpz_init

00000000 T __gmpz_inits

00000000 T __gmpz_init_set

00000000 T __gmpz_init_set_d

00000000 T __gmpz_init_set_si

00000000 T __gmpz_init_set_str

00000000 T __gmpz_init_set_ui

         U __gmpz_init

         U __gmpz_init_set_ui

         U __gmpz_init

         U __gmpz_init

         U __gmpz_init_set

         U __gmpz_init_set_str

         U __gmpz_init

         U __gmpz_init2

         U __gmpz_init_set

         U __gmpz_init

         U __gmpz_init_set

> email to emscripten-discuss+unsub...@googlegroups.com.

[Marcos Scriven]

I also tried just cloning the GMP 5.0.2 from here: https://github.com/kripken/gmp.js.git

But get exactly the same issue.

Quite frustrating, as this shows so much promise but is so opaque. 

Can anyone recommend any further reading/tutorials around this?

The best thing I've found so far is stepping through the runner.py tests in a Python debugger, but it's less than ideal inspecting the variables in get_library in Python just to figure out what env variables it's setting up for emscripten in cmake.

[Marcos Scriven]

I added debugging, but still can't see any issues:

marcosscriven@ubuntu-laptop:~/sources/gmp-5.0.2$ EMCC_DEBUG=1 ~/sources/emscripten/emcc -O0 --closure 0 test.c .libs/libgmp.a -o complete.js

emcc invocation:  /home/marcosscriven/sources/emscripten/emcc -O0 --closure 0 test.c .libs/libgmp.a -o complete.js 

(Emscripten: Running sanity checks)

emcc: compiling to bitcode

emcc: compiling source file:  test.c

emcc running: /usr/local/bin/clang -m32 -U__i386__ -U__x86_64__ -U__i386 -U__x86_64 -Ui386 -Ux86_64 -U__SSE__ -U__SSE2__ -U__MMX__ -UX87_DOUBLE_ROUNDING -UHAVE_GCC_ASM_FOR_X87 -DEMSCRIPTEN -U__STRICT_ANSI__ -U__CYGWIN__ -D__STDC__ -Xclang -triple=i386-pc-linux-gnu -D__IEEE_LITTLE_ENDIAN -fno-math-errno -fno-ms-compatibility -nostdinc -Xclang -nobuiltininc -Xclang -nostdsysteminc -Xclang -isystem/home/marcosscriven/sources/emscripten/system/local/include -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/emscripten -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/bsd -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/libc -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/libcxx -Xclang -isystem/home/marcosscriven/sources/emscripten/system/lib/libcxxabi/include -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/gfx -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/net -Xclang -isystem/home/marcosscriven/sources/emscripten/system/include/SDL -U__APPLE__ -U__linux__ -D_LIBCPP_HAS_NO_DELETED_FUNCTIONS -emit-llvm -c test.c -o /tmp/tmp_pOV54/test_0.o

emcc: copying library file:  .libs/libgmp.a

emcc: will generate JavaScript

emcc: considering including libcxx: we need set([]) and have set([])

emcc: considering including libcxxabi: we need set([]) and have set([])

emcc: considering including libc: we need set(['realloc', 'malloc', 'memcpy', 'free']) and have set([])

emcc: including libc

emcc: linking:  ['/tmp/tmp_pOV54/test_0.o', '/tmp/tmp_pOV54/libgmp_1.a', '/home/marcosscriven/.emscripten_cache/libc.bc']

emcc: llvm-linking: ['/tmp/tmp_pOV54/test_0.o', '/home/marcosscriven/.emscripten_cache/libc.bc']

emcc: saving intermediate processing steps to /tmp/emscripten_temp

emcc: LLVM opts: ['-internalize', '-internalize-public-api-list=main', '-globaldce']

emcc:    step took 0.01 seconds

emcc:    step took 0.00 seconds

emcc: LLVM => JS

emscript: ll=>js

  emscript: scan took 0.000277042388916 seconds

  emscript: split took 0.000169038772583 seconds

  emscript: phase 1 took 0.173984050751 seconds

  emscript: phase 2 working on 1 chunks  (intended chunk size: 1.00 MB, meta: 0.00 MB, forwarded: 0.01 MB, total: 0.01 MB)

  emscript: phase 2 took 0.206316947937 seconds

  emscript: phase 2b took 0.000115871429443 seconds

  emscript: phase 2c took 0.00135588645935 seconds

  emscript: phase 3 took 0.170216083527 seconds

emcc:    step took 0.60 seconds

emcc: total time: 1.62 seconds

[Marcos Scriven]

Checked the tmp dir:

marcosscriven@ubuntu-laptop:/tmp/emscripten_temp$ ls -altr

total 168

-rw-rw-r-- 1 marcosscriven marcosscriven 43356 Feb 22 19:27 emcc-0-basebc.bc

-rw-rw-r-- 1 marcosscriven marcosscriven  2272 Feb 22 19:27 emcc-1-linktime.bc

-rw-rw-r-- 1 marcosscriven marcosscriven 12749 Feb 22 19:27 emcc-2-ll.ll

-rw-rw-r-- 1 marcosscriven marcosscriven 97591 Feb 22 19:27 emcc-3-original.js

drwxrwxrwt 9 root          root           4096 Feb 22 19:27 ..

drwxrwxr-x 2 marcosscriven marcosscriven  4096 Feb 22 19:27 .

marcosscriven@ubuntu-laptop:/tmp/emscripten_temp$ llvm-nm emcc-0-basebc.bc | grep gmpz_init

         U __gmpz_init

         U __gmpz_init_set_ui

marcosscriven@ubuntu-laptop:/tmp/emscripten_temp$ llvm-nm emcc-1-linktime.bc | grep gmpz_init

         U __gmpz_init

         U __gmpz_init_set_ui

marcosscriven@ubuntu-laptop:/tmp/emscripten_temp$ node emcc-3-original.js 

/tmp/emscripten_temp/emcc-3-original.js:2452

      ___gmpz_init(((5242892)|0));

      ^

TypeError: undefined is not a function

    at _pidigits (/tmp/emscripten_temp/emcc-3-original.js:2452:7)

    at Object._main (/tmp/emscripten_temp/emcc-3-original.js:2429:7)

    at Object.callMain (/tmp/emscripten_temp/emcc-3-original.js:2644:24)

    at doRun (/tmp/emscripten_temp/emcc-3-original.js:2679:20)

    at run (/tmp/emscripten_temp/emcc-3-original.js:2703:12)

    at Object.<anonymous> (/tmp/emscripten_temp/emcc-3-original.js:2725:13)

    at Module._compile (module.js:449:26)

    at Object.Module._extensions..js (module.js:467:10)

    at Module.load (module.js:356:32)

    at Function.Module._load (module.js:312:12)

[Alon Zakai]

-U means it is undefined.

Could be the problem is semantics of .a linking, as discussed on irc.

- azakai

>> > email to emscripten-disc...@googlegroups.com.

>> > For more options, visit https://groups.google.com/groups/opt_out.
>> >
>> >
>

> --
> You received this message because you are subscribed to the Google Groups
> "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to emscripten-disc...@googlegroups.com.

[Marcos Scriven]

Got further with this - in the latest (5.1.1) version, there's a new configure option: --disable-assembly

http://gmplib.org/gmp5.1.html

That ends up producing a generic C .a and .so file - *but*, now the test runs, it doesn't actually finish.

The exact test I am running is: https://github.com/kripken/gmp.js/blob/master/test.c

I added a printf statement at the *before* the call to pidigits - this only gets output in the Javascript version if I comment out the call to pidigits!

Is there some kind of out-of-order execution going on here?

Marcos

[Marcos Scriven]

Profiling in Chrome, it's stuck in this function:

function ___gmpn_mul_1($rp, $up, $n, $vl) {

var label = 0;

label = 2;

while(1) switch(label) {

case 2:

var $1;

var $2;

var $3;

var $4;

var $ul;

var $cl;

var $hpl;

var $lpl;

var $__x0;

var $__x1;

var $__x2;

var $__x3;

var $__ul;

var $__vl;

var $__uh;

var $__vh;

var $__u;

var $__v;

$1=$rp;

$2=$up;

$3=$n;

$4=$vl;

label = 3; break;

case 3:

label = 4; break;

case 4: label = 5; break; case 5: label = 6; break; case 6:

$cl=0;

label = 7; break;

case 7:

var $10=$2;

var $11=(($10+4)|0);

$2=$11;

var $12=HEAP32[(($10)>>2)];

$ul=$12;

label = 8; break;

case 8:

var $14=$ul;

$__u=$14; var $15=$4; $__v=$15; var $16=$__u; var $17=$16 & (undef); $__ul=$17; var $18=$__u; var $19=$18 >>> 32; $__uh=$19; var $20=$__v; var $21=$20 & (undef); $__vl=$21; var $22=$__v; var $23=$22 >>> 32; $__vh=$23; var $24=$__ul; var $25=$__vl; var $26=((($24)*($25))&-1); $__x0=$26; var $27=$__ul; var $28=$__vh; var $29=((($27)*($28))&-1); $__x1=$29; var $30=$__uh; var $31=$__vl; var $32=((($30)*($31))&-1); $__x2=$32; var $33=$__uh; var $34=$__vh; var $35=((($33)*($34))&-1); $__x3=$35; var $36=$__x0; var $37=$36 >>> 32; var $38=$__x1; var $39=((($38)+($37))|0); $__x1=$39; var $40=$__x2; var $41=$__x1; var $42=((($41)+($40))|0); $__x1=$42;

var $43=$__x1;

var $44=$__x2;

var $45=(($43)>>>0) < (($44)>>>0);

if ($45) { label = 9; break; } else { label = 10; break; }

case 9:

var $47=$__x3;

var $48=((($47)+((undef)))|0);

$__x3=$48;

label = 10; break;

case 10:

var $50=$__x3;

var $51=$__x1;

var $52=$51 >>> 32;

var $53=((($50)+($52))|0);

$hpl=$53;

var $54=$__x1;

var $55=$54 << 32;

var $56=$__x0;

var $57=$56 & (undef);

var $58=((($55)+($57))|0);

$lpl=$58;

label = 11; break;

case 11:

var $60=$cl;

var $61=$lpl;

var $62=((($61)+($60))|0);

$lpl=$62;

var $63=$lpl;

var $64=$cl;

var $65=(($63)>>>0) < (($64)>>>0);

var $66=(($65)&1);

var $67=$hpl;

var $68=((($66)+($67))|0);

$cl=$68;

var $69=$lpl;

var $70=$1;

var $71=(($70+4)|0);

$1=$71;

HEAP32[(($70)>>2)]=$69;

label = 12; break;

case 12:

var $73=$3;

var $74=((($73)-(1))|0);

$3=$74;

var $75=(($74)|0)!=0;

if ($75) { label = 7; break; } else { label = 13; break; }

case 13:

var $77=$cl;

return $77;

default: assert(0, "bad label: " + label);

}

[Marcos Scriven]

Finally got it working!

I was using 64bit linux to compile on, but changed to a 32bit linux VM, and hey presto, it worked. Phew!

[Nicolas Desmoulins]

Hello Marcos,

I'm also trying to compile gmp under javascript, alas without success.
I tried with the gmp.js project, following exactly the given instructions, but the compilation fails.

I then tried with gmp 5.1.2, using the commands you are using:

Here are the steps I did:

-  normal ./configure && make

-  ~/emscripten/emconfigure ./configure --build=none --host=none --disable-assembly

-  Edit config.h and disable HAVE_QUAD_T, HAVE_OBSTACK_VPRINTF, HAVE_LONG_DOUBLE and HAVE_LONG_LONG

-  EMMAKEN_CFLAGS="-g" make -j 2

-  ~/emscripten/emcc -O0 --closure 0 test.c .libs/libgmp.a -o complete.js


I obtain a complete.js file, but when launched with node (node complete.js), it doesn't work.
With my main 64-bit distribution it uses a lot of CPU but do nothing.
Under a 32 VM it fails with an error similar to what you had at some point:

missing function: __gmpz_init
-1:
Error
    at abort (/home/adminuser/devel/gmp-5.1.2/complete.js:409:32)
    at ___gmpz_init (/home/adminuser/devel/gmp-5.1.2/complete.js:980:56)
    at Object._main (/home/adminuser/devel/gmp-5.1.2/complete.js:2736:3)
    at Object.callMain (/home/adminuser/devel/gmp-5.1.2/complete.js:4673:26)
    at doRun (/home/adminuser/devel/gmp-5.1.2/complete.js:4712:31)
    at run (/home/adminuser/devel/gmp-5.1.2/complete.js:4735:12)
    at Object.<anonymous> (/home/adminuser/devel/gmp-5.1.2/complete.js:4751:1)
    at Module._compile (module.js:446:26)
    at Object..js (module.js:464:10)
    at Module.load (module.js:353:32)

/home/adminuser/devel/gmp-5.1.2/complete.js:4681
      throw e;


Can you spot a mistake in the command I ran?

Did your compiled version actually works?


Thanks for your help.

[Marcos Scriven]

Hi Nicolas

I got GMP compiling a while back. It was a dependency of CGAL, along with MPFR and Boost. 

You can clone my repo here: https://github.com/marcosscriven/cgaljs

Try that, it builds GMP and there's a test to run against it - let me know if you're still having trouble.

Marcos

[Nicolas Desmoulins]

Thanks a lot Marcos.

It seems to work this time. At least the test program works.

Note however that it's only ok when compiled under my 32-bit VM, and not a 64-bit distrib.

Comparing the two config.h produced, the following parameters differ:

/* The size of `mp_limb_t', as computed by sizeof. */
#define SIZEOF_MP_LIMB_T 4

/* The size of `unsigned', as computed by sizeof. */
#define SIZEOF_UNSIGNED 4

/* The size of `unsigned long', as computed by sizeof. */
#define SIZEOF_UNSIGNED_LONG 4

/* The size of `unsigned short', as computed by sizeof. */
#define SIZEOF_UNSIGNED_SHORT 2

/* The size of `void *', as computed by sizeof. */
#define SIZEOF_VOID_P 4

Under a 64 bit architecture, all '4' values are replaced by '8'.
But just changing theses values is not enough.

And when adding the option ABI=32 during the gmp configure it fails:

"emcc  has sizeof(long)==4... no
configure: error: could not find a working compiler, see config.log for details"

Well whatever. At least I have a way to make it works.

Thanks again.

Nicolas

2013/6/6 Marcos Scriven <mar...@scriven.org>

Marcos

--
You received this message because you are subscribed to a topic in the Google Groups "emscripten-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/emscripten-discuss/kGi8O1vGBVQ/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to emscripten-disc...@googlegroups.com.

[Nicolas Desmoulins]

Hello again,

After some additional tests, it seems that libgmp is not very reliable when compiled into javascript.

I have tested it with a program of mine, which make heavy use of mpz_t  (big integers), and the results are very often not correct (but sometimes are...).

Too bad...

Nicolas

2013/6/6 Marcos Scriven <mar...@scriven.org>

Hi Nicolas

Marcos

--

[Ashnur Nasir Pal]

Hi, I've tried to use this. It is really great, because I have no idea why, with your configs and commands it compile under arch linux x86_64. However it hangs.
I also tried it under a 32bit chroot but there it fails saying it can't find a compatible compiler.

I am really stuck on this, have been trying to compile it for 2 weeks now. Any kind of ideas or suggestions are welcome. Thanks.