Can ./configure get proper data type sizes somehow?

[Boris Gjenero]

When running emconfigure ./configure, the detected data type sizes are those of the host system. When configuring on a 64-bit system, long and pointer sizes will be wrong:

checking size of unsigned long... 8

checking size of int *... 8

Configure will succeed and the code will probably compile, but anything depending on those sizes won't work properly. I know stdint.h or SDL types can be used instead of those tests, but the fact remains that plenty of software uses those Autoconf tests. If the data type sizes could be detected properly this would be one less reason to add Emscripten-specific stuff to Autoconf-related files. 

[Alon Zakai]

This is a limitation of how we "fake" configure results. We invoke the native compiler but with our libraries and defines, so this can fail if types are different. The reason we do this is that configure can do anything natively - read/write to a file, open a process, etc., stuff we can't do in compiled JS.

There is an EMCONFIGURE_JS env var you can set, to force use of JS, which might help here.

Perhaps we should default to EMCONFIGURE_JS, and force people to modify configure scripts the other way - but either way, there will be manual work.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bruce Mitchener]

On Sat, Feb 7, 2015 at 9:40 AM, Alon Zakai <alon...@gmail.com> wrote:

This is a limitation of how we "fake" configure results. We invoke the native compiler but with our libraries and defines, so this can fail if types are different. The reason we do this is that configure can do anything natively - read/write to a file, open a process, etc., stuff we can't do in compiled JS.

There is an EMCONFIGURE_JS env var you can set, to force use of JS, which might help here.

Perhaps we should default to EMCONFIGURE_JS, and force people to modify configure scripts the other way - but either way, there will be manual work.

In this case, you could use the right flags with clang to request a 32 bit build.

On Linux and FreeBSD, pass -m32, on OS X, pass -arch i386.

Seems like you could do that in this chunk:

    env['HOST_CC'] = CLANG_CC

    env['HOST_CXX'] = CLANG_CPP

    env['HOST_CFLAGS'] = "-W" #if set to nothing, CFLAGS is used, which we don't want

    env['HOST_CXXFLAGS'] = "-W" #if set to nothing, CXXFLAGS is used, which we don't want

    env['PKG_CONFIG_LIBDIR'] = path_from_root('system', 'local', 'lib', 'pkgconfig') + os.path.pathsep + path_from_root('system', 'lib', 'pkgconfig')

    env['PKG_CONFIG_PATH'] = os.environ.get ('EM_PKG_CONFIG_PATH') or ''

    env['EMSCRIPTEN'] = path_from_root() 

 - Bruce

[Alon Zakai]

True, but -m32 wouldn't work if 32-bit compat libraries aren't installed, etc. Overall I think the best solution here is to get EMCONFIGURE_JS on by default, so configure tests the exact right setup, and for it to give clear errors where it fails, so people know what to modify in configure scripts.

- Alon

[Alon Zakai]

With one fix, I think I see the test suite passing with EMCONFIGURE_JS on.

Pushed to incoming - please test on your projects that use configure scripts!

- Alon

[Boris Gjenero]

This works with Em-DOSBox, Bochs 2.6.7 and Synaesthesia. However, it doesn't seem to be an improvement. Both Bochs (with configure from http://sourceforge.net/projects/bochs/files/bochs/2.6.7/bochs-2.6.7.tar.gz/download ) and Em-DOSBox (with configure generated by Autoconf 2.69) still detect wrong sizes for long and int *. Configure runs a lot slower. Bochs configure also gives these errors, though they don't actually cause any problems:

checking dynamic linker characteristics... ERROR    root: no input files

note that input files without a known suffix are ignored, make sure your input files end with one of: ('.c', '.C', '.cpp', '.cxx', '.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.m', '.mm', '.bc', '.o', '.obj', '.dylib', '.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP', '.HH') 

checking for ld used by /home/bgjenero/gitapps/emsdk_portable/emscripten/incoming/em++... /home/bgjenero/gitapps/emsdk_portable/emscripten/incoming/emcc

checking if the linker (/home/bgjenero/gitapps/emsdk_portable/emscripten/incoming/emcc) is GNU ld... yes

ERROR    root: no input files

note that input files without a known suffix are ignored, make sure your input files end with one of: ('.c', '.C', '.cpp', '.cxx', '.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.m', '.mm', '.bc', '.o', '.obj', '.dylib', '.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP', '.HH')

Configure leaves a.out and a.out.js files.

It seems impossible to turn off this feature via EMCONFIGURE_JS because os.environ.get() returns a string, which Python treats as True: https://github.com/kripken/emscripten/commit/9be0275876217eacd30c848b1c7cea68d1a3d986#diff-7d2ad1bd249c50ae58c7226135c3ff33R193

[Alon Zakai]

I fixed the turning off problem, thanks.

Slower is expected. However, wrong type sizes and that error are both surprising. Can you please find out what the configure scripts are doing that fails there (i.e. how emcc is invoked and with what inputs)?

- Alon

[Boris Gjenero]

Thanks! I confirm that EMCONFIGURE_JS=0 works as expected now.

The ./configure test basically just does 'fopen ("conftest.val", "w");' and then uses fprintf() to output sizeof() for the requested type into conftest.val. It gives the wrong value because emcc detects the 'fopen' and ''w'' and sets use_js = 0 because JS can't write to files: https://github.com/kripken/emscripten/blob/16d378c78fd9a76f7cf1de856379912e001a1d62/emcc#L211 . So, the host compiler is always used for these tests.

./configure scripts also contain another way to determine sizes when cross compiling. It performs a binary search using compiler errors in expressions of the form 'static int test_array [1 - 2 * !(($2) >= $ac_mid)];'. In other words, if (($2) >= $ac_mid), it will try to make an array of size -1, and if (($2) < $ac_mid) it will make an array of size 1. The script assumes it is not cross compiling when it can run compiled executables, so it uses simple method I described in the last paragraph.

[Alon Zakai]

I see. So if those tests printed instead of writing to a file, I guess this would work. Still would be nice to have this work out of the box.

We could try to get fprintf to work in node by writing directly to the files synchronously. But we don't currently have support for mounting the entire filesystem as NODEFS, it is tricky due to devices, see https://github.com/kripken/emscripten/pull/3152 . However, perhaps we should support an option to mount it, ignoring devices, just for this purpose?

- Alon

[Boris Gjenero]

What about ability to compile all tests to JS without ability to run compiled code? Projects which are properly set up with Autoconf in a way that enables cross-compiling should configure successfully that way. It could be enabled with something like "EMCONFIGURE_JS=2".

[Alon Zakai]

Nice idea, added.

- Alon

--

[Boris Gjenero]

I was specifically asking for inability to run compiled tests with
EMCONFIGURE_JS=2 so that Autoconf ./configure will assume that it is
cross compiling. I'm not sure that this change is supposed to do that.
It seems like
https://github.com/kripken/emscripten/commit/3702801d1fddc80d2a058507b953ede9dff09b11
simply always compiles to JS even if that would fail to run properly.

The "checking whether we are cross compiling" test must fail to run the
compiled binary in order for Autoconf ./configure to assume that it is
cross compiling. If that binary runs but later tests fail to run
properly, then test results may be bad, for example outputting empty
defines like this:
#define SIZEOF_UNSIGNED_CHAR
#define SIZEOF_UNSIGNED_SHORT
#define SIZEOF_UNSIGNED_INT
#define SIZEOF_UNSIGNED_LONG
#define SIZEOF_UNSIGNED_LONG_LONG
#define SIZEOF_INT_P

Surprisingly, I got the results I desire. The cross compiling test
binary failed to execute with "./configure: ./conftest: /usr/bin/node:
bad interpreter: No such file or directory" because it was looking for
/usr/bin/node and there was only /usr/local/bin/node. Then Bochs 2.6.7
proceeded to configure and build without any ./configure hacks being
needed. Impressive! However, if I create a /usr/bin/node symlink, I
can't make ./configure conclude that it is cross compiling.

I guess tests compiled to JS couldn't be run with EMCONFIGURE_JS=1
either, but most tests were being compiled to native code, so this
wasn't causing problems. I see only one Bochs test triggering the error
with EMCONFIGURE_JS=1: "checking for ANSI C header files".

BTW. Failure to run compiled C programs can cause an error like this:

checking whether we are cross compiling... configure: error: in
`/home/bgjenero/projects/bochs/bochs-2.6.7':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
ERROR root: Configure step failed with non-zero return code 1!
Command line: ['./configure', '--without-x', '--with-sdl2',
'--disable-show-ips', '--enable-fast-function-calls',
'--enable-handlers-chaining'] at /home/bgjenero/projects/bochs/bochs-2.6.7

The solution to that error is simple and self-explanatory. I added
--host=le32-unknown-nacl because --host=asmjs-unknown-emscripten isn't
recognized.

On 2015-02-23 07:51 PM, Alon Zakai wrote:
> Nice idea, added.
>
> - Alon
>
>
> On Wed, Feb 18, 2015 at 11:42 AM, Boris Gjenero <boris....@gmail.com
> <mailto:boris....@gmail.com>> wrote:
>
> What about ability to compile all tests to JS without ability to run
> compiled code? Projects which are properly set up with Autoconf in a
> way that enables cross-compiling should configure successfully that
> way. It could be enabled with something like "EMCONFIGURE_JS=2".
>
> On Tuesday, 17 February 2015 14:59:15 UTC-5, Alon Zakai wrote:
>
> I see. So if those tests printed instead of writing to a file, I
> guess this would work. Still would be nice to have this work out
> of the box.
>
> We could try to get fprintf to work in node by writing directly
> to the files synchronously. But we don't currently have support
> for mounting the entire filesystem as NODEFS, it is tricky due
> to devices, see

> https://github.com/kripken/__emscripten/pull/3152
> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fkripken%2Femscripten%2Fpull%2F3152&sa=D&sntz=1&usg=AFQjCNFOZkbhLTUYLxBJ2DgWy2fgyYMSXg>

> . However, perhaps we should support an option to mount it,
> ignoring devices, just for this purpose?
>
> - Alon
>
> --
> You received this message because you are subscribed to the Google
> Groups "emscripten-discuss" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to emscripten-disc...@googlegroups.com

> <mailto:emscripten-disc...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/emscripten-discuss/zLMDkPUmZIQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> emscripten-disc...@googlegroups.com
> <mailto:emscripten-disc...@googlegroups.com>.

[Brion Vibber]

On Mon, Feb 23, 2015 at 6:25 PM, Boris Gjenero <boris....@gmail.com> wrote:

[snip]

The solution to that error is simple and self-explanatory. I added --host=le32-unknown-nacl because --host=asmjs-unknown-emscripten isn't recognized.

 

That reminds me -- we should probably submit a patch to GNU for autoconf's config.sub to recognize asmjs-unknown-emscripten as a legit host triple. For projects with an old autoconf config, updating the config.sub with a fixed version may help get configure working in cross-compilation mode...

The current master version appears to still not know of emscripten:
http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD

This would let me clean up a couple remaining configure hacks for Xiph's ogg/vorbis/theora libraries I think...

-- brion vibber (brion @ pobox.com / bvibber @ wikimedia.org)

[Alon Zakai]

Oh, I did not understand you properly before.

The EMCONFIGURE_JS option I added forces JS usage in emconfigure, no exceptions. What you suggest is to also add an option that makes the "executables" emitted fail to run, intentionally (how should they fail to run, btw)? Perhaps we should call that EMCONFIGURE_JS_NOEXECUTE?

- Alon

    send an email to emscripten-discuss+unsub...@googlegroups.com
    <mailto:emscripten-discuss+unsub...@googlegroups.com>.

    For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "emscripten-discuss" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/emscripten-discuss/zLMDkPUmZIQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to

emscripten-discuss+unsub...@googlegroups.com
<mailto:emscripten-discuss+unsub...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Boris Gjenero]

Normally if you cross-compile for a different architecture and can't run
the created binary, the error should be ENOEXEC (Exec format error).
However, Autoconf's ./configure does not care about the error code. All
that matters is that the file fails to execute. If you simply output the
JS to a file, that shouldn't be executable without a "#!".

BTW. Note that Autoconf ./configure will search for these possible
output files: "a.out conftest.exe conftest a.exe a_out.exe b.out
conftest.*". It will figure out the extension of executable files based
on what it finds. So, if you produced conftest.js it would assume .js is
the extension. I think that it should also select .html if both .js and
.html exist. At least bash will sort files matching a glob pattern. Then
that extension is automatically used when compiling the target. So, it
would be possible to just "emconfigure ./configure && make" and get JS
output automatically.

On 2015-02-24 04:17 PM, Alon Zakai wrote:
> Oh, I did not understand you properly before.
>
> The EMCONFIGURE_JS option I added forces JS usage in emconfigure, no
> exceptions. What you suggest is to also add an option that makes the
> "executables" emitted fail to run, intentionally (how should they fail
> to run, btw)? Perhaps we should call that EMCONFIGURE_JS_NOEXECUTE?
>
> - Alon
>
>
> On Mon, Feb 23, 2015 at 6:25 PM, Boris Gjenero <boris....@gmail.com
> <mailto:boris....@gmail.com>> wrote:
>
> I was specifically asking for inability to run compiled tests with
> EMCONFIGURE_JS=2 so that Autoconf ./configure will assume that it is
> cross compiling. I'm not sure that this change is supposed to do
> that. It seems like

> https://github.com/kripken/__emscripten/commit/__3702801d1fddc80d2a058507b953ed__e9dff09b11

> `/home/bgjenero/projects/__bochs/bochs-2.6.7':

> configure: error: cannot run C compiled programs.
> If you meant to cross compile, use `--host'.
> See `config.log' for more details
> ERROR root: Configure step failed with non-zero return code 1!
> Command line: ['./configure', '--without-x', '--with-sdl2',

> '--disable-show-ips', '--enable-fast-function-calls'__,
> '--enable-handlers-chaining'] at
> /home/bgjenero/projects/bochs/__bochs-2.6.7

>
> The solution to that error is simple and self-explanatory. I added

> --host=le32-unknown-nacl because --host=asmjs-unknown-__emscripten
> isn't recognized.

[Alon Zakai]

I think I see. But to be honest, I don't know that much about autotools, I've never used them (more than running ./configure now and then) ;)  Would you perhaps like to write a pull request for this stuff?

- Alon