SDL2 and Emscripten Ports updates

[Alon Zakai]

Over the last several months, Daft-Freak and gsathya have done great work to port SDL2 (see https://github.com/kripken/emscripten/issues/2404 ). You can see that in use already in the recent Humble Bundle,

https://www.humblebundle.com/?asmjs_bundle

To make SDL2 easier to use (until now it was in a separate repo you had to grab and build), we've created Emscripten Ports, now on the incoming branch, documented here:

http://kripken.github.io/emscripten-site/docs/compiling/Building-Projects.html#emscripten-ports

All Ports are is a set of repos on github that emcc knows about and can fetch and build. If you do something like

./emcc tests/sdl2glshader.c -s USE_SDL=2 -s LEGACY_GL_EMULATION=1 -o sdl2.html

on the incoming branch, then USE_SDL=2 makes it download and build SDL2 for you.

Currently SDL2 and SDL2_image are in the ports repos. As mentioned on the docs page, it is fairly easy to add more, if there are other common libraries people are using.

There are tests for both SDL2 and Ports in the test suite now, but please test both of those out, issues are very possible.

- Alon

[Mark Callow]

On 22/10/2014 05:45, Alon Zakai wrote:

...

To make SDL2 easier to use (until now it was in a separate repo you had to grab and build), we've created Emscripten Ports, now on the incoming branch, documented here:

This is great news. Will emsdk provide pre-compiled versions of the ports?

All Ports are is a set of repos on github that emcc knows about and can fetch and build. If you do something like

./emcc tests/sdl2glshader.c -s USE_SDL=2 -s LEGACY_GL_EMULATION=1 -o sdl2.html

on the incoming branch, then USE_SDL=2 makes it download and build SDL2 for you.

Is there a way to build just libSDL2?

Regards

    -Mark

--

注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情報の使用を固く禁じております。エラー、手違いでこのメールを受け取られましたら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies.

[Alon Zakai]

We could provide precompiled versions in the sdk, but it would increase the sdk size quite a bit, especially as we add more ports. So not sure what is best here.

Not sure what you mean by "build just libSDL2"? It gets built and stored in the cache (~/.emscripten_cache), then used from there. So it is built on its own (once), then linked in.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark Callow]

On 23/10/2014 03:47, Alon Zakai wrote:

We could provide precompiled versions in the sdk, but it would increase the sdk size quite a bit, especially as we add more ports. So not sure what is best here.

I was thinking the SDK would show a list of available ports, much in the way it shows a list of available tools now. You could then install the ports of interest. They wouldn't be downloaded until install so there would be virtually no impact on the size of the sdk download.

Not sure what you mean by "build just libSDL2"? It gets built and stored in the cache (~/.emscripten_cache), then used from there. So it is built on its own (once), then linked in.

You explained that adding -s USE_SDL=2 would cause emcc to fetch and build SDL2 and include it in the app. I was wondering how you could fetch and build just SDL2. If the result is stored in the cache though, there is probably not much need to download and build it separately.

I am generating MSVS projects with GYP that include configurations for building for Win32, x64 and Emscripten platforms. (I have modified GYP to understand the vs-tool properties; I need to write some tests before Google will incorporate the changes.) The issue I have is that, due to a fundamental limitation of GYP, I have to use the same list of libraries for each configuration. Therefore it would really be helpful if

• emcc (emlink?) recognized the names of built-in libraries (e.g. -l libGLESv2) and ignored them, instead of searching for a library with that name. I have had to create empty libraries to avoid errors.
  
• libraries from ports were added via, e.g. -lSDL2 during the link step rather than -s USE_SDL=2 during compile. This would also be more in keeping with native tool chains.  At present I am using a pre-compiled libSDL2 from gsathya with -lSDL2 on the link step and it is working fine.
  

[Mark Callow]

On 23/10/2014 11:43, Mark Callow wrote:

I am generating MSVS projects with GYP that include configurations for building for Win32, x64 and Emscripten platforms. (I have modified GYP to understand the vs-tool properties; I need to write some tests before Google will incorporate the changes.) The issue I have is that, due to a fundamental limitation of GYP, I have to use the same list of libraries for each configuration. Therefore it would really be helpful if

• emcc (emlink?) recognized the names of built-in libraries (e.g. -l libGLESv2) and ignored them, instead of searching for a library with that name. I have had to create empty libraries to avoid errors.
  
• libraries from ports were added via, e.g. -lSDL2 during the link step rather than -s USE_SDL=2 during compile. This would also be more in keeping with native tool chains.  At present I am using a pre-compiled libSDL2 from gsathya with -lSDL2 on the link step and it is working fine.
  

These changes would be helpful for anybody using vs-tool and creating an Emscripten config from an existing windows config, as they will longer need to make changes in the new Emscripten configuration in order for it to build.

[Alon Zakai]

On Wed, Oct 22, 2014 at 7:43 PM, Mark Callow <callo...@artspark.co.jp> wrote:

On 23/10/2014 03:47, Alon Zakai wrote:

We could provide precompiled versions in the sdk, but it would increase the sdk size quite a bit, especially as we add more ports. So not sure what is best here.

I was thinking the SDK would show a list of available ports, much in the way it shows a list of available tools now. You could then install the ports of interest. They wouldn't be downloaded until install so there would be virtually no impact on the size of the sdk download.

emcc --show-ports lists the available ports. We could improve the presentation there though.
 

Not sure what you mean by "build just libSDL2"? It gets built and stored in the cache (~/.emscripten_cache), then used from there. So it is built on its own (once), then linked in.

You explained that adding -s USE_SDL=2 would cause emcc to fetch and build SDL2 and include it in the app. I was wondering how you could fetch and build just SDL2. If the result is stored in the cache though, there is probably not much need to download and build it separately.

I am generating MSVS projects with GYP that include configurations for building for Win32, x64 and Emscripten platforms. (I have modified GYP to understand the vs-tool properties; I need to write some tests before Google will incorporate the changes.) The issue I have is that, due to a fundamental limitation of GYP, I have to use the same list of libraries for each configuration. Therefore it would really be helpful if

• emcc (emlink?) recognized the names of built-in libraries (e.g. -l libGLESv2) and ignored them, instead of searching for a library with that name. I have had to create empty libraries to avoid errors.
  
• libraries from ports were added via, e.g. -lSDL2 during the link step rather than -s USE_SDL=2 during compile. This would also be more in keeping with native tool chains.  At present I am using a pre-compiled libSDL2 from gsathya with -lSDL2 on the link step and it is working fine.
  

We did consider using -lSDL2, but Jukka convinced me that it was too brittle - could have confusion with a local library of that name, etc. So -s USE_SDL=2 is an explicit way to say "use the ports version of this", while -lSDL2 is "use a local copy of it".

I think that is a safer default behavior, but I wouldn't be opposed to having an option to treat -lSDL2 as using it from ports.

- Alon

[Mark Callow]

On 24/10/2014 05:18, Alon Zakai wrote:

We did consider using -lSDL2, but Jukka convinced me that it was too brittle - could have confusion with a local library of that name, etc. So -s USE_SDL=2 is an explicit way to say "use the ports version of this", while -lSDL2 is "use a local copy of it".

I think that is a safer default behavior, but I wouldn't be opposed to having an option to treat -lSDL2 as using it from ports.

The more Emscripten diverges from the standard behavior of compiler tool chains, the harder it is to adapt other build tools to work with it. The issues I am having with GYP are symptomatic of this.

In the specific case of "-s USE_SDL=2"  vs the standard "-lSDL2", if the latter is brittle, why isn't -lc brittle for regular C/C++ compilers? There is a well defined search path; directories specified with -L come before the default directories. In other words, it is not brittle at all. If you specify a -L that contains a library with the same name as a system library, you are expected to know what you are doing.

Emscripten should likewise search any specified -L directories and, if a specified library isn't found, then it should search its built-in libraries and equivalent functions and then the ports.

It should also not issue errors that a library specified with -l can't be found, if that libraries functionality is built into Emscripten. E.g. specifying -lGLESv2 should not cause emlink to spit out "Error: libGLESv2.a: File not Found" (or something like that) as the GLESv2 functions are all included in Emscripten.

This changes would make Emscripten much more friendly to other build tools.

[Bruce Mitchener]

On Tue, Oct 28, 2014 at 9:07 AM, Mark Callow <callo...@artspark.co.jp> wrote:

On 24/10/2014 05:18, Alon Zakai wrote:

We did consider using -lSDL2, but Jukka convinced me that it was too brittle - could have confusion with a local library of that name, etc. So -s USE_SDL=2 is an explicit way to say "use the ports version of this", while -lSDL2 is "use a local copy of it".

I think that is a safer default behavior, but I wouldn't be opposed to having an option to treat -lSDL2 as using it from ports.

The more Emscripten diverges from the standard behavior of compiler tool chains, the harder it is to adapt other build tools to work with it. The issues I am having with GYP are symptomatic of this.

In the specific case of "-s USE_SDL=2"  vs the standard "-lSDL2", if the latter is brittle, why isn't -lc brittle for regular C/C++ compilers? There is a well defined search path; directories specified with -L come before the default directories. In other words, it is not brittle at all. If you specify a -L that contains a library with the same name as a system library, you are expected to know what you are doing.

Emscripten should likewise search any specified -L directories and, if a specified library isn't found, then it should search its built-in libraries and equivalent functions and then the ports.

It should also not issue errors that a library specified with -l can't be found, if that libraries functionality is built into Emscripten. E.g. specifying -lGLESv2 should not cause emlink to spit out "Error: libGLESv2.a: File not Found" (or something like that) as the GLESv2 functions are all included in Emscripten.

This changes would make Emscripten much more friendly to other build tools.

For what little it is worth, this sounds good / right to me.

 - Bruce

 

[Alon Zakai]

If Jukka is ok with that, I am too. I'm not that picky, nor knowledgeable about C toolchain things.

- Alon

--

[Jukka Jylänki]

The current -s USE_SDL2=1 option is overloaded to download the SDL2 port on demand. If we reused -lSDL2 for that, it'd mean that we would spin off http downloads on demand. What if someone actually does have a library called libSDL2.a on his system that he built manually (a lot of existing users already do) and he instead wants to link to that? This means we should check the -L paths manually and only revert to downloading if none of those paths contained any of predefined names libSDL2.a, libSDL2.so, libSDL2.dylib, libSDL2.lib, SDL2.lib or SDL2.a. That would kind of work, and users who built SDL2 manually would then be able to link to the library they built themselves, and users who did not, would get the download on demand.

However this scheme is so brittle that it's just picking blood from your own nose, because it has the unfortunate effect that if there are any mistakes in a build (-L directories were passed wrong, an earlier build step had failed to produce a libSDL2.a file to some expected location, a build clean directive has been called to SDL2 before building main, and there's no dependency tracking, or something else), we would not be able to detect any of these, but instead would proceed to happily assuming that the user was not looking to link to his own SDL2, but we should download and build the ports SDL2 on demand. I can imagine someone mistaking a relative link path -L../../sdl2/lib when he was intending a -L../sdl2/lib or something else, and getting Emscripten downloading files off the web as a random result. This is not very robust nor explicit and I don't think we should go there.

The same rationale applies for GLES, GLUT, GLFW and others, where I think we should not overload -lGLESv2, -lGLUT and so on to mean to silently link to Emscripten-provided ones if others are not present.

I somewhat dislike the current behavior of -s USE_SDL2=1 at link time for downloading and building SDL2 on demand. This is because the scheme does not feel scalable. What should we do with other potential libraries like FreeType and Regal and zlib and other common ones that people are using? I think it would be better if we had separate scripts $EMSCRIPTEN/ports/install_{sdl2, freetype, zlib, ...}.py that would download and build the appropriate ports to the cache - and only do so when user explicitly runs one of those scripts, and Emscripten would never do it during compilation or linking. In that scenario, people could pass "-L~/.emscripten_ports/lib -lSDL2" to link, or we could even allow omitting -L ~/.emscripten_ports/lib as a link directory and always treat that as a system built-in link directory that exists and is searched last. There would not be any confusion there, because a copy of SDL2 would never appear there unless someone explicitly had installed one via a call to $EMSCRIPTEN/ports/install_sdl2.py, so the danger of silently doing the wrong fallback would not exist.

For linking to the built-in provided src/library_x.js files, I'm leaning on removing the current autolinking support to any of these, but how about if we dropped the --js-library and instead did "-lglut.js" ("-lsdl.js" and so on) to mean to link to the file "library_glut.js" that should exist somewhere in the filesystem in the -L paths (searching $EMSCRIPTEN/src as last one)? The suffix .js to -l would make it explicit that one is looking to link to a library_x.js file. That would be a more consistent way instead of adding heuristics that "-lGLESv2 should map to library_gl.js, -lGLUT should map to library_glut.js, -lal, -lsoft_oal should map to library_al.js, ..." and so on.

   Jukka

[Alon Zakai]

On Tue, Oct 28, 2014 at 2:27 PM, Jukka Jylänki <juj...@gmail.com> wrote:

I somewhat dislike the current behavior of -s USE_SDL2=1 at link time for downloading and building SDL2 on demand. This is because the scheme does not feel scalable. What should we do with other potential libraries like FreeType and Regal and zlib and other common ones that people are using?

I'm not sure why you feel this is not scalable? It would take literally just a matter of minutes to add e.g. zlib to emscripten-ports: make the repo on github, upload the content, write a short tools/ports/zlib.py script that builds it, add an option to settings.js (USE_ZLIB), and we're basically done. The amount of work is basically minimal - I don't see how any of those steps just mentioned could be removed.

If people want zlib, freetype, regal, etc., let's just add them. Happy to do it myself, just not sure which libraries people want. (Regarding regal, it doesn't work last I checked, so that's a different issue.)

 

For linking to the built-in provided src/library_x.js files, I'm leaning on removing the current autolinking support to any of these, but how about if we dropped the --js-library and instead did "-lglut.js" ("-lsdl.js" and so on) to mean to link to the file "library_glut.js" that should exist somewhere in the filesystem in the -L paths (searching $EMSCRIPTEN/src as last one)? The suffix .js to -l would make it explicit that one is looking to link to a library_x.js file. That would be a more consistent way instead of adding heuristics that "-lGLESv2 should map to library_gl.js, -lGLUT should map to library_glut.js, -lal, -lsoft_oal should map to library_al.js, ..." and so on.

On the one hand I like this. On the other hand though, it's a big change for users, and I worry about inconveniencing them for this.

- Alon

[Brion Vibber]

As a user, I'd also be inclined to want a separate package manager/installer for the ports rather than having them downloaded magically during compilation/linking.

Something like:

  # to install relevant packages

  emports install SDL2 zlib freetype

  # to compile, make sure to reference the libs explicitly

  emcc mystuff.c -lSDL2 -lz -lft -o mystuff.js

As for the linker flags and magic versus regular -lWhatever... Super-ideally, a magic build flag like --js-library or -lglut.js vs -lglut could be provided by something like pkg-config, and that would also 'just work' with autoconf-based projects.

For example, proper linking to a native libogg as installed on my Mac via Homebrew can be determined with pkg-config:

  $ pkg-config --libs ogg
  -L/usr/local/Cellar/libogg/1.3.2/lib -logg

so the special-ports version of glut could specify the magic --js-library in the pkg-config file which would be retrieved via `pkg-config --libs glut` inside an emconfigure or emmake environment.

Of course not every package comes with pkg-config metadata... Not sure if it'd be better to add custom entries or provide a similar interface on the port manager program (`emports --cflags SDL2` etc).

I'd also kind of prefer for the ports to be installed on a per-project basis as well, like npm packages, rather than globally, cause I like clean build environments. That might require explicitly setting a base path explicitly.

-- brion vibber (brion @ pobox.com / bvibber @ wikimedia.org)

On Tue, Oct 28, 2014 at 2:27 PM, Jukka Jylänki <juj...@gmail.com> wrote:

[Alon Zakai]

I added zlib to ports now, and tested it with BananaBread,

https://github.com/kripken/emscripten/commit/0681d355102c329cb594675f62648199ed8965cc

- Alon

[Mark Callow]

On 29/10/2014 07:23, Brion Vibber wrote:

As a user, I'd also be inclined to want a separate package manager/installer for the ports rather than having them downloaded magically during compilation/linking.

Something like:

  # to install relevant packages

  emports install SDL2 zlib freetype

  # to compile, make sure to reference the libs explicitly

  emcc mystuff.c -lSDL2 -lz -lft -o mystuff.js

Yes. This is much preferable to automagic downloads especially if they mean -S USE_SDL2=1.

... that would also 'just work' with autoconf-based projects.

Not everyone uses autoconf, which BTW, many people feel is a giant hack to paper over more fundamental issues such as failure of tools to follow standard practices.

On 29/10/2014 06:27, Jukka Jylänki wrote:

The current -s USE_SDL2=1 option is overloaded to download the SDL2 port on demand. If we reused -lSDL2 for that, it'd mean that we would spin off http downloads on demand. What if someone actually does have a library called libSDL2.a on his system that he built manually (a lot of existing users already do) and he instead wants to link to that? This means we should check the -L paths manually and only revert to downloading if none of those paths contained any of predefined names libSDL2.a, libSDL2.so, libSDL2.dylib, libSDL2.lib, SDL2.lib or SDL2.a. That would kind of work, and users who built SDL2 manually would then be able to link to the library they built themselves, and users who did not, would get the download on demand.

I don't understand what you mean by "check the -L paths manually". In most compile/link tools following a search path and using default locations when nothing is found on them is a fundamental part of the linker's task. Why so averse to having emlink do this? Anybody familiar with compilers will expect this behavior and will be surprised by its absence.

However this scheme is so brittle that it's just picking blood from your own nose, because it has the unfortunate effect that if there are any mistakes in a build (-L directories were passed wrong, an earlier build step had failed to produce a libSDL2.a file to some expected location, a build clean directive has been called to SDL2 before building main, and there's no dependency tracking, or something else), we would not be able to detect any of these, but instead would proceed to happily assuming that the user was not looking to link to his own SDL2, but we should download and build the ports SDL2 on demand. I can imagine someone mistaking a relative link path -L../../sdl2/lib when he was intending a -L../sdl2/lib or something else, and getting Emscripten downloading files off the web as a random result. This is not very robust nor explicit and I don't think we should go there.

How does -S USE_SDL=2 allow you to build dependencies?

I agree that automatic downloads greatly raise the cost of a mistake in a -L specification. The solution is explicit downloading as suggested by Brion and following standard compile/link practices, instead of inventing something entirely new.

There are those of us who wish to use tools beside autoconf and look with distaste at needing a special version of make. If you would just follow standard practices, those may not be necessary.

The same rationale applies for GLES, GLUT, GLFW and others, where I think we should not overload -lGLESv2, -lGLUT and so on to mean to silently link to Emscripten-provided ones if others are not present.

I thought libGLESv2 was built into the emsdk. Are you proposing making it subject to automagic downloading?

Please stop this and follow standard compiler practices. We are quite capable of knowing which libraries we want to use and running "emports install" to get them and specifying -lfoo on our command lines, which is what we have to do for native builds anyway.

As I wrote yesterday, the more you diverge from standard practices the harder it becomes to use other build tools.

[Mark Callow]

There are still some outstanding questions on this plus I have a new one.

On 29/10/2014 11:46, Mark Callow wrote:

... I don't understand what you mean by "check the -L paths manually". In most compile/link tools following a search path and using default locations when nothing is found on them is a fundamental part of the linker's task. Why so averse to having emlink do this?

?

... How does -S USE_SDL=2 allow you to build dependencies?

?

I agree that automatic downloads greatly raise the cost of a mistake in a -L specification. The solution is explicit downloading as suggested by Brion and following standard compile/link practices, instead of inventing something entirely new.

I want to clarify that this cost of a mistake is not a reason to use the -S USE_SDL=2

... I thought libGLESv2 was built into the emsdk. Are you proposing making it subject to automagic downloading?

?

And the new question ...

In  the emsdk 1.22.0 I am currently using, the existing SDL 1.x is included in the generated .js file even though my app is not calling it. Will this be changed? I'd hate to continue having both SDL 1.x and SDL 2 taking space.

[Alon Zakai]

Please file an issue with a testcase for SDL1 and 2 both being linked in, on e.g. browser.test_sdl2_jpeg I don't see that happen.

Regarding the linking debate, would using pkg-config be a solution that everyone is happy with? Using pkg-config is pretty standard stuff, and we could make it emit USE_SDL=2 etc. under the hood, as internal flags that users would never see.

- Alon

--

[Mark Callow]

On 07/11/2014 02:45, Alon Zakai wrote:

Please file an issue with a testcase for SDL1 and 2 both being linked in, on e.g. browser.test_sdl2_jpeg I don't see that happen.

I am still using Emscripten 1.22.0 together with gsathya's build of libSDL2. If I see evidence of SDL1 being included once the SDL2 port is mainstream, I'll open an issue.

Regarding the linking debate, would using pkg-config be a solution that everyone is happy with? Using pkg-config is pretty standard stuff, and we could make it emit USE_SDL=2 etc. under the hood, as internal flags that users would never see.

I don't really see how pkg-config helps. It just looks like another set of files you'll have to maintain. It certainly doesn't help my situation with MS visual studio configs.

If the ports are installed in a standard place within the Emscripten sdk, it seems much easier just to make emlink (emcc) search that place for includes and libraries as well as searching the location of the libraries that are included as standard with Emscripten.

juj's objection to library search paths in the end is only about the cost of a mistaken search path given as part of the command. I don't think it is that big a deal. Following the standard practice of other compilers is much more important and makes Emscripten usable with a wider range of build tools. The cost can be minimized by either requiring manual download of ports (my preference) or asking for confirmation when automatically starting a download.

You mentioned that the compiled port would be put into Emscripten's cache. How does that differ from generating a bitcode library from it and saving it in a directory searched by the linker?

While on the subject of library search paths, could you also please change emcc/emlink to not emit errors when it can't find libraries specified on the command line because they happen to be included in Emscripten. The most obvious examples of this are libGLESv2 and libEGL.

[Alon Zakai]

On Mon, Nov 10, 2014 at 10:39 PM, Mark Callow <callo...@artspark.co.jp> wrote:

On 07/11/2014 02:45, Alon Zakai wrote:

Please file an issue with a testcase for SDL1 and 2 both being linked in, on e.g. browser.test_sdl2_jpeg I don't see that happen.

I am still using Emscripten 1.22.0 together with gsathya's build of libSDL2. If I see evidence of SDL1 being included once the SDL2 port is mainstream, I'll open an issue.

Regarding the linking debate, would using pkg-config be a solution that everyone is happy with? Using pkg-config is pretty standard stuff, and we could make it emit USE_SDL=2 etc. under the hood, as internal flags that users would never see.

I don't really see how pkg-config helps. It just looks like another set of files you'll have to maintain. It certainly doesn't help my situation with MS visual studio configs.

I don't know enough about MS Visual Studio to know what would work well there. But for typical makefiles, I was thinking that

  `empkg-config sdl2 --cflags`

would be more standard and nice than

  -s USE_SDL=2

 

which is completely emscripten-specific.

If the ports are installed in a standard place within the Emscripten sdk, it seems much easier just to make emlink (emcc) search that place for includes and libraries as well as searching the location of the libraries that are included as standard with Emscripten. 

juj's objection to library search paths in the end is only about the cost of a mistaken search path given as part of the command. I don't think it is that big a deal. Following the standard practice of other compilers is much more important and makes Emscripten usable with a wider range of build tools. The cost can be minimized by either requiring manual download of ports (my preference) or asking for confirmation when automatically starting a download.

I don't have a strong opinion about this part. However, it would be additional work to do it (implement it, add testing, etc.), so in addition to deciding if we want to do it, we would need someone with time to spend on it.
 

You mentioned that the compiled port would be put into Emscripten's cache. How does that differ from generating a bitcode library from it and saving it in a directory searched by the linker?

The cache is automatically managed by emscripten, for example, we clear it on version updates automatically. This avoids e.g. a fastcomp update that uses different bitcode, in which case if the cache were not cleared, an error would happen. We could of course add such versioning of another location for the linker to search, but it adds some complexity to the design. (Again, though, I'm not opposed categorically.)
 

While on the subject of library search paths, could you also please change emcc/emlink to not emit errors when it can't find libraries specified on the command line because they happen to be included in Emscripten. The most obvious examples of this are libGLESv2 and libEGL.

Please file an issue with the details there (not sure exactly what you mean from that description).

- Alon
 

Regards

    -Mark

--

注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情報の使用を固く禁じております。エラー、手違いでこのメールを受け取られましたら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies.

--

[Stevie Trujillo]

On Thu, 13 Nov 2014 11:39:15 -0800
Alon Zakai <alon...@gmail.com> wrote:

> I don't know enough about MS Visual Studio to know what would work
> well there. But for typical makefiles, I was thinking that
>
> `empkg-config sdl2 --cflags`

Could emcmake/emconfigure define some pkg-config variables as explained
here?

https://autotools.io/pkgconfig/cross-compiling.html

Then build scripts might work without any changes?

[Jukka Jylänki]

2014-11-11 8:39 GMT+02:00 Mark Callow <callo...@artspark.co.jp>:

juj's objection to library search paths in the end is only about the cost of a mistaken search path given as part of the command. I don't think it is that big a deal. Following the standard practice of other compilers is much more important and makes Emscripten usable with a wider range of build tools. The cost can be minimized by either requiring manual download of ports (my preference) or asking for confirmation when automatically starting a download.

I am arguing against the practice that e.g. -lSDL2 would trigger a download-on-demand if none of the -L paths did not contain a matching file. That is not standard practice in any compiler environment that I know of. Like I mentioned before, I am also in favor of the method that downloading the ports is always manual and requires running a separate install command first. I don't like going the route of adding interactive confirmation prompts to the compilation progress, but perhaps we could mitigate the issue by matching -lXXX to a known port and issuing a hint message that explains that a library exists in the ports.

I am also arguing against string-matching "-lSDL -> library_sdl.js, -lm -> /dev/nul, -lal -> library_openal.js, -loal -> library_openal.js, -lsoft_oal -> library_openal.js, -lGLESv2 -> library_gl.js" and so on, because that will be error prone and a maintenance mess. I like the idea of reusing -lxxx.js to mean --js-library library_xxx.js and reusing -L directories to also search for paths in -lxxx.js, because the explicit suffix .js ensures there won't be any problems or backwards issues with existing builds systems.

[Brion Vibber]

On Mon, Nov 17, 2014 at 1:17 PM, Jukka Jylänki <juj...@gmail.com> wrote:

2014-11-11 8:39 GMT+02:00 Mark Callow <callo...@artspark.co.jp>:

juj's objection to library search paths in the end is only about the cost of a mistaken search path given as part of the command. I don't think it is that big a deal. Following the standard practice of other compilers is much more important and makes Emscripten usable with a wider range of build tools. The cost can be minimized by either requiring manual download of ports (my preference) or asking for confirmation when automatically starting a download.

I am arguing against the practice that e.g. -lSDL2 would trigger a download-on-demand if none of the -L paths did not contain a matching file. That is not standard practice in any compiler environment that I know of. Like I mentioned before, I am also in favor of the method that downloading the ports is always manual and requires running a separate install command first. I don't like going the route of adding interactive confirmation prompts to the compilation progress, but perhaps we could mitigate the issue by matching -lXXX to a known port and issuing a hint message that explains that a library exists in the ports.

+1 I like this. Explicit installation and a nice hint message if libraries matching a known signature not found.

I am also arguing against string-matching "-lSDL -> library_sdl.js, -lm -> /dev/nul, -lal -> library_openal.js, -loal -> library_openal.js, -lsoft_oal -> library_openal.js, -lGLESv2 -> library_gl.js" and so on, because that will be error prone and a maintenance mess. I like the idea of reusing -lxxx.js to mean --js-library library_xxx.js and reusing -L directories to also search for paths in -lxxx.js, because the explicit suffix .js ensures there won't be any problems or backwards issues with existing builds systems.

Is there any way we could have -lfoo link in a JS library called 'library_foo.js' if there's no matching libfoo.bc/.lib/.so/.dylib in the library path?

In that case, the JS library implementations could simply be renamed to match their native equivalents, and if port *installation* is manual there should be no surprises...

-- brion

[Alon Zakai]

All of this sounds very reasonable to me.

- Alon

--

[Jukka Jylänki]

2014-11-17 23:26 GMT+02:00 Brion Vibber <br...@pobox.com>:

Is there any way we could have -lfoo link in a JS library called 'library_foo.js' if there's no matching libfoo.bc/.lib/.so/.dylib in the library path?

In that case, the JS library implementations could simply be renamed to match their native equivalents, and if port *installation* is manual there should be no surprises...

If we overload -lfoo to link to library_foo.js, it interacts with more than just the ports (and actually it does not interact with the ports at all, since none of the ports we currently have contain any library_x.js files, they are currently all built only from C sources). There are several js library files in paths $EMSCRIPTEN/src/library_x.js. That is why I don't like the approach, and the explicit -lx.js is safer, because otherwise we would be overloading things like -luuid, -ltrace, -lsignals, -lsdl, -lpath, -lhtml5, -lgc, -lfs, -lbrowser, -lasync to name some of the more generic ones. Instead, if the suffix is required in -lx.js, then we can safely add -L$EMSCRIPTEN/src/ as a default system library, and walk towards a future path where we start dropping autolinking to each file $EMSCRIPTEN/src/library_x.js that we currently do.

So, my proposal is that:

   1. Add -L/path/to/emscripten/built/ports/output/libs to the list of hardcoded Emscripten system library directories as the LAST item.

   2. Never perform automatic download of a ports based on command line flags, but have a separate command line utility 'emports install sdl2' (the name and syntax can be discussed, just throwing some words in as example) which downloads and builds a port. After this is done, because of 1), it's only needed to pass the appropriate -lportName (e.g. -lSDL2) will link to that port.

   3. Add -L$EMSCRIPTEN/src to the list of hardcoded Emscripten system library directories as the LAST item after 1).

   4. Add -lx.js to mean "--js-library x.js" which is looked up in the directories specified by -L.

Some time later/as a separate step, we can then perform the following:

   5. Add an explicit blacklist feature to allow dropping autolinked JS libraries that user does not want: -s NO_JS_LIBS=['library_sdl.js','library_jansson.js']

   6. Start dropping some of the more uncommon JS libs from being autolinked (library_jansson.js, library_xlib.js, library_gc.js, and so on). Currently here is the list of libs each codebase always links to: https://github.com/kripken/emscripten/blob/master/src/modules.js#L428 . In my view, we should only autolink to libraries that are core Emscripten ones which only introduce symbols that can't conflict (emscripten_xxx ones). When people want to link to these, they need to issue a directive like -ljansson.js -lgc.js and so on.

The steps 1 and 2 should cover the ports, then steps 3 and 4 should make linking look more "standard" looking by reusing -l and -L, and 5 and 6 give future development paths for other libs that we currently hand-implement but in the future might compile from asm.js. With these, we should have a strategy that helps combat the list from https://github.com/kripken/emscripten/blob/master/src/modules.js#L428 from growing.

[Alon Zakai]

Sounds good, nice summary.

- Alon

--

[Mark Callow]

On 18/11/2014 07:50, Jukka Jylänki wrote:

If we overload -lfoo to link to library_foo.js, it interacts with more than just the ports (and actually it does not interact with the ports at all, since none of the ports we currently have contain any library_x.js files, they are currently all built only from C sources). There are several js library files in paths $EMSCRIPTEN/src/library_x.js. That is why I don't like the approach, and the explicit -lx.js is safer, because otherwise we would be overloading things like -luuid, -ltrace, -lsignals, -lsdl, -lpath, -lhtml5, -lgc, -lfs, -lbrowser, -lasync to name some of the more generic ones. Instead, if the suffix is required in -lx.js, then we can safely add -L$EMSCRIPTEN/src/ as a default system library, and walk towards a future path where we start dropping autolinking to each file $EMSCRIPTEN/src/library_x.js that we currently do.

So, my proposal is that:

   1. Add -L/path/to/emscripten/built/ports/output/libs to the list of hardcoded Emscripten system library directories as the LAST item.

   2. Never perform automatic download of a ports based on command line flags, but have a separate command line utility 'emports install sdl2' (the name and syntax can be discussed, just throwing some words in as example) which downloads and builds a port. After this is done, because of 1), it's only needed to pass the appropriate -lportName (e.g. -lSDL2) will link to that port.

   3. Add -L$EMSCRIPTEN/src to the list of hardcoded Emscripten system library directories as the LAST item after 1).

   4. Add -lx.js to mean "--js-library x.js" which is looked up in the directories specified by -L.

Thanks for summarizing so nicely.

I agree whole-heartedly with 1, 2 & 3.

I don't fully understand the objections to having -lfoo link to library_foo.js. That fact that the library may actually be named library_foo.js seems like an implementation detail. Why not use the standard names for the libraries, e.g. libGLESv2.js instead of library_glesv2.js? (Actually there is no such library so this is not a good example. I see only library_gl.js which I suppose has both GL & GLES entry points.) As for the name "overloading", it doesn't seem to be a problem for native compilers, why is it a problem for Emscripten? In a Unix-like system's /lib I see for example such generic names as libm, libutil, libcheck, libpthread, etc.

Some time later/as a separate step, we can then perform the following:

   5. Add an explicit blacklist feature to allow dropping autolinked JS libraries that user does not want: -s NO_JS_LIBS=['library_sdl.js','library_jansson.js']

   6. Start dropping some of the more uncommon JS libs from being autolinked (library_jansson.js, library_xlib.js, library_gc.js, and so on). Currently here is the list of libs each codebase always links to: https://github.com/kripken/emscripten/blob/master/src/modules.js#L428 . In my view, we should only autolink to libraries that are core Emscripten ones which only introduce symbols that can't conflict (emscripten_xxx ones). When people want to link to these, they need to issue a directive like -ljansson.js -lgc.js and so on.

+1.