Integrate SDL2 into Emscripten

[Sathyanarayanan Gunasekaran]

Hello all,

I've been working on porting SDL2 to emscripten with help from
Daft-Freak. It's in a usable state and almost all of the SDL2 tests[0]
have been ported over. The changes are going to be up streamed to SDL2
soon[1].

Here's the code -
https://github.com/gsathya/SDL-emscripten/

Here's more information on how to use it -
http://gsathya.in/blog/mozilla/sdl2-emscripten.html

I think SDL2 is used enough that emscripten benefits from integrating
the cross compiled SDL2 into master, but this could to increase in
size(the cross compiled bitcode is 1.4M). Do we want to integrate this
into emscripten? Thoughts?

Thanks,
--Sathya

[0]: https://github.com/gsathya/SDL-emscripten/wiki/SDL-Tests
[1]: http://lists.libsdl.org/pipermail/sdl-libsdl.org/2014-July/095475.html

[Alon Zakai]

I tend to agree, we should have SDL2 as the recommended cross-platform toolkit for people to use (alongside Jukka's HTML5-specific API), in which case it makes sense to have it in trunk without any hassle for people to get it.

What is the total code size we would be including? (Just the core + emscripten parts, not other backends)

And just out of curiosity, what is the code size (JS) when you take one of the simple SDL tests for which you have a 1.2 and 2.0 version? I assume 2.0 would be larger, but hopefully not by much, I am hoping LLVM dead code elimination would remove almost all of it if little is used.

Assuming we merge to trunk, we should get in and automate relevant tests from their test suite, depending on their number and time to execute.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark Callow]

On 08/07/2014 04:04, Sathyanarayanan Gunasekaran wrote:

I think SDL2 is used enough that emscripten benefits from integrating
the cross compiled SDL2 into master, but this could to increase in
size(the cross compiled bitcode is 1.4M). Do we want to integrate this
into emscripten? Thoughts?

I think SDL2 should be integrated to master. The currently included SDL version is not useful for porting as there is no native version.

I am still trying to get the test version to work in my project. I was off for a couple of days but am about to resume the effort.

Regards

    -Mark

--

注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情報の使用を固く禁じております。エラー、手違いでこのメールを受け取られましたら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies.

[Bruce Mitchener]

On Tue, Jul 8, 2014 at 12:49 PM, Mark Callow <callo...@artspark.co.jp> wrote:

On 08/07/2014 04:04, Sathyanarayanan Gunasekaran wrote:

I think SDL2 is used enough that emscripten benefits from integrating
the cross compiled SDL2 into master, but this could to increase in
size(the cross compiled bitcode is 1.4M). Do we want to integrate this
into emscripten? Thoughts?

I think SDL2 should be integrated to master. The currently included SDL version is not useful for porting as there is no native version.

I am still trying to get the test version to work in my project. I was off for a couple of days but am about to resume the effort.

Maybe I'm nuts, but why effectively create a fork of SDL2? Why not have this merge upstream?

A build bot could still be set up to build and pull SDL2 and run tests using the current / incoming / whatever version of emscripten.

 - Bruce

[Sathyanarayanan Gunasekaran]

Hi Alon,

On Mon, Jul 7, 2014 at 7:23 PM, Alon Zakai <alon...@gmail.com> wrote:
> I tend to agree, we should have SDL2 as the recommended cross-platform
> toolkit for people to use (alongside Jukka's HTML5-specific API), in which
> case it makes sense to have it in trunk without any hassle for people to get
> it.
>
> What is the total code size we would be including? (Just the core +
> emscripten parts, not other backends)

Total code size of the ported SDL2? The cross compiled
bitcode(libSDL2.a) is 1.4M

> And just out of curiosity, what is the code size (JS) when you take one of
> the simple SDL tests for which you have a 1.2 and 2.0 version? I assume 2.0
> would be larger, but hopefully not by much, I am hoping LLVM dead code
> elimination would remove almost all of it if little is used.

test_sdl2_canvas_blank
104K test.html
5.1M test.js

test_sdl_canvas_blank
104K test.html
372K test.js

This is pretty crazy.

> Assuming we merge to trunk, we should get in and automate relevant tests
> from their test suite, depending on their number and time to execute.

I've ported over most of the SDL1.x emscripten tests[0], which we can
use for testing. Only some of the SDL2 tests can be automated.

Thanks,
--Sathya

[0]: https://github.com/gsathya/SDL-emscripten/wiki/Emscripten-tests

[Sathyanarayanan Gunasekaran]

Hi,

On Mon, Jul 7, 2014 at 10:52 PM, Bruce Mitchener
<bruce.m...@gmail.com> wrote:
>
> Maybe I'm nuts, but why effectively create a fork of SDL2? Why not have this
> merge upstream?

It is going to be merged upstream. See -
http://lists.libsdl.org/pipermail/sdl-libsdl.org/2014-July/095475.html
https://twitter.com/icculus/status/485639842823700481

> A build bot could still be set up to build and pull SDL2 and run tests using
> the current / incoming / whatever version of emscripten.

Good idea! Although not sure if we want to run the SDL2 tests at all.
The emscripten SDL1.x tests have been ported over to SDL2[0][1], so
that should be good enough IMHO.

Thanks,
--Sathya

[0]: https://github.com/gsathya/SDL-emscripten/wiki/Emscripten-tests

[1]: https://github.com/gsathya/emscripten/tree/sdl

> - Bruce

[Mark Callow]

On 08/07/2014 04:04, Sathyanarayanan Gunasekaran wrote:

Here's more information on how to use it -
http://gsathya.in/blog/mozilla/sdl2-emscripten.html

It falls at the first hurdle. "C compiler cannot create executables". What am I doing wrong?

$ emconfigure ./configure --host=asmjs-unknown-emscripten --disable-assembly --disable-threads --enable-cpuinfo=false CFLAGS="-O2"
cygwin warning:
  MS-DOS style path detected: C:/Program Files/Emscripten/clang/e1.21.0_64bit/pnacl-clang
  Preferred POSIX equivalent is: /cygdrive/c/Program Files/Emscripten/clang/e1.21.0_64bit/pnacl-clang
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
DEBUG    root: JAVA not defined in ~/.emscripten, using "java"
checking build system type... x86_64-unknown-cygwin
checking host system type... asmjs-unknown-emscripten
checking how to print strings... printf
checking for asmjs-unknown-emscripten-gcc... /cygdrive/c/Program Files/Emscripten/emscripten/1.21.0/emcc
checking whether the C compiler works... no
configure: error: in `/cygdrive/c/Users/callow_mark/GitHub/SDL-emscripten':
configure: error: C compiler cannot create executables
See `config.log' for more details
ERROR    root: Configure step failed with non-zero return code 77! Command line: ['./configure', '--host=asmjs-unknown-emscripten', '--disable-assembly', '--disable-threads', '--enable-cpuinfo=false', 'CFLAGS=-O2']
 

[Jukka Jylänki]

This looks like hitting an unsupported feature: The library has not been built with Windows via configure before, and cygwin+configure scripts being what they are, it's trying to compile an executable during configure time to deduce whether the target supports some feature.

I see three options here:

  1) Migrate to configuring via CMake instead of configure. This has the benefit of being more compatible across platforms since we maintain a CMake toolchain script for cross-compiling CMake-based builds.

  2) Try to develop a machinery to SDL2 configure script that it would be able to compile the checks with emscripten and run those e.g. in node.

  3) In the configure scripts, when Emscripten is being targeted, remove all such compilation attempts and write statically into the configure script the platform features and .h files that Emscripten compiler has. This would effectively work around not being able to build+run during configure.

I would recommend going for option 1), because CMake is a better cross-platform configure tool than configure.

--

[Chad Austin]

On Mon, Jul 7, 2014 at 12:04 PM, Sathyanarayanan Gunasekaran <gsa...@mozilla.com> wrote:

I've been working on porting SDL2 to emscripten with help from
Daft-Freak. It's in a usable state and almost all of the SDL2 tests[0]
have been ported over. The changes are going to be up streamed to SDL2
soon[1].

Nice work!

I think SDL2 is used enough that emscripten benefits from integrating
the cross compiled SDL2 into master, but this could to increase in
size(the cross compiled bitcode is 1.4M). Do we want to integrate this
into emscripten? Thoughts?

My preference is to replace the existing "SDL" with this port, or at least make SDL 2 the default.  I tried to port one SDL 1.2 game to Emscripten, and got bit by missing functionality or crazy errors caused by the JavaScript implementation of SDL 1.3 returning different values than the header indicated.

Plus, I think it's a good idea to align work with the future, and SDL 2, being newer and permissively licensed, is definitely the future.  :)

[Sathyanarayanan Gunasekaran]

Hi,

On Mon, Jul 7, 2014 at 11:07 PM, Sathyanarayanan Gunasekaran
<gsa...@gatech.edu> wrote:
>
>> And just out of curiosity, what is the code size (JS) when you take one of
>> the simple SDL tests for which you have a 1.2 and 2.0 version? I assume 2.0
>> would be larger, but hopefully not by much, I am hoping LLVM dead code
>> elimination would remove almost all of it if little is used.
>
> test_sdl2_canvas_blank
> 104K test.html
> 5.1M test.js
>
> test_sdl_canvas_blank
> 104K test.html
> 372K test.js
>
> This is pretty crazy.
>

Ah, running with -O2 reduces the size -
test_sdl2_canvas_blank
104K test.html
1.3M test.js

Thanks,
--Sathya

[Alon Zakai]

So it goes from 372K to 1.3MB, when moving to SDL2? That seems very high. Not on topic for this thread, but we should investigate why it's so big.

- Alon

[Alon Zakai]

As mentioned this is being done with upstream.

But the question here is whether we should import a copy SDL2 into our repo, or use a separate repo, where people would need to download it separately. I don't think the separate repo can be SDL upstream since we might need some fix before it gets reviewed and merged there, so we do need a fork. So the question is, should our fork be in the main emscripten repo, or in another.

There has been discussion in the past about a "package manager" or "package repo" system, specifically empkg by LCID Fire. That approach didn't get much interest at the time from the community (and I admit to have been too busy back then to get into it), and seems to have been taken down meanwhile. But perhaps this is a good time to revisit it.

In summary, here are some options here:

1. Merge SDL2 into our main repo. People just grab emscripten, and can then build with -lSDL2 and it works. This is what we do with libc, libc++abi, libc++ (and some JS-implemented libraries like SDL1.3, glew, glut, etc.). This seems easiest for everyone.

2. Keep our SDL2 fork in a separate repo by itself. People need to grab it manually and link with it.

3. Keep our SDL2 fork in a separate repo of ported projects, that has some system for making it convenient to use. That is, there is an easy way to get and build and integrate with emcc any of the projects there.

- Alon

On Mon, Jul 7, 2014 at 10:52 PM, Bruce Mitchener <bruce.m...@gmail.com> wrote:

[Chad Austin]

On Tue, Jul 8, 2014 at 2:20 PM, Alon Zakai <alon...@gmail.com> wrote:

So it goes from 372K to 1.3MB, when moving to SDL2? That seems very high. Not on topic for this thread, but we should investigate why it's so big.

As you're quite familiar, I would appreciate any investigation into how we can make the generated code smaller.  :)
 

--

Chad Austin

Technical Director, IMVU

http://engineering.imvu.com

http://chadaustin.me

[Mark Callow]

On 09/07/2014 06:32, Alon Zakai wrote:

In summary, here are some options here:

1. Merge SDL2 into our main repo. People just grab emscripten, and can then build with -lSDL2 and it works. This is what we do with libc, libc++abi, libc++ (and some JS-implemented libraries like SDL1.3, glew, glut, etc.). This seems easiest for everyone.

2. Keep our SDL2 fork in a separate repo by itself. People need to grab it manually and link with it.

3. Keep our SDL2 fork in a separate repo of ported projects, that has some system for making it convenient to use. That is, there is an easy way to get and build and integrate with emcc any of the projects there.

I prefer option 1 and as Chad said, it should replace SDL 1.3 since that is now a non-existent version in native. You won't find it at libsdl.org. I think 2 is unacceptable given that we need to replace SDL 1.3.

[Bruce Mitchener]

On Wed, Jul 9, 2014 at 4:32 AM, Alon Zakai <alon...@gmail.com> wrote:

As mentioned this is being done with upstream.

But the question here is whether we should import a copy SDL2 into our repo, or use a separate repo, where people would need to download it separately. I don't think the separate repo can be SDL upstream since we might need some fix before it gets reviewed and merged there, so we do need a fork. So the question is, should our fork be in the main emscripten repo, or in another.

There has been discussion in the past about a "package manager" or "package repo" system, specifically empkg by LCID Fire. That approach didn't get much interest at the time from the community (and I admit to have been too busy back then to get into it), and seems to have been taken down meanwhile. But perhaps this is a good time to revisit it.

In summary, here are some options here:

1. Merge SDL2 into our main repo. People just grab emscripten, and can then build with -lSDL2 and it works. This is what we do with libc, libc++abi, libc++ (and some JS-implemented libraries like SDL1.3, glew, glut, etc.). This seems easiest for everyone.

libc, libc++abi and libc++ are critical the functioning of emscripten. SDL2 isn't. Any version here will eventually diverge from the upstream.  SDL isn't part of the "emscripten product" and there's no harm to having people pull it separately.  If there are difficulties in that area, they should get resolved by improving the toolchain.

If people *really* want it packaged together, that sounds like something that could be done as part of emsdk and some new buildbot tasks to ensure that things are kept functioning.

As for the JS-implemented libraries ... I think that the library_gc.js should probably die in favor of people just getting and using Boehm and SDL 1.3 should be removed.  While those were fine for a research project, I don't think they rise to the quality, performance or usefulness to be part of the "emscripten product" or platform.

2. Keep our SDL2 fork in a separate repo by itself. People need to grab it manually and link with it.

Nothing wrong with a separate repo as long as it is intended to only have differences from upstream on a short term basis.

 

3. Keep our SDL2 fork in a separate repo of ported projects, that has some system for making it convenient to use. That is, there is an easy way to get and build and integrate with emcc any of the projects there.

This sounds good too.

 - Bruce

[Mark Callow]

On 08/07/2014 23:18, Jukka Jylänki wrote:

I would recommend going for option 1), because CMake is a better cross-platform configure tool than configure.

Sathya,

Will you modify SDL's cmake files so they can be used to configure Emscripten builds?

[Jukka Jylänki]

I would recommend we don't copy SDL2 source tree to Emscripten repository, because we will lose history, and we'll become detached of SDL2 updates. Instead, I'd recommend we write a script that git clones+pulls+checkouts+builds the SDL2 repository from a known good fork+branch. This gives an easy update path for each developer, but also keeps it simple (run one script at first setup) if people just want the latest master.

--

[Mark Callow]

On 09/07/2014 20:17, Jukka Jylänki wrote:

I would recommend we don't copy SDL2 source tree to Emscripten repository, because we will lose history, and we'll become detached of SDL2 updates. Instead, I'd recommend we write a script that git clones+pulls+checkouts+builds the SDL2 repository from a known good fork+branch. This gives an easy update path for each developer, but also keeps it simple (run one script at first setup) if people just want the latest master.

+1.

[Sathyanarayanan Gunasekaran]

Hi,

On Tue, Jul 8, 2014 at 9:20 PM, Mark Callow <callo...@artspark.co.jp> wrote:
> On 08/07/2014 23:18, Jukka Jylänki wrote:
>
>
> I would recommend going for option 1), because CMake is a better
> cross-platform configure tool than configure.
>
> Sathya,
>
> Will you modify SDL's cmake files so they can be used to configure
> Emscripten builds?
>

I'm afraid I don't know much about CMake. Jukka is looking into this
and has made some progress already! See -
https://github.com/gsathya/SDL-emscripten/pull/7
https://github.com/gsathya/SDL-emscripten/wiki/Building-with-CMake

Thanks,
--Sathya

> Regards
>
> -Mark
>
> --
> 注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合が有ります。正式なメール受信者では無い場合はメール複製、
> 再配信または情報の使用を固く禁じております。エラー、手違いでこのメールを受け取られましたら削除を行い配信者にご連絡をお願いいたし ます.
>
> NOTE: This electronic mail message may contain confidential and privileged
> information from HI Corporation. If you are not the intended recipient, any
> disclosure, photocopying, distribution or use of the contents of the
> received information is prohibited. If you have received this e-mail in
> error, please notify the sender immediately and permanently delete this
> message and all related copies.
>

[Alon Zakai]

Ok, looks like at least 2 people feel strongly about this, and I see their point. I created

https://github.com/emscripten-ports

as a github org where we can put repos of ports like SDL2. The next question is how to automate the process for people to get those projects. One option is to write a new package manager tool, another that came up in a talk today with Jukka and Sathya was to use the existing emsdk tool. That seems simpler, so worth trying. We should still not make it a requirement to use the sdk, though, should be clear instructions to build manually as well.

- Alon