Much Larger Javascript File Size With STL Vector

[awt]

Hi,

I wrote 2 simple programs to compare the size of the Javascript files generated in unoptimized mode for both array and vector:

1. Array version:

int main()

{

int arr1[2000];

add(arr1);

       return 0;

}

void add(int arr[])

{

  for(int i=0;i<2000;i++)

  {

 arr[i]=i;

  }

}

2. Vector version:

int main()

{

vector<int> v1;

add(v1);

        return 0;

}

void add(vector<int>& v)

{

for(int i=0;i<2000;i++)

{

v.push_back(i);

}

}

I was quite surprised that the size of the JS file generated for the array is only 203KB but the vector version took up 2393KB. Are there built-in optimizations for STL classes in Emscripten that I could leverage on to shrink the file size or do I have to implement my own vector classes? Any suggestions would be much appreciated. Thanks in advance.

[Floh]

2.3MB is indeed a bit suprising. I can only imagine there's a lot of actually unused code pulled in which probably isn't dead-code eliminated because optimization is off, so first step I would suggest is to try with optimization, and make sure that dead-code elimination actually works.

There *is* C/C++ runtime overhead of course, but for just std::vector that shouldn't (hopefully) be as much (e.g. my 'engine hello world' with printf output is about 617kByte unpacked / 150kByte gzipped).

Personally I'm not using stl containers but have rolled my own (not for code-bloat reasons, but because I don't want to depend on exceptions).

Cheers,

-Floh.

[Alon Zakai]

std::vector causes emscripten to pull in libc++, a full implementation of the C++ standard library. It then tries to strip out code it can prove is not used, but in practice just including std::vector is enough to keep quite a bit alive. std::iostream is worse.

Building with LTO may decrease the size somewhat (--llvm-lto 1).

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[awt]

Thanks Floh and Alon for your detailed replies. Really appreciate it.

I tried to use both --llvm-opts and --llvm-lto but interestingly, the size of the JS increased from 2202kb to 2328kb. These are my build parameters:

emcc vector.cpp -o hello.html --llvm-opts 3 --llvm-lto 1

Is there something wrong with the way that I am using the parameters above? I could use -Oz and -profiling because that would still give me debuggable code but just want to clarify if the -Oz optimization already includes --llvm-lto?

I am currently trying to port some legacy code which uses a only few STL containers such as vectors and maps. Is there a way for me to not build STL with the generated JS and replace the vector and map classes with my own implementation without touching the legacy code? Such as thru DEFAULT_LIBRARY_FUNCS_TO_INCLUDE for e.g.?

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Floh]

Hmm, LTO might also increase code size because it inlines more stuff (but I found that this inlining is quite important for performance), on the other hand it should a do pretty aggressive dead-code elimination.

Not sure if this helps in your case, but I'm currently compiling like this:

Object files: -Oz

Linker Stage : -Oz --llvm-lto 3 --closure 1

(plus some other stuff which shouldn't be relevant for size or performance). 

I've used -O2 before, and may decide to switch back depending on performance tests I'm doing from time to time.

[Jukka Jylänki]

Just for cross-referencing, https://github.com/kripken/emscripten/issues/2527 would help immensely here in tracking what causes big output sizes and why, although we don't currently have anyone working on such a tool.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

[Kevin Cheung]

Thanks for the link, I will check out tools/find_bigfuncs.py when I
have the time.

Just to bring out my previous question in case it got lost in the thread:

I am currently trying to port some legacy code which uses only a few

STL containers such as vectors and maps. Is there a way for me to not
build STL with the generated JS and replace the vector and map classes
with my own implementation without touching the legacy code? Such as
thru DEFAULT_LIBRARY_FUNCS_TO_INCLUDE for e.g.?

>>>>> an email to emscripten-disc...@googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "emscripten-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to emscripten-disc...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "emscripten-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/emscripten-discuss/WTHJTJcsWPA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> emscripten-disc...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
In His grace,
Kevin Cheung

[Alon Zakai]

If you use STL things and the linker sees unresolved symbols, then STL will be linked for you. If you implement yourself parts of STL, then there will be no unresolved symbols and it should not include STL.

- Alon

[Kevin Cheung]

Thanks for your reply Alon. Unfortunately, I am inheriting some legacy
code which I do not hope to touch, so is it possible to still keep the
code that references a vector intact e.g.:

#include <vector>

int main()
{
std::vector<int> a;
a.push_back(5);
}

but force Emscripten to not link with STL (perhaps thru a compiler
setting) and at the same time, put in my own Vector implementation?

[Alon Zakai]

As mentioned above, if you link with your own implementation of the STL, emscripten will not link in the emscripten STL (libc++). It only links it if it sees it is needed. But if you want to forcefully disable linking of the STL, on the incoming branch you can use

EMCC_FORCE_STDLIBS=libc,libcextra,libcxxabi,gl EMCC_ONLY_FORCED_STDLIBS=1 emcc [..]

which forces the linker to use those stdlibs (can remove gl for example, if not using opengl) and no others.

- Alon

[Kevin Cheung]

Thanks Alon, I have verified that Emscripten indeed picks up my own
implementation of the vector instead of the STL when it is able to
find the former.