Garbage Collection very slow on Android with latest stable line
Chris Jimison
We have a game engine the uses V8 on Android based phones. I have
just upgraded our version of V8 to use the current stable SVN line and
I am seeing a HUGE slowdown on the GC. When the GC does a
MARK_COMPACTOR run I am seeing times jump up to 175 ms on a Nexus S
phone (one of the faster android based phones) however non
MARK_COMPACTOR times are at about 1-2 ms. The MARK_COMPACTOR sweep
happens about once every 10 - 15 seconds for us and it introduces a
very noticeable frame rate hitch.
So I have a couple of questions for the group.
1) Is there anything I can do to potentially amortize this cost accost
multiple frames (or GC calls)?
2) If not is there anyway I can speed this up?
Thank you so much for any help or insights.
-Chris
Vyacheslav Egorov
What version were you using before upgrading to HEAD and how long were
pauses at that version? If there was a major degradation we are very
interested in reproducing it.
Can you share a bit more about your use of V8?
Do you use it inside the browser or do you embed it into your
(C/C++/Java) game directly?
If you want to reduce GC cost we'll first need to understand what
exactly is causing (compacting) GCs.
It seems that promotion rate is not high (judging from 1-2ms
scavenging pauses) but it's hard to say anything without looking at GC
logs. (e.g. --trace-gc, --trace-gc-verbose ones; plus maybe some
additional debug prints in Heap::SelectGarbageCollector to see why V8
chooses MARK_COMPACTOR).
--
Vyacheslav Egorov, Software Engineer, V8 Team
Google Denmark ApS
> --
> v8-users mailing list
> v8-u...@googlegroups.com
> http://groups.google.com/group/v8-users
>
Alejandro F. Reimondo
I tried to build V8 (shell sample and library) on windows from trunk,
but the current SCONS support do not let us build for Android
devices yet.
Then I tried to build from Android git files; it is an old version
of V8 that it is said that it can be built but not using SCONS...
It did not worked on my installation (windows platform, after
a lot megabytes of downloads).
It would be nice to have support for Android via SCONS
on windows.
I was thinking in investing more time to get a build, but don�t
want to waste time if it will be incorporated soon to trunk.
Is there any plan to support Android platform?
Is there any person that have succeded to build
trunk version of V8 for Android on Windows 7 ?
thanks in advance, any information will be really appreciated,
Ale.
Jakob Kummerow
Hi,
I tried to build V8 (shell sample and library) on windows from trunk,
but the current SCONS support do not let us build for Android
devices yet.
Then I tried to build from Android git files; it is an old version
of V8 that it is said that it can be built but not using SCONS...
It did not worked on my installation (windows platform, after
a lot megabytes of downloads).
It would be nice to have support for Android via SCONS
on windows.
I was thinking in investing more time to get a build, but don´t
want to waste time if it will be incorporated soon to trunk.
Is there any plan to support Android platform?
Is there any person that have succeded to build
trunk version of V8 for Android on Windows 7 ?
thanks in advance, any information will be really appreciated,
Ale.
Mikhail Naganov
under OS X. The point is that you need a cross-compiler for ARM.
AFAIK, for windows, there is only one for cygwin. So you'll need a
working cygwin port of V8, because first V8 must be compiled for the
host architecture. But cygwin port of V8 is supported only by
enthusiasts, not by the core V8 team.
So I'd suggest just using linux for ARM compilation.
Chris Jimison
Sorry I took so long to get back to this.
> What version were you using before upgrading to HEAD and how long were
> pauses at that version?
January/February of 2010. The person who I inherited this code from
did not leave an SVN revision number anywhere I can find (and it looks
like they removed any reference files inside of V8 that had this
information).
> how long were pauses at that version?
difference is with this old version of V8 the exact same Javascript
code would not trigger a MARK_COMPRESS sweep. However when I upgraded
V8 I am now seeing the MARK_COMPRESS trigger every 10 sec or so which
introduces a very noticeable frame hitch.
> Can you share a bit more about your use of V8?
lower level logic (rendering with OpenGL, managing device touches,
etc, etc) and once a GL frame we send a large Command String to the V8
engine for processing (it has to be a string currently in order for us
to maintain compatibility with iOS which does not use V8) and once the
Javascript has completed the frame tick it will send back a Command
String for the C++ engine to process.
The general take away here is we are VERY string heavy in our
processing.
> It seems that promotion rate is not high (judging from 1-2ms
> scavenging pauses) but it's hard to say anything without looking at GC
> logs. (e.g. --trace-gc, --trace-gc-verbose ones; plus maybe some
> additional debug prints in Heap::SelectGarbageCollector to see why V8
> chooses MARK_COMPACTOR).
data log file generated on the device (however we are building using
ANT and I could have just screwed something up). I did put some debug
prints around the SelectGarbageCollector and found we are hitting the
following case:
// Is enough data promoted to justify a global GC?
if (OldGenerationPromotionLimitReached()) {
isolate_->counters()->gc_compactor_caused_by_promoted_data()-
>Increment();
return MARK_COMPACTOR;
}
Interesting notes:
One thing I am in the process of doing is moving all our string
process messaging into C++. This way the string objects can be more
tightly managed and we keep the uber long frame command string out of
V8 memory. This had a positive impact for us in that it increased the
amount of time from 10 sec to about 45 sec between MARK_COMPRESS
sweeps. However the MARK_COMPRESS still took about 200ms.
Another thing I tried was just forcing all GC to SCAVENGER. This had
a much better impact for our use case. While the SCAVENGER time jump
from 1~2 ms up to about 4~5 ms it did this only 1 per second and the
game ran very smoothly. I have now exposed a custom flag in our
version of V8 to turn off MARK_COMPRESS from Javascript code so that
games when they are in long wait situations (waiting for file IO or
Network Messaging) can turn on MARK_COMPRESS and if they are in a
performance critical area they can turn it off.
I am a little nervous about this direction. Can you foresee any major
issues I may run into with such a change? Some of the risk I am
thinking about are Heap fragmentation issues and slower object
allocation times.
Also any general suggestion/comments would be greatly appreciated and
thanks again for your help!
-Chris
Vyacheslav Egorov
> I enabled these flags but I am not seeing any additional TTY spew or a
> data log file generated on the device (however we are building using
> ANT and I could have just screwed something up).
It should just print things to stdout when enabled. You can try
editing flag-definitions.h to turn these flags on during compile time.
Alternatively you can call v8::V8::SetFlagsFromString to pass them to V8.
> The pauses where the same (about 1~2 ms per SCAVENGER Sweep) but the
> difference is with this old version of V8 the exact same Javascript
> code would not trigger a MARK_COMPRESS sweep.
So did the old version only trigger scavenges? Or did it trigger
scavenges and mark-sweeps but no mark-compacts?
> Another thing I tried was just forcing all GC to SCAVENGER. This had
> a much better impact for our use case. While the SCAVENGER time jump
> from 1~2 ms up to about 4~5 ms it did this only 1 per second and the
> game ran very smoothly. I have now exposed a custom flag in our
> version of V8 to turn off MARK_COMPRESS from Javascript code so that
> games when they are in long wait situations (waiting for file IO or
> Network Messaging) can turn on MARK_COMPRESS and if they are in a
> performance critical area they can turn it off.
If you disable MARK_COMPACTOR then your app might start accumulating
garbage because scavenger can only collect things in the young
generation. Also if you disable MARK_COMPACTOR entirely your app will
just crash with OOM when V8 hits the allocation limit in oldspace
because none of the GCs it will force will free any memory in the old
space.
You can try to run app with --never-compact to disable compaction
phase of the mark-sweep collector to see whether it improves situation
or not. This will lead to fragmentation but is much more stable then
disabling full collection entirely.
--
Vyacheslav Egorov
Chris Jimison
> editing flag-definitions.h to turn these flags on during compile time.
> Alternatively you can call v8::V8::SetFlagsFromString to pass them to V8
statements:
platform-posix.cc(10721): *** Scavenger GC Type Selected
platform-posix.cc(10721): **** Profile Complete <****
PerformGarbageCollection> 5
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): Scavenge 29.5 -> 29.2 MB,
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): 5 ms.
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): *** Scavenger GC Type Selected
platform-posix.cc(10721): **** Profile Complete <****
PerformGarbageCollection> 2
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): Scavenge 33.4 -> 32.5 MB,
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): 3 ms.
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): *** Old Generation Promotion Limit Reached,
MARK_COMPRESS GC Type selected
platform-posix.cc(10721): (149)Logging here
D/jni/NgAndroidApp.cpp(10721): (48)Finished v8 Garbage Collection
(0.179s)
platform-posix.cc(10721): **** Profile Complete <****
PerformGarbageCollection> 180
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): Mark-sweep 36.2 -> 26.8 MB,
platform-posix.cc(10721): (149)Logging here
platform-posix.cc(10721): 179 ms.
platform-posix.cc(10721): (149)Logging here
>So did the old version only trigger scavenges? Or did it trigger
>scavenges and mark-sweeps but no mark-compacts?
> You can try to run app with --never-compact to disable compaction
> phase of the mark-sweep collector to see whether it improves situation
> or not. This will lead to fragmentation but is much more stable then
> disabling full collection entirely.
While still too slow this leads me to think that it was not the
compression part that was slow but the sweep. I thought I was getting
hammered by all the memmoves but it looks like the sweep was the real
issue.
So a couple of questions for you all.
1) Since we are in a limited memory environment we have no issue with
setting a hard limit to how much memory V8 can use (Android will kill
the application if you try to allocate more memory then the OS says is
appropriate). Is there a way to create just one young generation to a
set size (say 16 megs), then force nothing but Scavenger sweeps and
just keep using the one heap? We would also need to know when we are
getting close to the high water mark and then start forcing GC's every
frame to get this our footprint down.
2) Is there a good way to limit how much data is swept durning the
Mark-Sweep phase. For example: I only want to sweep up to N objects,
or maybe spend X amount of time durning the sweep phase? It looks
like I am spending most of my cycles in: void
LargeObjectSpace::FreeUnmarkedObjects(). It would be cool if I could
just release N number of objects each GC call and then keep doing GC's
every frame until all the unmarked objects are released.
As always thanks for the help!
-Chris
Vyacheslav Egorov
> While still too slow this leads me to think that it was not the
> compression part that was slow but the sweep. I thought I was getting
> hammered by all the memmoves but it looks like the sweep was the real
> issue.
Lets additionally enable --trace-gc-nvp flag to get timings for
separate phases of mark-sweep GC.
> 1) Since we are in a limited memory environment we have no issue with
> setting a hard limit to how much memory V8 can use (Android will kill
> the application if you try to allocate more memory then the OS says is
> appropriate). Is there a way to create just one young generation to a
> set size (say 16 megs), then force nothing but Scavenger sweeps and
> just keep using the one heap? We would also need to know when we are
> getting close to the high water mark and then start forcing GC's every
> frame to get this our footprint down.
>
I am not sure I understand you question. But here is the number of concerns:
a) V8's heap has an unorthogonal structure so some spaces (code, map,
largeobject) are not managed by scavenger.
b) scavenger is a copying collector which performs best when there are
little to copy around. managing the whole heap (32 mb in your case)
with scavenger will completely destroy performance (it will be far
worse than MarkSweep).
c) Your heap is larger than 16 mb.
Also here is a small idea for you: try bumping max_semispace_size_
and initial_semispace_size_ (see heap.cc). Increase
initial_semispace_size_ to 1 or 2 mb and max_semispace_size_ to 4 or 8
mb and see how it affects GC timings.
> 2) Is there a good way to limit how much data is swept durning the
> Mark-Sweep phase. For example: I only want to sweep up to N objects,
> or maybe spend X amount of time durning the sweep phase? It looks
> like I am spending most of my cycles in: void
> LargeObjectSpace::FreeUnmarkedObjects(). It would be cool if I could
> just release N number of objects each GC call and then keep doing GC's
> every frame until all the unmarked objects are released.
This is possible. (We are doing similar stuff in our experimental GC
developed on experimental/gc branch but it is not thoroughly tested on
ARM yet, so I am not advising to use it).
FreeUnmarkedObjects is a very simple routine and I would not expect it
to consume a lot of time.
Can you add some debug prints into it? For example like this:
Index: src/spaces.cc
===================================================================
--- src/spaces.cc (revision 9090)
+++ src/spaces.cc (working copy)
@@ -2932,6 +2932,12 @@
void LargeObjectSpace::FreeUnmarkedObjects() {
+ double start = OS::TimeCurrentMillis();
+ int alive = 0;
+ int alivebytes = 0;
+ int freed = 0;
+ int freedbytes = 0;
+
LargeObjectChunk* previous = NULL;
LargeObjectChunk* current = first_chunk_;
while (current != NULL) {
@@ -2941,7 +2947,11 @@
heap()->mark_compact_collector()->tracer()->decrement_marked_count();
previous = current;
current = current->next();
+ alive++;
+ alivebytes += object->Size();
} else {
+ freed++;
+ freedbytes += object->Size();
// Cut the chunk out from the chunk list.
LargeObjectChunk* current_chunk = current;
current = current->next();
@@ -2962,6 +2972,14 @@
current_chunk->Free(current_chunk->GetPage()->PageExecutability());
}
}
+ double end = OS::TimeCurrentMillis();
+ PrintF("FreeUnmarkedObjects took %d ms\n"
+ " %d bytes in %d objects alive, %d bytes in %d objects freed\n",
+ static_cast<int>(end - start),
+ alivebytes,
+ alive,
+ freedbytes,
+ freed);
}
--
Vyacheslav Egorov, Software Engineer, V8 Team.
Google Denmark ApS.