Soft references freed too early

[brian.s...@googlemail.com]

Hi there,

I'm using a caching mechanism for some objects which have a small
memory footprint but are computation expensive to instantiate. Also,
I'm using a loop to prebuffer all those objects on application
startup, so that they are already in my cache when my UI starts
displaying them later.

Right now, my objects are so few and so small that I might just keep
them all in memory. I first coded my caching based on standard
references and it worked just fine. But my objectsd might grow in size
and/or number, so I started using java.lang.ref.SoftReference inside
my cache so that they won't cause an OutOfMemoryException.

I have to admit that I never used soft references before, but after
reading I'm rather sure I understood them correctly.

Actually, something like that happens:
Object 1 is cached
Object 2 is cached
Object 3 is cached
Object 4 is cached
Object 5 is cached
Objects 1 to 4 are cleared by the GC
Object 6 is cached
Object 7 is cached
Object 8 is cached
Object 9 is cached
Objects 5 to 8 are cleared by the GC
and so on.

So each time the UI want to display e.g. Object 1, it's not in the
cache and I must recostruct it which takes about 300ms. Because of
this, my caching is completely pointless.

I know that this is legal behavior for Soft References: The GC may or
may not clear them when he encounters them, ideally dependant on the
amount of free memory, and it must clear all of them before an out of
memory exception is thrown.

The only odd thing is, my memory is mostly free, the heap grows up to
3 MB but the GC keeps the real usage under 2 MB most of the time.
After some time running, when each object has been reconstructed 2 or
3 times (and the user has been annoyed by the time it took), suddently
the GC changes its mind and stops clearing the refenreces, and all
objects (about 40) are kept in memory and the UI works smoothly and my
caching makes sense.

Is there any way to bias the way those references are handled, e.g.
tell the GC that it is totally ok to have a heap of 4 MB? In the end,
thats what the GC does automatically after 30 seconds of doint it
wrong.

with best regards,
Brian Schimmel

[fadden]

On Jul 23, 5:50 am, "brian.schim...@googlemail.com"

<brian.schim...@googlemail.com> wrote:
> Actually, something like that happens:
> Object 1 is cached
> Object 2 is cached
> Object 3 is cached
> Object 4 is cached
> Object 5 is cached
> Objects 1 to 4 are cleared by the GC
> Object 6 is cached
> Object 7 is cached
> Object 8 is cached
> Object 9 is cached
> Objects 5 to 8 are cleared by the GC
> and so on.

I don't believe this is the desired behavior. Looking at tryMalloc()
in dalvik/vm/alloc/Heap.c, it initially calls gcForMalloc(false),
which should not be attempting to collect soft references. Only
later, when confronted with the possibility of throwing OOM, does it
call gcForMalloc(true).

Hmm. In dvmHeapSizeChanged() there's a "SOFT_REFERENCE_GROWTH_SLACK"
value that appears to be trying to strike a balance between expanding
without limit and collecting objects. If there's no cap we might fill
16MB with softly-reachable objects before we discard any of them,
which isn't desirable on a device that has multiple applications
resident in memory. It appears the function wants the GC to collect
"some" of the soft references, which sounds good, but it looks like
SR_COLLECT_SOME doesn't receive special treatment during the mark &
sweep. In the current implementation, it's all or nothing.

So my guess is you've got more than 128KB of softly-reachable objects
(i.e. the things that are softly referenced, and the things that are
only reachable through them). When the GC has to choose between
expanding the current heap size and chucking the soft references, it's
choosing to discard "some", which in the current implementation means
"all".

The observed behavior changes after you have a bunch of other stuff
allocated, because the "soft limit" is higher, and the GC doesn't have
to choose between expanding the heap and collecting the soft
references.

You might be able to work around this behavior by calling
dalvik.system.VMRuntime.setMinimumHeapSize() (which really shouldn't
be exposed in the API, but there it is). If you start it out at 4MB
that might prevent you from thrashing like this right after the app
starts up. I think the issue will come back if you manage to get
close to the soft limit again, but I don't think that's avoidable
without a change to the GC behavior. (There are some kluges you could
employ to suppress the behavior, like periodically grabbing hard
references to all cached items, allocating something large, discarding
it, and un-refing the cache entries, but that's ugly and might not
work anyway.)

I'll file a bug against the GC.

[brian.schimmel]

Hi fadden,

thanks for your answer and for looking things up in the dalvik source.
Sounds very much like you are right.

I didn't know of dalvik.system.VMRuntime.setMinimumHeapSize() but now
tried it. Invoking that with 4 MB or 5MB has no visible effect. But
invoking it with 8MB solves my problem.

So while my needs are fullfilled now, at least until Android 1.6 or
2.0 comes out and dropps support for setMinimumHeapSize, I had some
insights I want to share:

Interestingly, while doing some benchmarking it turned out that the
real heap size and heap usage is not increased by this. I called
setMinimumHeapSize jsut before the caching/prebuffering starts. I've
compiled my app with different values and used the eclipse DDMS view
to query the heap usage at three important times in my application
life cycle:

Step 1: Start the application and wait for the caching to be done,
then GC
Step 2: Open up another activity that displays a visual list, showing
the first 9 objects, then GC
Step 3: Scroll several times up and down through the list, until
scrolling is smooth, then GC

Without using setMinimumHeapSize:
Step 1: 3,258 used: 2,473
Step 2: 3,445 used: 2,571
Step 3: 3,508 used: 2,639

runtime.setMinimumHeapSize(4000000);
runtime.setTargetHeapUtilization(0.9f);
Step 1: 3,195 used: 2,485
Step 2: 3,508 used: 2,580
Step 3: 3,508 used: 2,583

runtime.setMinimumHeapSize(5000000);
Step 1: 3,250 used: 2,386
Step 2: 3,258 used: 2,517
Step 3: 3,383 used: 2,595

runtime.setMinimumHeapSize(8000000);
Step 1: 3,320 used: 2,480
Step 2: 3,320 used: 2,519
Step 3: 3,320 used: 2,531

As you can see, there is now big difference in those values. Anyway,
only when setting the heap to 8MB, none of my weakly referenced
objects where thrown away, and in Step 3 I instantly get a smooth
scrolling list.

The usage is always around 76%. I noticed there is a method
setTargetHeapUtilization and tried it once, supplying 0.9f as
parameter. As you can see in the above numbers, doing it results in
the same actual usage around 76%.

with best regards,
Brian schimmel

[fadden]

On Jul 24, 3:40 am, "brian.schimmel" <brian.schim...@googlemail.com>
wrote:

> Interestingly, while doing some benchmarking it turned out that the
> real heap size and heap usage is not increased by this. I called
> setMinimumHeapSize jsut before the caching/prebuffering starts. I've
> compiled my app with different values and used the eclipse DDMS view
> to query the heap usage at three important times in my application
> life cycle:

I would expect it to modify the "soft" limit, i.e. the point you're
allowed to grow to before a GC happens. The full size of the heap is
determined by the -Xmx "command line" argument, passed in during VM
creation by the app framework.

One thing that may be useful in your experiments: the system event log
receives a lot of detail about the state of the heap after a GC. You
can view it with the dalvik/tools/gclog.py script. A copy is
available from http://bigflake.com/gclog.py.txt .

(Set debug = True in handleGcInfo() to get the full dump.)