One of my pet RFE's is 4724038 "(fs) Add unmap method to
MappedByteBuffer".  This is currently in fifth place on the RFE voting list.
While I understand the problems with unmapping a specific buffer, the
suggestion in several of the comments to that RFE for a method to force
unmapping of unreachable buffers seems feasible. Especially given the
fix to 6417205 (fc) "Out of address space because mapped ByteBuffers are
not freed".

For my own needs, it would be no problem to ensure that any buffers,
that I wanted unmapped, were unreachable. Possibly the specification may
only promise to make a "best effort" to unmap as many buffers as possible.

Regards,
Mark Thornton

It was inevitable that someone would bring up this topic :-)   From your
mail it seems you have a  good appreciation of the issues as to why
there isn't an unmap method. The suggestion for a "big hammer" to
initiate reference discovery so that unreachable buffers are unmapped is
something that we have looked into. To some extent you can do this
yourself  by invoking the GC and this this will cause unreachable
buffers to be unmapped some time after. If you have a weak reference to
the buffer then it will be cleared and enqueued when the buffer becomes
finalizable. That may be good enough for some environments where you are
willing to take the performance hit. If your motive is to do some post
unmap clean-up (such as removing the file) then there is a timing issue
in that the buffer is unmapped after it becomes finalizable. One the
prototypes we did in this area provided a notification when the region
was unmapped and that could be useful for such cases. However it doesn't
address all requirements - for example it would be too expensive for
cases where you need to resize or you are sliding an aperture through a
huge file. For the later we have looked into supporting remapping but
the inability of some operating systems to support remapping as an
atomic operation means the solution isn't feasible everywhere. The fix
for 6417205 that you mention isn't pretty and essentially just invokes
the GC when there is insufficient memory to map the new region. In any
case, I'm interested to hear about your environment and if you would be
willing to take a hit to ensure that all unreachable buffers are unmapped.

-Alan.

As the originator of that REF, I suppose I am the most likely candidate.

As you say the difficulty here is the need to wait for an indeterminate
(and possibly release dependent) period to allow the clean-up thread to
actually unlap the buffers. A an implementation within NO would at least
know when the cleanup activity had completed.

My issues arose when implementing a geographic database. A typical map
of the UK results in a database size of around 900MB. Most of the time
it is used read only, but naturally has to be built and occasional
changes are made as well. Updates from the mapping agencies are received
at regular intervals (quarterly in the case of the main UK data set). We
are usually working in 32 bit Windows.

The first attempt at implementing this used a single map. When the
database needed to be enlarged, a new mapping was created to cover the
entire enlarged size. When this failed, I tried using multiple mappings,
typically of 64MB in each piece. This also failed (see bugs 4954921,
4938372).

Subsequently I wrote an alternative paging layer for the database which
doesn't use mapped files at all. I would like to return to the mapped
file approach and see if it can be made to work acceptably. The recent
fixes to 4938372 and 4954921 may make this feasible even without a
satisfactory way of unmapping unreachable buffers.

In my actual use, I only delete files which have been mapped within a
process when running test problems. Not being able to delete the file
litters my disk with often rather large relics of test runs. In this
regard I also hoped that the delete on exit mechanism would operate
during shut down AFTER files had been unmapped. The actual database
system does however attempt to truncate the database file to the length
actually used, and this operation fails (on Windows at least) for the
same reason as delete. This operation is not at all performance
critical. A workaround is to perform the truncation in a second process
which runs after the first has completed.

Some may ask why is it important to truncate the file at all given the
current cost of disk space. First, the file is distributed to our
clients, so excessive size is a nuisance. More importantly though is
that when used it is completely mapped into the process address space,
and files of this magnitude swallow an appreciable fraction of the
available address space (and we still need space for a sizeable heap).

I hope this gives some idea of what I have been attempting to do with
mapped files.

Mark Thornton