The JVM can segfault due to truncating files under mmap... and the API says this is 'undocumented'.

459 views
Skip to first unread message

Kevin Burton

unread,
Oct 4, 2013, 9:20:43 PM10/4/13
to mechanica...@googlegroups.com
I just found out that the following chunk of code will segfault a JVM:

        int max = 100;
        String path = "/tmp/test.txt";
        FileOutputStream fos = new FileOutputStream( path );
        fos.write( new byte[max] );
        fos.close();
        FileInputStream fis = new FileInputStream( path );
        FileChannel channel = fis.getChannel();
        MappedByteBuffer buff = channel.map( FileChannel.MapMode.READ_ONLY, 0, max );
        assertEquals( 0, buff.getInt() );
        fos = new FileOutputStream( path );
        fos.write( new byte[0] );
        fos.close();
        assertEquals( 0, buff.getInt() );

You will end up with this super fun message:

Exception in thread "main" java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code

this is documented due to mmap. I think I had read about it but when I implemented this API forgot about it... 


SIGBUS
Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file).

But the JDK doesn't really clear this up:

Many of the details of memory-mapped files are inherently dependent upon the underlying operating system and are therefore unspecified. The behavior of this method when the requested region is not completely contained within this channel's file is unspecified. Whether changes made to the content or size of the underlying file, by this program or another, are propagated to the buffer is unspecified. The rate at which changes to the buffer are propagated to the file is unspecified. 

Considering Java has taken MASSIVE steps to prevent this sort of thing, it's amazing that a regular use using the standard API for mmap is totally open to segfaulting the JVM at any time.

In fact, this could be consider a pretty active attack against the JVM if you know the underlying details.  I'm sure on *NIX pretty much every Java app is prone to this fun and you could DoS apps just by truncating an underlying file. 

Kevin Burton

unread,
Oct 4, 2013, 9:25:47 PM10/4/13
to mechanica...@googlegroups.com
Actually... I misspoke... it's a SIGBUS... which throws an InternalError...

I mean... it's POSSIBLE to clean up the JVM after this but it looks like the JVM is picking this up at weird places.. possibly after safepoints?

So you'd have to catch InternalError in your code and clean up after it... which I don't think is really possible.

So in practice, most apps would probably shut down.

Kevin

Peter Lawrey

unread,
Oct 5, 2013, 4:15:52 AM10/5/13
to mechanica...@googlegroups.com

Which OS version do you have? I have found these sort of edge cases dissappear with newer kernel versions. ;)

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Reply all
Reply to author
Forward
0 new messages