An Unsafe safety quiz question

[Gil Tene]

There has been much chatter about use of unsafe, and how smart people that want to run with scissors should be able to do so. Unsafe is there. It's a fact, and until someone is wise (or unwise) enough to cause rapid deprecation in it's use, it will be probably continue to be used by many performance-centric people.

So rather than argue about it's accessibility. I thought it might be useful to demonstrate some of it's danger by exploring some of the intricate issues about rules that unsafe-using code must be aware of and follow, and the dangers of programming without knowing what those rules are.

Remember, by using Unsafe, you've usually already decided to break compiler-imposed type safety by type casting what is supposed to be a completely opaque object and looking inside it (the label "unsafe" of the stuff you are calling is your string hint that this is so). Once you do that, what rules are left to follow? What do you need to be careful or afraid of? (and I'm not referring to the danger that the cool call you are making with two long arguments may change it's meaning in the next JVM patch release).

Here is a simple "unsafe safety" quiz question: What are the rules being broken by the following piece of code? I mean what can specifically go wrong here? And yes, there are other ways to write the same with unsafe that don't quite break the same rules, but what is it that makes them safer?

This is the sort of code that will run without a problem and pass most QA and stress tests, but has a horrible memory corruption bug in it. It's the fact that most people can't articulate the various ways this can go badly wrong (including the 1:1,000,000 chance russian roulette problem hidden in here), and then take the approach of "if it's run reliably in my environment for a few hours, it's probably ok".

Have at it!

import sun.misc.Unsafe;

import java.lang.reflect.Field;

public class UnsafeLongArrayExample {

    // What (specifically) is wrong with this code?

    // If there are better unsafe ways to write this. What (specifically, other than style and such) makes

    // them safer?

    private static final Unsafe unsafe;

    private static final int arrayBaseOffset;

    private static final long internalLongArrayOffset;

    static

    {

        try

        {

            Field field = Unsafe.class.getDeclaredField("theUnsafe");

            field.setAccessible(true);

            unsafe = (Unsafe)field.get(null);

            arrayBaseOffset = unsafe.arrayBaseOffset(long[].class);

            Field arrayField = UnsafeLongArrayExample.class.getDeclaredField("internalLongArray");

            internalLongArrayOffset = unsafe.objectFieldOffset(arrayField);

        }

        catch (Exception e)

        {

            throw new RuntimeException(e);

        }

    }

    final long[] internalLongArray;

    UnsafeLongArrayExample(int size) {

        this.internalLongArray = new long[size];

    }

    long[] getArray() {

        return internalLongArray;

    }

    // A faster way to get to an element (faster than value = getArray()[index];)

    long getArrayElement(int index) {

        long baseAddress = unsafe.getLong(this, internalLongArrayOffset);

        long elementAddress = baseAddress + arrayBaseOffset + (index * 8);

        return unsafe.getLong(elementAddress);

    }

    // A faster way to set an element (faster than getArray()[index] = value;)

    void setArrayElement(int index, long value) {

        long baseAddress = unsafe.getLong(this, internalLongArrayOffset);

        long elementAddress = baseAddress + arrayBaseOffset + (index * 8);

        unsafe.putLong(elementAddress, value);

    }

}

 

[Wojciech Kudla]

Having no means to run code at the moment my best educated guess would be you may run into trouble after the first copying/compacting gc cycle because the logic of accessing internalLongArray which is stored on heap crosses the boundary of a single safe point.
On the other hand, knowing Gill also tells me this is much more twisted puzzle than that. :-)

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Sand Stone]

This is a great post to have an open discussion on this. 

>   final long[] internalLongArray;

In this specific Java example, there are too many places the code could go wrong: read garbage and corrupt memory for lack of boundary checking and gc issue for small array sizes. The root of all evil is to manipulate a memory structure created by "someone else" (in this case JDK/JVM).   Programming Unsafe should obey the rules of C programming in general. And one has to live with the consequence of C programming. 

For my usage, I created offheap data structures myself and to GC, it's just a long value in a GC-aware Java object. So effectively I am using Unsafe along with the rest of core JDK as a portable C runtime. The code has been running in production over a year, on various versions of JDK1.6 and 1.7 on various versions of Linux and Windows. It's stable and robust. I did try on JDK1.8 too, though no production experience. 

Is it worth it? Why not using JNI? Or heck just C or C++. 

While I cannot share much what I do actually at the moment, I can say that so far it seems worth the pain. The offheap memory can grow to 100s of GBs, and with almost no GC pause, ever. It's better than programming in C as I can still organize the code in OO fashion (namespace, class and all that) and have a portable executable between Linux, and Windows. And of course Mac, the dev. environment mostly.  Compared to C++, I have the love-n-hate relationship with its template design and not to mention the baroque STL (Boost). Of course the more advanced programming needs to deal with i/o, locks, threads and memory model are still C++ compiler and platform specific. 

So if one knows a high performance and portable language runtime as rich and proven as Java (with Unsafe :-), please share it.  Go is somewhat promising, not it at the moment. 

Java with Unsafe seems to be the only game in town as a proven high performance server side portable runtime (a mouthful, I know). With better SIMD instructions support, it will be almost perfect (assume future JDKs don't remove the iCMS mode, or fix the CMS). 

My two cents.  

 

--

[Nitsan Wakart]

Here goes:

1. Computing the absolute offset in the array, then accessing memory directly is problematic as the object may have moved, one should compute the offest within the reference to the array and call putLong(this.internalLongArray, offestWithinArray, value) similarly on the get method one should use getLong(this.internalLongArray, offestWithinArray)

2. The code below is further broken by assuming the reference size is a long, so reading getLong(this, internalLongArrayOffset) will lead to reading an extra 4 bytes of random data in a 32bit VM. If you want to get a reference, use Unsafe.getObject(), but there's really no point in this case, we have the field.

3. Finally, to access an offset within an array one should verify arrayIndexScale and shift accordingly. So the computation of the offestWithinArray is potentially broken.

To read a correct version of all of the above look no further then j.u.c.AtomicLongArray :-)

Gil, I await your verdict, please don't hit me too hard...

-Nitsan

---

From: Wojciech Kudla <wojciec...@gmail.com>
To: Gil Tene <g...@azulsystems.com>
Cc: mechanica...@googlegroups.com
Sent: Sunday, June 23, 2013 8:19 AM
Subject: Re: An Unsafe safety quiz question

[Rüdiger Möller]

1. better use putLong( longarray, baseOff+index*longScale, value ) and getLong(longarray, baseOff+index*longScale) to avoid direct memory access. if the long gets moved, code might crash (if GC happens inbetween computation and access to baseArrayOffset)

2. assumes internalLongArrayOffset, arrayBaseOffset is constant over runtime (which is true currently)

3. assumes long Scale factor is 8.

4. assumes array is allocated in a continous chunk of memory

5. computation of baseAdress computes baseAddress of the pointer to the long array, not the array itself 

[Gil Tene]

Pretty good. Lets go over some of the points (skipping over the casting issues and other style rather than actual semantic risk things)

First, there was the GC thing:

People of this list seem to get the GC-can-happen-between-code-lines-and-move-my-obejcts-around risk, which can make a computed base address wrong. The getLong(long, long) signature [and all other get/put variant with direct address+offset parameters) should only be used for off-heap, known-to-remain-static addresses. They can NEVER be safely used to access content in the heap, as the input address is stale by definition.

In my experience, this GC-can-move-your-stuff thing is probably the biggest pitfall in the current unsafe API [So this means people posting on this list are pretty good, since you all noted it]. While most people would instinctively balk at the casting of a reference into a long for address computation purposes, their objections are usually on the grounds of "code cleanliness" and type safety. This usually means that if they think they know what they are doing, they would be willing to do some dangerous but well understood casting in a contained piece of code to achieve something. After all, that is exactly what good practice would be in C (make your cross-type casting in well contained inline functions or macros). The GC thing often does not come up as a reason unless you point to it and ask "what can go wrong *there*?, and even there most people I've tried this with can't quite articulate what could happen, just that it's against some rule and probably not a good idea. The problem is compounded by the fact that this one-line-race-withGC is very hard to expose in testing, since GC events are extremely (dynamically, form an instruction count point of view) rare, and the chance of a relocating GC hitting you right between the specific two lines in a race is very low. The fact that in low latency systems there are very few actually running threads at most times, and that GC is usually triggered at an allocation site, makes it virtually impossible to design a QA test that will actually hit this sort of thing in a full system without aid from the JVM (using it's various non-product stress modes), and even then it's just "more likely" to be found. [BTW, you guys that do use unsafe should probably be baking your code with fastdebug builds of OpenJDK with all sorts of modes like ScavengeALot and GCALotAtAllSafepoints turned on].

Next, there was the reference size thing is another item (could be 32 bit or 64 bit) Nitsan noted. But it's representation is also an issue (e.g. compressedOops). But that doesn't matter, because there is NEVER a safe way to read a reference field as anything other than an Object type. So while casting a reference field is possible, all scalar representations for the contents of a reference field are useless by definition because they become stale immediately after the read. This includes all those "non-addressing" use-mode excuses I've seen attempted, like use for keying and comparison purposes.

Next, there were assumptions about constant behaviors over time:

Rudiger noted the assumptions about internalLongArrayOffset, arrayBaseOffset being constant over runtime, including the important "(which is true currently)" note. Both Nitsan and Rudiger noted the assumption that the long scale factor is 8, and Nitsan noted that a less dangerous way would be to use arrayIndexScale to derive the proper scale. But both forgot that there is an assumption that this scale is constant over time (which is also "currently true").

The "constant over time" assumption is one of the greatest weaknesses of the current Unsafe API, and one of the reasons I expect it to be deprecated in some future point in time. Those "currently true" statements can become non-true at any time in the future, and will actually have to do so if some of the perfectly valid under-the-hood optimization techniques JVM researches are working on make it to production. JVMs have the power to safely change object layout over time through a combination of all-reference-fixups-traversal (usually folded into GC) and de-optimization. While none currently do this in production code, there is plenty of interesting work done by folks on layout optimization techniques that could change class layout at any safepoint in your code (read as: "between any two operations that you can state in Java"). These layout change things are usually motivated either by access pattern optimization work or by field and object compression work. I've even run into ones where arrays grow backwards in memory (making scale factor negative). If any of those layouts-change-during-runtime things ever makes it to production, you can kiss all the current unsafe methods that access heap contents good-bye, as their APIs will be forced to change. The reason is that while the reference itself can probably be guaranteed to remain non-stale, any relative offset computation may become stale-by-definition any day now. Possible victims include field offset computation, array base offset computation, and array scale factor computation, which would make some or all of the direct computed-offset access to heap objects unavailable in future unsafe APIs. Note that the pure (safe) Java means of accessing these things via reflection APIs will continue to work, and do so quite optimally, since the compilers will be able to de-optimize and re-optimize code that assumes these things are "temporarily constant" (a recorded assumption that can be used to trigger a de-optimization when they change).

Another assumption that none of you stated was that the various layout information points are constant at a point in time. e.g. the assumption that a given Java class has only one current value for field offsets, and that arrays of longs have a single scale at a given point in time. Here again there are various techniques (usually motivated by compression or instrumentation) that are being researched where different instances of the same Java class may have varying heap layout representations at the same point in time. The easiest to comprehend example of this are the on-again-off-again discussions on holding String contents in memory in 8-bits-per-char form when the string contains only ASCII chars. But I've seen heap-storage-reduction work that suggests doing the same for any primitive array where immutability or low chance of out-of-expected-value-range modification can be inferred. And I've seen sound academic papers that do similar thing to commonly used non-array classes. These thing usually (not always) take the form of providing a Java class with multiple under-the-hood class representations that are proper classes from the JVM point of view (each with constant notions of field offsets and scales), but are folded into a single class from a Java semantics point of view. Since the class that your unsafe APIs are working on is a Java semantics class, there is not necessarily going to be a good constant-for-this-class answer to many of the offset and scale querying things in the current unsafe API.

That's probably enough for this one example.

Hopefully this help explain some of the fragility of key parts of the unsafe API. While I think the unsafe APIs for off-heap access can and should be solidified and published as a safe-to-rely-on API (much like JNI), and so should things like fences in general and atomics for off-heap,  I see the current on-heap object manipulation via unsafe (by anything outside of the JDK libraries) as a very flakey abstraction that is likely to destabilize and change over time. This makes it very risky from a code-logenvity point of view to use them, even though they do work on most JVMs (Zing included). The JDK libraries have to use them, but as I noted before, their implementation can synchronously and safely be changed to match any changes to the unsafe APIs in the future.

As it stands, BTW, I've seen little performance based justification for (outside of JDK libs) use of most of these heap-accessing unsafe APIs, which, as noted previously, are very different from the off-heap ones even though they differ only in signature. I'd love to see examples (with measurements) of actual speed improvement through use of on-heap unsafe access. If JIT optimizations do not already make most of these performance-nuetral, we should keep working on those optimizations...

[Rüdiger Möller]

Most important Unsafe-feature is the ability to copy from char[], int[] to/from byte[]. Doing this explicitely is a massiv slowdown. Man message-passing high-speed java systems just would not be fast enough without that. Doing unguarded array access using Unsafe doe not pay of that much.

see this benchmark ("String performance" and "Native Arrays" illustrates the speed difference).

http://fast-serialization.googlecode.com/files/result-v1.13.html

[Martin Thompson]

Thanks Gil this is a great post.  It is good to keep people aware of the pitfalls when "running with scissors".

To answer your question from below, on-heap has a number of serious issues due to performance:

1. Strings are soooo inefficient for copying to and from a byte buffer of any type.  Copying to and from ASCII text needs to be optimised with minimal copying and encoding overhead.
   
2. ByteBuffer has a major omission in ability to copy to and from a byte[] efficiently for ranges of bytes.
3. ByteBuffer seems to always apply bounds checking for real world applications in my experience.
4. Java has no means of applying a visitor pattern over a byte buffer for IO in similar ways to C# or C without the ByteBuffer overhead.
5. AtomicFieldUpdaters are way too inefficient to be useful.

That's just 5 off the top of my head :-)

[Michael Barker]

An addendum to this list.

> Strings are soooo inefficient for copying to and from a byte buffer of any
> type. Copying to and from ASCII text needs to be optimised with minimal
> copying and encoding overhead.
> ByteBuffer has a major omission in ability to copy to and from a byte[]
> efficiently for ranges of bytes.

Also bulk operations with absolute positioning rather than relative.
You can access 8 bytes (i.e. a long) using absolute positioning but
not byte arrays. I've never understood this omission.

> ByteBuffer seems to always apply bounds checking for real world applications
> in my experience.
> Java has no means of applying a visitor pattern over a byte buffer for IO in
> similar ways to C# or C without the ByteBuffer overhead.
> AtomicFieldUpdaters are way too inefficient to be useful.

I switched the implementation of Sequence used in the Disruptor from
something based on the AtomicLongFieldUpdater to Unsafe and saw a
15-20% boost in performance. Admittedly I switched to avoid a bug in
the IBM implementation of AtomicLongFieldUpdater rather than
specifically for performance.

Another to add to the hit list would be an implementation of
AtomicLong that use the @Contended annotation. This cropped up in a
discussion on Concurrency Interest a few months ago, but nothing has
emerged so far. I pinged the that list and the JDK8 one to see if
this was likely to happen. I could completely drop all uses of the
Unsafe from the Disruptor with that one class.

Mike.

[Richard Warburton]

Hi,

I switched the implementation of Sequence used in the Disruptor from
something based on the AtomicLongFieldUpdater to Unsafe and saw a
15-20% boost in performance.  Admittedly I switched to avoid a bug in
the IBM implementation of AtomicLongFieldUpdater rather than
specifically for performance.

Out of interest, was that 15-20% number across different JVMs or is that just a hotspot number?

regards,

  Dr. Richard Warburton

  http://insightfullogic.com

  @RichardWarburto

[Michael Barker]

Just Hotspot.

[Jason Koch]

On Monday, 24 June 2013 03:09:28 UTC+10, Gil Tene wrote:

Pretty good. Lets go over some of the points (skipping over the casting issues and other style rather than actual semantic risk things)

First, there was the GC thing:

People of this list seem to get the GC-can-happen-between-code-lines-and-move-my-obejcts-around risk, which can make a computed base address wrong. The getLong(long, long) signature [and all other get/put variant with direct address+offset parameters) should only be used for off-heap, known-to-remain-static addresses. They can NEVER be safely used to access content in the heap, as the input address is stale by definition.

And here's an example demonstrating this very effect:

https://github.com/jasonk000/examples/blob/master/pointers/src/main/java/bluedevel/App.java

http://fasterjava.blogspot.com.au/2014/03/chasing-pointers-with-scissors-or.html

[punx120]

I've seen a few cases where people use Unsafe to "optimize" volatile write (and/or read), for example A Faster Volatile. I understand that this would be a problem the JVM starts changing objects layout in memory. But is this type of code also vulnerable to GC-can-move-your-stuff? My question shows that I don't really understand how GC move stuff around !

Thanks!

[Peter Lawrey]

You cannot safely address the heap using a pointer/reference.  For this reason the Unsafe methods access the heap using a reference to an object and a relative offset.  This reference will be correct/corrected no matter where/when the object is moved.

On 13 March 2014 11:47, punx120 <pun...@gmail.com> wrote:

I've seen a few cases where people use Unsafe to "optimize" volatile write (and/or read), for example A Faster Volatile. I understand that this would be a problem the JVM starts changing objects layout in memory. But is this type of code also vulnerable to GC-can-move-your-stuff? My question shows that I don't really understand how GC move stuff around !

Thanks!

--

You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gil Tene]

The simplest way to thnk of the "How GC moves stuff around" question in this context is this:

Between any two bytecodes in your program, the location of any and all objects in the heap can change. The references themselves do not logically change when objects are moved. It is just the address in the heap that the reference "points to" that changes. When any object is moved, the GC will hunt down and change the (heap address value) of any reference to any object that exists, anywhere in the world, so that it will point to the correct heap location when it is used to access the contents of an object. This is true for any GC algorithm in any relocating collector. Whether this stuff happens inside a stop-the-world event or not is a matter for collector implementation.

As Peter notes, when you use the unsafe forms that take a reference and a relative offset as arguments [e.g. putLong(Object o, long offset, long x) ] to access heap contents, the usage access is done "within a single byte code" (inside the unsafe method), and the object address will not change between the unsafe call's evaluation of the reference's associated heap address and the access to the field.

However, if you use the unsafe forms that take an absolute (long) address as an argument [e.g. putLong(long address, long x) ] to access heap contents, the address itself would have been computed in some previous instruction (e.g. via another unsafe call that extracted a long value from a reference representation), There is at least one byte code boundary between that computation and the call to the unsafe method accessing the heap. A GC operation could happen between those two byte codes, changing the heap address of the reference *without* changing the long value of "address" (the GC has no idea this long has anything to do with a reference to an object). This situation will then result in unhappy executions. The kind you really wish would crash your program, but can likely do even worse things to your day.

This is why the absolute (long) address forms of unsafe calls are only "safe" for use outside of the heap. [It is "safe" for off-heap access under some strange definitions of the word "safe", being used the describe something clearly labeled "unsafe"]. These forms are never safe to use on heap contents or on Java objects. We need to draw the subtle semantic definition difference between "Unsafe" and "NeverSafe", I guess, which is pretty comical...

To paraphrase Donald Rumsfeld: "... because it is safe to say that there are safe safes; there are things that are safe to do safely. There are safe unsafes; that is to say, there are unsafe things that we think we can do safely. But there are also unsafe unsafes – there are things that are unsafe to do unsafely."

(see this Wikipedia link for the original "known knowns" quote).

[Nitsan Wakart]

I wrote a post on the topic recently: http://psy-lob-saw.blogspot.com.au/2014/02/unsafe-pointer-chasing.html hope it helps remove some of the confusion.

The rule is: "Addresses are for off-heap, references are for on-heap", anything else will hurt you.

The example you referenced is safe as the object reference + field offset method is used to perform the on heap memory access.

Note that these methods (the ref,offset type methods) will use the offset as an address if the reference is null.

Also note that nothing is stopping you from giving an offset beyond the scope of your referred object.

On Thursday, March 13, 2014 2:47 AM, punx120 <pun...@gmail.com> wrote:

I've seen a few cases where people use Unsafe to "optimize" volatile write (and/or read), for example A Faster Volatile. I understand that this would be a problem the JVM starts changing objects layout in memory. But is this type of code also vulnerable to GC-can-move-your-stuff? My question shows that I don't really understand how GC move stuff around !

Thanks!

[Tobias Lindaaker]

Except that's not entirely true... (the "references are for on-heap" part, at least if we talk about the methods on Unsafe that takes a reference as it's first argument)

This being an unofficial API different JVM vendors behave slightly different, but at least on hotspot you can pass null as the reference and make the ref+offset methods treat the offset as an absolute address. This is how you would perform volatile-like memory access outside of the heap.

When I investigated this about three years ago J9 and JRockit did not support this, J9 would throw either IllegalArgumentException or NullPointerException (I don't remember which one), and JRockit would crash. But since I talked to the implementors about it after my discovery it is quite possible that they have changed the behaviour since then, I haven't re-investigated.

-tobias

[Darach Ennis]

Hi guys,

I for one like Nitsan's memory aid "Address are for off-heap, references are for on-heap" as it

is enough to keep me out of (deep) trouble on hotspot at least. Unsafe is in common enough

usage that the behaviour (or a subset) should be "Safe Unsafe" rather than "Unsafe Unsafe".

The Rumsfeld classification offered by Gil would be useful as a categorisation so "Unsafe Unsafe" usage

or behaviours can be avoided.

Cheers,

Darach.

[Nitsan Wakart]

Good to know :-), and as you point out this is all unofficial API so vendors can do what they will, including not having the method at all BTW. Please pre-pend my previous statement with "There are JVMs on which I know this to be true"

This is a compatibility problem for SBE(simple-binary-encoding) and others if this is still the case.