Sizeof in java

[tm jee]

Hello lads, 

Came across the following article, 

http://highlyscalable.wordpress.com/2012/02/02/direct-memory-access-in-java/

It's for 32 bits machine. Couple of questions I'd love some input from other fellow mechanical sympathysers ?

Q1-> Is this still applicable to Oracle and OpenJDK hotspot? Ie.  instance and class fields are still as it is (ignoring the fact that the address are 32-bit based)

Q2-> The sizeof object method is

public static long sizeOf(Object object) {

   Unsafe unsafe = getUnsafe();

   return unsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L );

}




Shouldn't it be




return (unsafe.getAddress(normalize(unsafe.getInt(object, 4L))) + 12L);

instead?

Tia. Cheers




[Remi Forax]

On 02/14/2014 12:42 AM, tm jee wrote:
> Hello lads,
>
> Came across the following article,
>
> http://highlyscalable.wordpress.com/2012/02/02/direct-memory-access-in-java/
>
>
> It's for 32 bits machine. Couple of questions I'd love some input from
> other fellow mechanical sympathysers ?
>
> Q1-> Is this still applicable to Oracle and OpenJDK hotspot? Ie.
> instance and class fields are still as it is (ignoring the fact that
> the address are 32-bit based)

No, with the removal of permgen in OpenJDK8, the internal class layout
was changed,
http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/tip/src/share/vm/oops/klass.hpp
(you can see that the layout_helper is not at the same location).

cheers,
Rémi

>
> oops-jdk7.png (508×234)
>
> Q2-> The sizeof object method is
> |public| |static| |long| |sizeOf(Object object) {|
> |||Unsafe unsafe = getUnsafe();|
> |||return| |unsafe.getAddress( normalize( unsafe.getInt(object, 4L) )
> + 12L );|
> |}|
> |
> |
> |Shouldn't it be |
> |
> |
> return (unsafe.getAddress(normalize(unsafe.getInt(object, 4L))) + 12L);
>
> instead?
>
>
> Tia. Cheers
>
> |
> |

> --
> You received this message because you are subscribed to the Google
> Groups "mechanical-sympathy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to mechanical-symp...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Peter Lawrey]

Most 64-bit JVMs still use 32-bit references with Compressed Oops, but the internals use 64-bit pointers.  AFAIK, The layout is basically the same whether 32-bit or 64-bit, only the sizes change but as Remi points out Java 8 will have a different structure, but again I would expect it is basically the same for 32/64 bit.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

[Aleksey Shipilev]

On 02/14/2014 03:42 AM, tm jee wrote:
> Q2-> The sizeof object method is
> |public| |static| |long| |sizeOf(Object object) {|
> | ||Unsafe unsafe = getUnsafe();|
> | ||return| |unsafe.getAddress( normalize( unsafe.getInt(object, 4L) )
> + 12L );|
> |}|
> |
> |
> |Shouldn't it be |
> |
> |
> return (unsafe.getAddress(normalize(unsafe.getInt(object, 4L))) + 12L);
>
> instead?

Whoa, that's the one scary move to make: reading the metainformation
straight from the HS internal data structures. I tried to do this in jol
[1] pulled out very quickly realizing how fragile that is, since the
layout can be changed without notice. If I was forced to do so today,
I'd try to reuse the Serviceability Agent (SA) [2] to decode the
metadata, because SA is bundled with JDK is has the arguably correct
decoder rings for the native parts of VM.

-Aleksey.

[1] http://openjdk.java.net/projects/code-tools/jol/
[2] http://openjdk.java.net/groups/hotspot/docs/Serviceability.html

[Aleksey Shipilev]

(sorry about the typos before, I need to wake up thoroughly before writing mail)

[Peter Lawrey]

Isn't bytes 1 - 4 of the header the hashCode? I don't think offset 4 is right or if it is, it won't work the versions of Oracle JVM I am using.

http://stackoverflow.com/a/20843303/57695

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

[Gil Tene]

Wow. Deceptively attractive, "rarely sharp" scissors with clear warning label saying "unsafe for poking at one's own eye" being used to poke at itch in eye. Unfortunately leading to invitation to re-publish eye-poking instructions (this time without warning label) for repeated use by others (see comments in the linked article).

The people lucky enough to crash on their attempts with 64 bit JVMs, and to give up on it early, saved themselves a lot of pain. The ones unlucky enough to not give up, and to instead "fix" it by figuring out the "proper" layout for repeating the same eye poking exercise on 64 bit JVMs (with or without compressed oops), are in for a future visit to an eye surgeon. Or worse.

Why? Here's why:

The code below:

... return unsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L );




is logically equivalent to the following on all current 32 bit Oracle HotSpot and OpenJDK HotSpot JVMs:

... return unsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L + ((randomNumber1 < someSmallFractionDenotingRareEnoyghToMakeSillyTestSometimesWorkButLargeEnoughToGetYouFired) ? randomNumber2 * heapSize : 0));

On 64 bit Oracle HotSpot and OpenJDK, some of the constants need to be "fixed" to values that (at runtime) depend on the heap size, JVM version, the phase of the moon, and the 7 bit hash of the current Chinese year's animal name. On other JVMs, the randomNumber threshold just moves from something smaller than 0.0001 to something closer 1.0.

The dominant source of the random number component, even on a current 32 bit HotSpot JVMs that might actually have the layout described in the article, comes from the fact that the klass portion of the header in such HotSpot JVMs is (currently) a logical reference to a heap object. And since a safepoint can occur at multiple points during the computation (e.g. between evaluating the value of (normalize( unsafe.getInt(object, 4L) ) + 12L) and the calling unsafe.getAddress()with that value), random things can happen to the meaning of it's raw-bitwise-value. E.g. during such a safepoint, the klass thing that this raw-bitwise-value currently "points" to can be moved somewhere else, and something else can take it's place. The getAddress() is therefore looking in a random location in the heap that has a somewhat-higher-than-average chance of containing a hidden structure who's fourth field sometimes contains a sometimes-equivalent-to-sizeOf integer.

[Nitsan Wakart]

I realize I'm a bit late to the party with this, but still:

   return unsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L );

Is shorthand for:
   long addressOfKlass = normalize( unsafe.getInt(object, 4L) );
   long size = unsafe.getAddress(addressOfKlass + 12L);
   return size;

But in http://hg.openjdk.java.net/jdk7/hotspot/hotspot/file/c7f3d0b4570f/src/share/vm/oops/klass.hpp we have:
line 205: jint        _layout_helper;

Which suggests the above should be:
   long addressOfKlass = normalize( unsafe.getInt(object, 4L) );
   long size = unsafe.getInt(addressOfKlass + 12L);
   return size;

Not saying it will be any less broken (as per Gil's colourful description)...

[tm jee]

tia guys, very informative, question answered.

[Serkan Özal]

java.lang.instrument.Instrumentation.getObjectSize(Object obj)

can be used to get size object from JVM itself.

Instrumentation can be taken from Java agent and it can be loaded at runtime also.

See https://github.com/serkan-ozal/jillegal-agent and https://github.com/serkan-ozal/jillegal-agent/blob/master/src/main/java/tr/com/serkanozal/jillegal/agent/JillegalAgent.java

--

Serkan ÖZAL