java.lang.VerifyError: Stack map does not match the one at exception handler

[xiaoxi zheng]

Hi,

I got VerifyError below when running application with JaCoCo(tried both on 0.7.2 and 0.7.9) instrumented class BaseEntityManager (attached).

Please note the class has already been instrumented by other bytecode tool, but it is good without -javaagent:jacocoagent.jar. The full error is also attached.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

java.lang.VerifyError: Stack map does not match the one at exception handler 917

Exception Details:

  Location:

    weblogic/ejb/container/manager/BaseEntityManager.executeUpdateStmt(Ljava/util/List;Ljavax/transaction/Transaction;Ljava/util/Set;ZZLjava/util/Collection;)V @917: astore

  Reason:

    Type 'java/lang/Object' (current frame, locals[20]) is not assignable to null (stack map, locals[20])

  Current Frame:

    bci: @893

    flags: { }

    locals: { 'weblogic/ejb/container/manager/BaseEntityManager', 'java/util/List', 'javax/transaction/Transaction', 'java/util/Set', integer, integer, 'java/util/Collection', '[Z', integer, 'java/sql/Connection', null, 'java/util/Map', 'weblogic/ejb/container/persistence/spi/CMPBean', integer, integer, '[Z', integer, null, integer, integer, 'java/lang/Object', top, top, top, top, top, top, top, top, top, top, top, top, 'weblogic/diagnostics/instrumentation/LocalHolder' }

    stack: { 'java/sql/SQLException' }

  Stackmap Frame:

    bci: @917

    flags: { }

    locals: { 'weblogic/ejb/container/manager/BaseEntityManager', 'java/util/List', 'javax/transaction/Transaction', 'java/util/Set', integer, integer, 'java/util/Collection', '[Z', integer, 'java/sql/Connection', null, 'java/util/Map', 'weblogic/ejb/container/persistence/spi/CMPBean', integer, integer, '[Z', integer, null, integer, integer, null, top, top, top, top, top, top, top, top, top, top, top, top, 'weblogic/diagnostics/instrumentation/LocalHolder' }

    stack: { 'java/sql/SQLException' }

[Evgeny Mandrikov]

Hi,

I assume that provided class file is after instrumentation by another agent, but before instrumentation by JaCoCo. Is it correct?

If this is the case, then maybe first agent made a damage. This is the most common cause of such errors and as of today we are not aware of examples where JaCoCo produces invalid class files from valid inputs. And if class file was damaged, then strange to expect that JaCoCo can recover it. Because of this and to avoid other problems - our general recommendation: JaCoCo agent should be applied first, i.e. should be listed in command-line prior to other agents. Could you please try this and report the result back?

Also provided class file has dependencies on other classes, and as far as I know there is no easy way to verify correctness of its bytecode or its instrumented version in absence of class files of its dependencies. However JaCoCo agent has parameter "classdumpdir" (see http://www.jacoco.org/jacoco/trunk/doc/agent.html) that allows to dump all classes prior to instrumentation. So will be beneficial if you will use this parameter and provide us all dumped classes, in order to be able to verify correctness of initial class and correctness of JaCoCo instrumentation. Or even better if you can somehow provide small, complete and isolated example demonstrating the same problem. In any case please also specify exact version of Java VM that you use.

As a last resort there are agent parameters "includes" and "excludes" that can be used to exclude problematic class from JaCoCo instrumentation.

Regards,

Evgeny

[xiaoxi zheng]

Many thanks for quick response,

Yes, the provided class has already been instrumented by other tool(not through agent, it was instrumented offline), but the instrumented class is good (perhaps not perfectly) since I am able to verify it with java $className;$CLASSPATH and it is working well during testing without jacoco agent.

I understood that Jacoco should be the first agent, but unfortunately, but that instrumentation is part of its product building, so we will have to use instrumented class + jacocoagent to run test and also with the instrumented classes to generate coverage report.

I used the classdumpdir option to dump the dependencies classes and attached it. 

Actually there are a number of classes are instrumented by the tool, but only one class has problem, so yes, I am currently excluding this offending class in the test.

Here is Java version

java version "1.8.0_131"

Java(TM) SE Runtime Environment (build 1.8.0_131-b11)

Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

在 2017年7月24日星期一 UTC+8下午10:10:21，Evgeny Mandrikov写道：

[Marc Hoffmann]

FYI, here are the decompiled versions of the original and instrumented class files:

    https://gist.github.com/marchof/c5ba614ea03015f304517bbb3d72d7c7

Method in question is quite complex, needs further analysis.

Regards,
-marc

--
You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/3b48a707-3c99-4a2e-a798-48f28e1fb6b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

[Evgeny Mandrikov]

Xiaoxi Zheng, thank you for providing class files.

Marc, FYI

new URLClassLoader(new URL[]{dir.toURL()})

  .loadClass("weblogic.ejb.container.manager.BaseEntityManager")

  .getDeclaredMethods();

triggers java.lang.VerifyError for instrumented classes,

and for original - java.lang.NoClassDefFoundError: javax/transaction/Transaction

still some classes are missing, but I suppose that NoClassDefFoundError happens after the place where VerifyError happens.

And indeed further analysis will require time as method is huge and complex.

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+unsubscribe@googlegroups.com.

[xiaoxi zheng]

Yes, sorry that I am not able to create a small reproducer, and appreciate your helping!

在 2017年7月28日星期五 UTC+8上午5:21:40，Evgeny Mandrikov写道：

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.

[Evgeny Mandrikov]

Hi Marc,

FYI:

I tried to instrument class from this thread by JaCoCo 0.6.4, where we had FrameTracker and got ClassCastException in it at line https://github.com/jacoco/jacoco/blob/v0.6.4/org.jacoco.core/src/org/jacoco/core/internal/instr/FrameTracker.java#L253 ("java.lang.Integer cannot be cast to java.lang.String"), i.e. during processing of AALOAD instruction.

We replaced our FrameTracker by ASM's AnalyzerAdapter that has another implementation for AALOAD - https://gitlab.ow2.org/asm/asm/blob/ASM_5_2/src/org/objectweb/asm/commons/AnalyzerAdapter.java#L665

None of our existing tests touches else-branch of this implementation, while it is touched by class from this thread. So I tried to build smaller example that will also touch this line - you'll find it in attachment:

• as said above - target class is not a reduction of class from this thread
  
• and actually code of target class has no sense
  
• however target class doesn't pass our FramesTest
  
• and test demonstrates pretty similar VerifyError ("Type 'java/lang/Object' is not assignable to null") when executed with JDK 8u131
  
• to trigger VerifyError, target class must first be read and written back by ASM ClassWriter.COMPUTE_FRAMES - this reflects statement that class from this thread is a result of instrumentation by another tool, and this is what happens in FramesTest
  

Hope this simplifies hunting of root cause and together we'll finally find it (hopefully not in our code :D)

[Evgeny Mandrikov]

Sorry, forgot to attach file, here it is.

[Marc Hoffmann]

Thanks Evgeny for this minimal Example! AALOAD is a perfect hint.

I added it to our FramesTest and indeed it fails. I'll investigate.

Cheers,
-marc

--

You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/34c8ae23-d518-43c7-8c87-951fe6b550a9%40googlegroups.com.

[Marc Hoffmann]

Hi Evgeny,

I think the problem ist with AnalyzerAdapter: When AALOAD is invoked on a variable with type null the resulting frame type is null, not "java/lang/Object". See Chapter "4.10.1.9 Type Checking Instructions" JVMS, section AALOAD:

The component type of an array of X is X. We define the component type of null to be null.

arrayComponentType(arrayOf(X), X).
arrayComponentType(null, null).

I try to create a standalone reproducer for ASM.

Cheers,
-marc

 

On 2017-09-19 14:56, Evgeny Mandrikov wrote:

--

You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/34c8ae23-d518-43c7-8c87-951fe6b550a9%40googlegroups.com.

[Marc Hoffmann]

Hi Evgeny,

with this change to AnalyzerAdapter the test becomes green again.

Cheers,
-marc

 

        case Opcodes.AALOAD:
            pop(1);
            t1 = pop();
            if (t1 instanceof String) {
                pushDesc(((String) t1).substring(1));
            } else if (t1 == Opcodes.NULL) {
                push(Opcodes.NULL);
            } else {
                push("java/lang/Object");
            }
            break;

 

 

On 2017-09-19 14:56, Evgeny Mandrikov wrote:

--

You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/34c8ae23-d518-43c7-8c87-951fe6b550a9%40googlegroups.com.

[Evgeny Mandrikov]

I came up to the same change. Nice that our thinking matches ;) And with this change original class from this thread after instrumentation passes verification!

However still suspicious that with or without modification our tests do not touch else-branch that is "push(t1)", especially that we observe bug near this place.

Maybe we can simply report this to ASM just citing JVMS. But I'm still wondering:

• is it possible to construct case that makes sense and triggers this problem? or in other words - does it mean that part of code from original class doesn't make sense? ;)
  
• why problem is observed specifically after compute-frames?
  

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+unsubscribe@googlegroups.com.

[Marc Hoffmann]

Hi Evgeny,

some more insights. Given this snippet:

Object[] array = null;
Object element = array[0];

Both compilers JDK and ECJ emit the following stack frame afterwards (if enforced by a merging branch):

[Ljava/lang/Object; java/lang/Object

With this frame AnalyzerAdapter works ok. Also the class passes the JVM verifier.

When I do a a frame recalculation with ASM (ClassWriter.COMPUTE_FRAMES) I get the following frame:

Null Null

Which is also ok for the JVM verifier but seems to trigger AnalyzerAdapter to calculate wrong frames in certain cases.

This explains why the problem is only triggered after frames have been recomputed with ASM.

 

 

Cheers,
-marc

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/b0088827-5115-4d12-b7fa-1b13eccfcaed%40googlegroups.com.

[Marc Hoffmann]

Hallo Evgeny,

sharing my analysis tools here:

https://github.com/marchof/jacoco-dev-bytecode

Cheers,
-marc

On 2017-09-20 00:37, Evgeny Mandrikov wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/b0088827-5115-4d12-b7fa-1b13eccfcaed%40googlegroups.com.

[Marc Hoffmann]

Hi Evgeny,

I've drafted a Bug report for ASM: https://gist.github.com/marchof/8d2047c34b512acfaac09928d4b59d35

What do you think?

Regards,
-marc

[Evgeny Mandrikov]

Just perfect! To save time - posted with reference to this thread and props to you about final wording: http://forge.ow2.org/tracker/?func=detail&aid=317793&group_id=23&atid=100023

Thanks.