Using ByteBuddy for adding logging to classes

Will Sargent

unread,

Feb 10, 2019, 10:41:36 PM2/10/19

to Byte Buddy

Hi all,

I've written small example showing how you can add logging to classes without touching the source code using ByteBuddy.

Example is here:

https://github.com/tersesystems/terse-logback/blob/master/example/src/main/java/example/ClassWithByteBuddy.java

and added a bit to the README to explain usage:

https://github.com/tersesystems/terse-logback/blob/master/README.md#instrumenting-logging-code-with-byte-buddy

Thanks,

Will.

Rafael Winterhalter

unread,

Feb 11, 2019, 3:30:11 AM2/11/19

to Will Sargent, Byte Buddy

Interesting use case, thanks for sharing!

--
You received this message because you are subscribed to the Google Groups "Byte Buddy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to byte-buddy+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Will Sargent

unread,

Jun 12, 2019, 11:15:12 AM6/12/19

to Rafael Winterhalter, Byte Buddy

I've added a blog post going into more detail:

https://tersesystems.com/blog/2019/06/11/application-logging-in-java-part-8/

Rafael Winterhalter

unread,

Jun 13, 2019, 3:49:02 AM6/13/19

to Will Sargent, Byte Buddy

Thanks for the link, good write up.

Small improvement tipp: You do not need to configure eager self-initialization, simply put "disableClassFormatChanges()" which gives you the effect you want, I think.

Also, why are you using the StringMatcher rather then the DSL for ElementMatchers? I think the latter would be more readable. I talso think that the default ignore matcher should be more efficient as it excludes bootstrap classes.

Best regards. Rafael

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/CAAvUidNX16_CB99gmCaogDoKbRJeKJJC3EFZwC9c%2Bm_Y7TuTMg%40mail.gmail.com.

Will Sargent

unread,

Jun 13, 2019, 10:51:43 AM6/13/19

to Rafael Winterhalter, Byte Buddy

Thanks, I didn't know about disableClassFormatChanges. So like this?

https://github.com/tersesystems/terse-logback/pull/52/files#diff-0e1e7bf41886c36fa78f9acf4cbe8ccaR51

> Also, why are you using the StringMatcher rather then the DSL for ElementMatchers? I think the latter would be more readable.

I want to (eventually) have a config based format, where I can specify classes / methods in a file.

> I talso think that the default ignore matcher should be more efficient as it excludes bootstrap classes.

Okay, I took out the ignore() line.

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/CA%2BDM0AmtX0XwppWJuS0sqTP4cNk67DwjCKFRo38q_uc70zgnhw%40mail.gmail.com.

Rafael Winterhalter

unread,

Jun 17, 2019, 4:35:45 AM6/17/19

to Will Sargent, Byte Buddy

Yes, just like that and I see the argument on the configuration.

Will Sargent

unread,

Aug 10, 2019, 9:13:52 PM8/10/19

to Rafael Winterhalter, Byte Buddy

I followed up on this and added configuration driven logging

https://github.com/tersesystems/terse-logback#instrumenting-compiled-code-with-logging-using-byte-buddy

I'm not able to add logging for java.lang.Thread.run, which makes me a bit sad, but it's still great you can do this from code.

Will Sargent

unread,

Aug 12, 2019, 12:57:22 AM8/12/19

to Rafael Winterhalter, Byte Buddy

I am getting a very odd bug when I start up, it just says "klass: java/lang/NoClassDefError" and then throws an exception:

https://gist.github.com/wsargent/4168308f913d22069fa6cd6f81c46114

Rafael Winterhalter

unread,

Aug 12, 2019, 3:20:46 AM8/12/19

to Will Sargent, Byte Buddy

I assume that you have injected code into Thread that is not visible to that class?

Make sure to make code visible on the bootstrap class loader if you are instrumenting core library classes. You can do so using Instrumentation.appendToBootSearchPath.

Will Sargent

unread,

Aug 12, 2019, 12:08:59 PM8/12/19

to Rafael Winterhalter, Byte Buddy

Derp, I did that in the securityfixer example but forgot why. Added it back in:

https://github.com/tersesystems/terse-logback/pull/69/files#diff-7b805dcd3b32005e182365d2492da242R35

And now everything works!

❱ bin/logback-bytebuddy

[Byte Buddy] DISCOVERY java.lang.System [null, null, loaded=true]

[Byte Buddy] TRANSFORM java.lang.System [null, null, loaded=true]

[Byte Buddy] COMPLETE java.lang.System [null, null, loaded=true]

89 TRACE java.lang.System - entering: java.lang.System.setSecurityManager(java.lang.SecurityManager) with arguments=[java.lang.SecurityManager@58c1670b]

91 TRACE java.lang.System - exiting: java.lang.System.setSecurityManager(java.lang.SecurityManager) with arguments=[java.lang.SecurityManager@58c1670b] => returnType=void

Security manager is set!

91 TRACE java.lang.System - entering: java.lang.System.setSecurityManager(java.lang.SecurityManager) with arguments=[null]

91 TRACE java.lang.System - exiting: java.lang.System.setSecurityManager(java.lang.SecurityManager) with arguments=[null] => returnType=void

ATTACK SUCCEEDED: Security manager was reset!

Will Sargent

unread,

Aug 23, 2019, 9:37:17 PM8/23/19

to Rafael Winterhalter, Byte Buddy

Is there a way that ByteBuddy can get at the LineNumberTable information, so I can add line numbers to instrumented code?

Will Sargent

unread,

Aug 26, 2019, 10:51:01 AM8/26/19

to Rafael Winterhalter, Byte Buddy

Looks like I can get at the line number table with

public static LineNumberNode findLineNumberForInstruction(InsnList insnList, AbstractInsnNode insnNode) {
    Validate.notNull(insnList);
    Validate.notNull(insnNode);

    int idx = insnList.indexOf(insnNode);
    Validate.isTrue(idx != -1);

    // Get index of labels and insnNode within method
    ListIterator<AbstractInsnNode> insnIt = insnList.iterator(idx);
    while (insnIt.hasPrevious()) {
        AbstractInsnNode node = insnIt.previous();

        if (node instanceof LineNumberNode) {
            return (LineNumberNode) node;
        }
    }

    return null;
}

https://stackoverflow.com/questions/49986872/how-to-determine-line-number-for-the-method-with-java-asm

Rafael Winterhalter

unread,

Aug 26, 2019, 12:05:37 PM8/26/19

to Will Sargent, Byte Buddy

You can so in ASM but this is not possible in Byte Buddy by intention. Line numbers are very instable and do not give you much value since many chained tools might apply changes where the granularity is too high.

You can use an exception from your code to get the current line number or since Java 9 the StackWalker API.

Best regards, Rafael

Will Sargent

unread,

Aug 26, 2019, 1:27:14 PM8/26/19

to Rafael Winterhalter, Byte Buddy

The problem is that filling in the StackTraceElement array is expensive, relying on a native method call. If that info can be extracted to compile time then it’s free.

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/CA%2BDM0AnO%2BHZM-bNGCnPW4ZjMtYGswD2t_unqF0SrvxhJQhEBgw%40mail.gmail.com.

Rafael Winterhalter

unread,

Aug 27, 2019, 2:31:38 AM8/27/19

to Will Sargent, Byte Buddy

In this case, you can register a custom MethodVisitor using an AsmVisitorWrapper that captures the current line number of the method. You can then in advice browse the method visitor hierarchy to get hold of this visitor to extract the method line number from its capturing.

Will Sargent

unread,

Aug 28, 2019, 12:05:48 PM8/28/19

to Rafael Winterhalter, Byte Buddy

Thanks for the tip!

So when you say register a custom method visitor, you're talking about something like a visit(new AsmVisitorWrapper.ForDeclaredMethods) like this:

https://github.com/raphw/byte-buddy/blob/master/byte-buddy-dep/src/main/java/net/bytebuddy/agent/builder/AgentBuilder.java#L7183

The LineNumberNode seems not to be there in the shaded ASM in Byte Buddy, going to look for a replacement...

Will Sargent

unread,

Aug 28, 2019, 12:35:00 PM8/28/19

to Rafael Winterhalter, Byte Buddy

> The LineNumberNode seems not to be there in the shaded ASM in Byte Buddy, going to look for a replacement...

Looks like MethodVisitor.visitLineNumber() is the thing to override, so something similar to https://github.com/raphw/byte-buddy/blob/master/byte-buddy-dep/src/main/java/net/bytebuddy/utility/visitor/LineNumberPrependingMethodVisitor.java

Rafael Winterhalter

unread,

Aug 29, 2019, 3:25:37 AM8/29/19

to Will Sargent, Byte Buddy

Yes, this is what you would be looking for.

Will Sargent

unread,

Sep 1, 2019, 1:26:23 AM9/1/19

to Rafael Winterhalter, Byte Buddy

I can't find a public method visitor hierarchy I can get at (Entry is protected), but I was able to set up a cache of MethodInfo from the visitor I can access later, so that works for me :-)

The only bit I have to deal with now is making sure that the signature matches up with the descriptor later, because they have different formats:

signature = (java.lang.String), descriptor = (Ljava/lang/String;)V

For posterity:

public AgentBuilder builderFromConfig(ElementMatcher<? super TypeDescription> typesMatcher,
                                      ElementMatcher<? super MethodDescription> methodsMatcher) {
    return new AgentBuilder.Default()
            .with(AgentBuilder.RedefinitionStrategy.RETRANSFORMATION)
            .with(AgentBuilder.InitializationStrategy.NoOp.INSTANCE)
            .with(AgentBuilder.TypeStrategy.Default.REDEFINE)
            .disableClassFormatChanges() // frozen instrumented types
            .type(typesMatcher) // for these classes...
            .transform((builder, type, classLoader, module) -> {
                // ...apply this advice to these methods.
                Advice to = Advice.to(INSTRUMENTATION_ADVICE_CLASS);
                AsmVisitorWrapper on = to.on(methodsMatcher);
                AsmVisitorWrapper lineWrapper = wrapper(METHOD_INFO_LOOKUP);
                return builder.visit(lineWrapper).visit(on);
            });
}

private AsmVisitorWrapper wrapper(Consumer<MethodInfo> consumer) {
    return new AsmVisitorWrapper.AbstractBase() {
        @Override
        public ClassVisitor wrap(TypeDescription instrumentedType,
                                 ClassVisitor classVisitor,
                                 Implementation.Context implementationContext,
                                 TypePool typePool,
                                 FieldList<FieldDescription.InDefinedShape> fields,
                                 MethodList<?> methods, int writerFlags, int readerFlags) {
            return new ClassVisitor(Opcodes.ASM5, classVisitor) {

                private String name;
                private String source;
                private String debug;

                @Override
                public void visitSource(String source, String debug) {
                    this.source = source;
                    this.debug = debug;
                    super.visitSource(source, debug);
                }

                @Override
                public void visit(int version, int access, String name, String signature, String superName, String[] interfaces) {
                    this.name = name != null ? name.replace('/', '.') : null;
                    super.visit(version, access, name, signature, superName, interfaces);
                }

                @Override
                public MethodVisitor visitMethod(int access,
                                                 String n,
                                                 String d,
                                                 String s,
                                                 String[] e) {
                    MethodVisitor methodVisitor = super.visitMethod(access, n, d, s, e);
                    return new MethodVisitor(Opcodes.ASM5, methodVisitor) {
                        @Override
                        public void visitLineNumber(int line, Label start) {
                            consumer.accept(new MethodInfo(access, n, d, s, e, name, source, debug, line));
                            super.visitLineNumber(line, start);
                        }
                    };
                }
            };
        }
    };
}

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/CA%2BDM0A%3DJy9PuEMWo_0%3DmJtuX7%2BJKz8qem4ASq5Ah6AopoG%2Bdpg%40mail.gmail.com.

Rafael Winterhalter

unread,

Sep 1, 2019, 3:58:09 PM9/1/19

to Will Sargent, Byte Buddy

Every MethodVisitor has a field "mv" that points to the method visitor that the current visitor is wrapping. You would need to register your visitor before the advice (which is also implemented as a visitor) and then access it from an OffsetMapping (the Target.StackManipulation gets hold of this wrapped MethodVisitor eventually). You can then map the line number to an advice argument. It does require some type casting but it is doable.

Best regards, Rafael

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/CAAvUidN%3DR%2Bow3xZPJbNEtVsYA49MWEig2_BpriYPcZy4JU3_PQ%40mail.gmail.com.

Will Sargent

unread,

Sep 1, 2019, 9:54:49 PM9/1/19

to Rafael Winterhalter, Byte Buddy

So accessing a method visitor from an Advice.OffsetMapping through a Target.StackManipulation is also something that requires several intermediate steps. I'm leaning on stagemonitor for how custom advice annotations are wired into the system:

https://github.com/stagemonitor/stagemonitor/blob/master/stagemonitor-core/src/main/java/org/stagemonitor/core/metrics/annotations/ExceptionMeteredTransformer.java#L79

and reading through https://github.com/raphw/byte-buddy/issues/110 and https://github.com/raphw/byte-buddy/issues/551

Currently I'm assuming something like this for getting at a method visitor through a stackmanipulation through an offsetmapping:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
protected @interface LineNumber {
}

static class LineNumberMethodVisitor extends MethodVisitor {
    public LineNumberMethodVisitor(MethodVisitor methodVisitor) {
        super(Opcodes.ASM5, methodVisitor);


    }

    @Override
    public void visitLineNumber(int line, Label start) {

        System.out.println("visitLineNumber " + line);
        super.visitLineNumber(line, start);
    }

    public void gotcha() {
        System.out.println("gotcha");
        // return accumulated state from here
    }
}

public static class LineNumberFactory implements Advice.OffsetMapping.Factory<LineNumber> {
    @Override
    public Class<LineNumber> getAnnotationType() {
        return LineNumber.class;
    }

    @Override
    public Advice.OffsetMapping make(ParameterDescription.InDefinedShape target,
                                     AnnotationDescription.Loadable<LineNumber> annotation,
                                     AdviceType adviceType) {
        // http://bytebuddy.net/javadoc/1.10.1/net/bytebuddy/asm/Advice.OffsetMapping.html
        return new Advice.OffsetMapping() {
            @Override
            public Target resolve(TypeDescription instrumentedType,
                                  MethodDescription instrumentedMethod,
                                  Assigner assigner,
                                  Advice.ArgumentHandler argumentHandler,
                                  Sort sort) {
                System.out.println("resolve: instrumentedMethod " + instrumentedMethod);

                // ??? Return a custom stack manipulation here?
                return Target.ForStackManipulation.of(instrumentedMethod.asDefined());
            };
        };
    }
}

but then after that it's not clear where the StackManipulation goes from there.  

I assume I'm supposed to extend a StackManipulation so I can get access through the method visitor, but the following doesn't work:

return new Advice.OffsetMapping.Target.AbstractReadOnlyAdapter() {
     public StackManipulation resolveRead() {
        return new StackManipulation() {
            @Override
            public boolean isValid() {
                return true;
            }

            @Override
            public Size apply(MethodVisitor methodVisitor,
                              Implementation.Context implementationContext) {
                System.out.println("apply: methodVisitor");

                if (methodVisitor instanceof LineNumberMethodVisitor) {
                    LineNumberMethodVisitor lnmv = (LineNumberMethodVisitor) methodVisitor;
                    lnmv.gotcha();
                }
                return StackSize.ZERO.toIncreasingSize();
            }
        };
    }

Rafael Winterhalter

unread,

Sep 2, 2019, 3:35:20 AM9/2/19

to Will Sargent, Byte Buddy

Yes, this is how I would approach it. Make sure to not hard code how many parent method visitors you resolve before casting but make it sensitive to the type you are using. Any wrapping levels might change with future Byte Buddy releases.

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/CAAvUidN_FL6MX%2BH5NnJHMhrVH%2BnfsEaJkssFqXWgk-D_5np-dQ%40mail.gmail.com.

Will Sargent

unread,

Sep 9, 2019, 12:20:22 PM9/9/19

to Byte Buddy

For posterity -- I had it working initially, then found that when I loaded in Logback and everything else to the bootstrap class loader, it really confused all the application code that assumed everything was using the system classloader.

So now I have it broken out -- libraries in the boot classpath are shadowed (including bytebuddy) so they won't pollute downstream classloading, and the impl classes call out to the system classloader but those are marked as "provided" because the app is responsible for them.

https://github.com/tersesystems/terse-logback/blob/master/logback-bytebuddy/src/main/java/com/tersesystems/logback/bytebuddy/LoggingInstrumentationAdvice.java#L153

Rafael Winterhalter

unread,

Sep 9, 2019, 3:34:38 PM9/9/19

to Will Sargent, Byte Buddy

For the boot loader, I think shadowing is essential. Otherwise, loading hierarchies can be very confused, especially in case of a version mismatch where some classes will be loaded on the system and some on the boot loader.

--

You received this message because you are subscribed to the Google Groups "Byte Buddy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to byte-buddy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/byte-buddy/da2ad48a-9e75-4745-bd7f-2b99e9745087%40googlegroups.com.

Reply all

Reply to author

Forward