@Delegate - make composition and the decorator pattern easy

[Peter Taylor]

Dear All,

May I suggest an annotation called @Delegate that could be placed on
class members to generate synthetic methods in the enclosing type, to
match an interface, that delegate back to the member.

This would help people create objects through composition and allow
them to easily decorate behaviors.

/**
* Put on any field or method to make lombok promote the interface to
the enclosing type.
* <p>
* Example:
* <pre>
* &#64;Delegate(Iterable.class)
* private List<T> list;
*
* &#64;Delegate(Runnable.class)
* private abstract Runnable delegate();
* </pre>
*
* will generate:
*
* <pre>
* public Iterator<T> iterator() {
* return list.iterator();
* }
*
* public void run() {
* delegate().run();
* }
* </pre>
*
* If any method defined by the interface(s) exists in the enclosing
type, a
* synthetic method will not be generated and no warning given since
it is
* assumed the method will be decorating or changing default behavior.
*
* If any method defined by the interface(s) exists in the enclosing
types
* superclass' (ignoring abstract methods) a compile error will occur
since
* the intent is unclear. A method should be written in the class to
call
* super or otherwise to prevent the compilation error - by providing
the intent.
*/
@Target({ElementType.FIELD,ElementType.METHOD})
@Retention(RetentionPolicy.SOURCE)
public @interface Delegate {
/**
* List the interfaces that you wish to promote to the enclosing
type.
*/
Class<?>[] value();
}

Regards,
Peter

[Reinier Zwitserloot]

Nice annotation isn't it?

So nice, you're not the first one that had the idea. The past guy that
suggested it caused us to kick @Delegate all the way to the top of the
feature chain. It's coming :P

The biggest problem we're facing with @Delegate is that lombok
_currently_ operates right after the tree construction phase, but
before the binding phase. It means that, given, say:

@Delegate
private List list = new ArrayList();


All we know is that "List" refers to a type, of some sort. We don't
know it's referring to java.util.List, and even if we did know*, we
don't know what java.util.List is - we can't query its methods, which
we would have to do to generate the wrapper methods. java.util.List
seems trivial, but if it's com.foo.myproject.Whatever, which is, say,
linked via a dependency that showed up in a project you're dependent
on in eclipse, then we obviously still need to find it. Read: We need
to hook into the binding architecture of both javac and eclipse,
before we can add this feature. Which we will do; I'm just explaining
why you shouldn't expect this feature to be in lombok next week.

*) We can actually do some rudimentary unqualified type to qualified
type resolution - we need to do so to figure out that with "@Data" you
actually mean "lombok.@Data" and not some other kind of Data
annotation. However, it's rather simplistic and it cannot make the
step from "I now know this is referring to java.util.List" to
"java.util.List's methods are [get, set, clear, size, etc, etc, etc]".

NB: There's some good news though. On javac we pretty much have the
binding results already, just need to update the lombok core APIs to
provide it, and on eclipse we don't, but, the resource filer is one of
the few parts of the JDT core we're working with that's actually not a
complete morass of insanity. Cripes, that code haunts my dreams.

NB2: We just nailed the 'rename type' refactor script. Progress
continues.

On Oct 10, 7:09 pm, Peter Taylor <taylor_ta...@hotmail.com> wrote:
> Dear All,
>
> May I suggest an annotation called @Delegate that could be placed on
> class members to generate synthetic methods in the enclosing type, to
> match an interface, that delegate back to the member.
>
> This would help people create objects through composition and allow
> them to easily decorate behaviors.
>
> /**
>  * Put on any field or method to make lombok promote the interface to
> the enclosing type.
>  * <p>
>  * Example:
>  * <pre>

>  *     @Delegate(Iterable.class)

>  *     private List<T> list;
>  *

>  *     @Delegate(Runnable.class)

[Peter Taylor]

Sorry! - I realize now this has actually already been suggested....

[Reinier Zwitserloot]

No need to apologize for posting ideas, Peter.

Also, I hadn't thought through the issue of implemented methods in
superclasses. I'm not entirely convinced we should force explicitness
instead of just assuming the programmer knows what he's doing, but you
have a point; I'll discuss it with Roel when the time comes to add
this feature.

[jpp]

Brilliant! Maybe we could also have a prefix property for the
annotation. It would allow Lombok to generate two series of methods
delegating, for instance, the Iterable interface to two different
objects:

public class House {

// generates public Iterator<Room> roomIterator()
@Delegate(for=Iterable.class, prefix="room")
private final List<Room> rooms;

// generates public Iterator<Person> peopleIterator()
@Delegate(for=Iterable.class, prefix="people")
private final List<Person> people;

}

What would also be interesting is to allow to specify an interface in
the annotation that is *not* implemented by the actual delegation
object. Here's a use case: List or Collection, for instance, specify a
pretty long list of methods, all of which rarely need delegation. Most
of the time, I need public read-only access to a private list. So
maybe I'd like the methods contains, containsAll, equals (maybe with a
forced prefix in that case?), isEmpty, iterator, and size to be
delegated. The actual interface that we would specify could be our own
interface ReadOnlyList, for instance, that would specify these
methods.

Now of course we'd need to check that the actual objects supports
delegation for all of the specified methods in the ReadOnlyList
interface. That could be done each time the Lombok parser sees that
annotation, or we could require the specified interface to be itself
annotated like so:

@DelegationInterface(for=List.class)
public interface ReadOnlyList {

public int size();

// etc.

}

This would mark ReadOnlyList's methods as being a subset of the union
of all methods declared in the specified types, which could be checked
(in the general case, we could indeed allow several types; here I only
have one, List.class).

Coming back to the original field: the interface type you'd specify in
the annotation would either be (i) an interface implemented by the
object itself, or (ii) an delegation interface annotated by
@DelegationInterface, where the object would implement all the types
declared in the @DelegationInterface annotation.

I would consider such a mechanism very helpful. Is it getting too
complicated?

Cheers,
J.-P.

[Reinier Zwitserloot]

That's rather heavy composition there; wouldn't it generally be better
to just create 2 methods that return the delegates? Typing foo.getRoom
().iterator() is not too much trouble, and you already lost the
benefit of having your foo object be an Iterable, as it obviously
won't be.

Delegating in methods that you don't have to have will of course be
supported (not doing that would add needless rules), and we'll
definitely need to look into specifying a subset of methods. I'd like
to avoid the need to enumerate them in strings, though we kind of set
the precedent for that with the 'of' parameters in @Data and the like.
I think you're on to something with that; we will probably already let
you specify exactly which type you'd like to delegate, defaulting to
the type of the field, so:

@Delegate(Collection.class) List<String> list = new ArrayList<String>
();

was already in the books. All we really need to do to let you pick a
subset of methods is to say that the type in the @Delegate parameter
does not have to be a (super)type of the type of the field you're
annotating; the only rule is that each method named in the delegate's
type must be available either directly in your class, or in the type
of the field. I don't think @DelegationInterface is even needed - we
can provide it so you can document your type (this is _intended_ to be
a subset of list's methods), possibly even fielding a check that each
method mentioned in the interface is a copy of something in the target
type, but it would just be there for its documentation purposes.

[jpp]

> That's rather heavy composition there; wouldn't it generally be better
> to just create 2 methods that return the delegates?

Only if, in the case I was thinking of, I returned
Collections.unmodifiableList(...). I find this unnecessarily costly:
most of the time I just want to know its size or whether it contains
an element. That's needlessly one new object plus one other method
call per such operation.

> Typing foo.getRoom
> ().iterator() is not too much trouble, and you already lost the
> benefit of having your foo object be an Iterable, as it obviously
> won't be.

Indeed: this prefix thing would let you have the possibility to both
(i) implement Iterable for foo, and (ii) delegate some prefixIterator
() to a member.

Of course, we can argue when Lombok should stop and when it is up to
the programmer to do things themselves; however, this sounded easy
enough to add to the annotation and quite practical.

> Delegating in methods that you don't have to have will of course be
> supported (not doing that would add needless rules), and we'll
> definitely need to look into specifying a subset of methods. I'd like
> to avoid the need to enumerate them in strings, though we kind of set
> the precedent for that with the 'of' parameters in @Data and the like.
> I think you're on to something with that; we will probably already let
> you specify exactly which type you'd like to delegate, defaulting to
> the type of the field, so:
>
> @Delegate(Collection.class) List<String> list = new ArrayList<String>
> ();
>
> was already in the books. All we really need to do to let you pick a
> subset of methods is to say that the type in the @Delegate parameter
> does not have to be a (super)type of the type of the field you're
> annotating; the only rule is that each method named in the delegate's
> type must be available either directly in your class, or in the type
> of the field. I don't think @DelegationInterface is even needed - we
> can provide it so you can document your type (this is _intended_ to be
> a subset of list's methods), possibly even fielding a check that each
> method mentioned in the interface is a copy of something in the target
> type, but it would just be there for its documentation purposes.

It's not needed, I agree—maybe it would even be too restrictive as it
would force the programmer to be in control of the type being
specified it is not assignable from the field's declared type. But I'd
imagined it could make things easier for Lombok—check once that the
methods in the delegation interface are OK, and then, from the
@Delegate annotation, you don't need to check each public method of
the provided type against the methods of the declared type of the
object each time, you'd just trust @DelegationInterface. I thought
this might speed things up—I'd expect to use some variant of
ReadOnlyList often.

[Reinier Zwitserloot]

That kind of performance overhead is just not going to show up, ever.
I'll eat my left shoe if you're ever in a situation where removing the
creation of one iterator object is going to make the difference
between unusably slow and usable.


Also, you can't just hand off the iterator; it has a 'remove' method
which would allow callers to change your list, so you'd either have to
pass back Collections.unmodifiableList(foo).iterator(), or, you'd have
to create a custom iterator that wraps the real iterator but replaces
remove() with an exception.


All this very clearly points into a direction lombok, by design, does
not go: We handle the standard, common, cases. This isn't one of them.
Special cases aren't something lombok tries to address unless the
feature is _awesome_ and virtually impossible to recreate with vanilla
java. This very very clearly does not qualify as a feature that fits
the design aesthetic of lombok. Writing your own add-on that does it
is of course perfectly fine and lombok ought to be written so that
this is easy (and, if @Delegate works, I don't see how rolling your
own version with a prefix parameter would be difficult), but it's not
for the core lombok transformations library.

[jpp]

> That kind of performance overhead is just not going to show up, ever.
> I'll eat my left shoe if you're ever in a situation where removing the
> creation of one iterator object is going to make the difference
> between unusably slow and usable.

One iterator: no. All of them in the whole list of objects in your
model: probably.

A guy in my former team was working on some simulator, which was
extremely slow. Turned out there was no real single point where he'd
done something wrong. It was "a bit wrong" all over the place. A
systematically inefficient use of the collection classes: copy,
lookup, filters, etc.

What I'm saying is, we're always told that it's wrong to try and
optimize code before you really know it's causing an unacceptable
performance problem. I think it's equally wrong never to dismiss all
performance concerns that this kind of practice causes. Overhead keeps
adding up, and the next thing you know is you have to rewrite the
whole object model. Or worse: your manager decides Java's too slow and
makes you start over in C++. Urgh.

> Also, you can't just hand off the iterator; it has a 'remove' method
> which would allow callers to change your list, so you'd either have to
> pass back Collections.unmodifiableList(foo).iterator(), or, you'd have
> to create a custom iterator that wraps the real iterator but replaces
> remove() with an exception.

Point taken, bad example.

> All this very clearly points into a direction lombok, by design, does
> not go: We handle the standard, common, cases. This isn't one of them.
> Special cases aren't something lombok tries to address unless the
> feature is _awesome_ and virtually impossible to recreate with vanilla
> java. This very very clearly does not qualify as a feature that fits
> the design aesthetic of lombok. Writing your own add-on that does it
> is of course perfectly fine and lombok ought to be written so that
> this is easy (and, if @Delegate works, I don't see how rolling your
> own version with a prefix parameter would be difficult), but it's not
> for the core lombok transformations library.

Granted, the prefix thing isn't awesome: it's just convenient. Looking
at my object model, I find I apply this pattern quite often. Anybody
else out there interested in such a prefix, or is it really marginal
practice?

[Reinier Zwitserloot]

As they say, "pictures, or it didn't happen". I don't believe you can
be nickle-and-dimed on performance, at least, not on issues like this.

We're not going to make design concessions based on a hypothetical
performance issue. We would make design concessions readily if there's
an _actual_, proven performance issue.

If you're really using prefixes that often, you've created quite a
case for doing our best to keep lombok extensible :P

[jpp]

> If you're really using prefixes that often, you've created quite a
> case for doing our best to keep lombok extensible :P

Then that's good news! :-)

[Moandji Ezana]

On Mon, Oct 12, 2009 at 1:19 AM, jpp <jppe...@gmail.com> wrote:

A guy in my former team was working on some simulator, which was
extremely slow. Turned out there was no real single point where he'd
done something wrong. It was "a bit wrong" all over the place.

This might not be the best place to discuss this, but I often wonder about this issue, as well. Surely there are things to avoid, without falling into the premature optimisation trap?

Finding big bottlenecks isn't too difficult, but how does one go about finding the little, cumulative ones? Is it really ever an issue, unless you're doing totally bone-headed things? (and what are some of those bone-headed things?). Anyone know of good resources for performance basics on JVM 5/6?

Moandji

[Reinier Zwitserloot]

Here's a golden rule:

All the OLD optimization rules? Don't use them, they will actively
screw up your performance.


So (these all USED to be true for old versions of java! - beware of
any documents you find that state these things. Don't read them, they
are wrong):

WRONG: Don't create objects; creating them is expensive. Re-use them.
GOOD: Create objects. Loads of em. Especially if it means your objects
are immutable. Avoiding a 'synchronized' keyword gains you a
relatively huge amount of performance, and due to the garbage
collector's eden system, the system can handle 10 short lived objects
better than 1 long lived one. Either way, re-using objects is bad,
bad, bad, in all ways.

WRONG: Stick final on every method and every class, so that method
resolution is faster.
GOOD: That's just not relevant anymore; the JIT can figure this stuff
out quite well. Of course, there are purely code aesthetics reasons to
do it.

WRONG: Field access beats setters and getters.
GOOD: Not relevant anymore, and tends to lead to 'synchronized' usage,
or creating a lot of needless method (as mentioned, creating objects
isn't bad, but compared to simply handing out an existing one, it's
not good). Avoiding even 1 synchronized block buys you a couple
hundred virtual method lookups, and thousands of inlined getter calls.
Avoiding the copying mostly saves on memory, which is nice.

Something which is still true, but which will probably change sooner
rather than later:

DUBIOUS: Primitives beat objects; instead of bundling related state
into a class, pass a handful of primitives around.
REASON: This is definitely true. For example, a List<Byte> takes at
least 12x the space of a byte[], in bad cases (full 64-bit) even over
24x, and it's about 4 to 5 times slower. However, it's not entirely
infeasible for java to gain some sort of official @NonNull system so
the List itself can optimize this, and even without that, the JIT
compiler should be able to figure this out if the List never escapes,
or escapes only to a small set of methods.

DUBIOUS: Recursive methods are slower than rewriting it to be
interative.
REASON: True-ish, but, tail call optimization keeps being mentioned as
a likely candidate for the JVM at some point. Even without tail call
optimization, the difference is tiny. Of course, the stack is much
smaller than the heap, so if you find you'll be recursing for many
thousands of times, you SHOULD rewrite, at least to a tail-call-
optimizable form, and to be practical, to an iterative loop instead.

DUBIOUS: Lazily instantiate your singletons.
REASON: Most people who write that tired old getInstance() schtick to
create a singleton on the first call and return the old one on all
future calls get it wrong; the flag has to be volatile or your doubly-
nested if will actually fail on multi-core systems. Usually people are
thinking that every class that's in the entire classpath will be
initialized before your code run, which is obviously nuts. Upon first
use of a class, its initialized. So, just write: "private static final
INSTANCE = new MyClass();" in your singleton. The only point in the
lazy load scheme is if your singleton also has static classes that
code might use, but why would you do that to a singleton? Class
Loading is guaranteed to be streamlined properly on a multi-core
system (1 class will never be loaded and/or initialized twice due to a
threading issue).

Things that really do help performance:

RIGHT: Use java.util.concurrent instead of trying to handroll your
threading with synchronized blocks all over the place. An
AtomicInteger is way quicker than synchronizing access to 1 handrolled
mutable Integer instance. Also, 'volatile' beats synchronized, if you
know how to use it.

RIGHT: Always, always, think about the performance of your algorithm.
Computer science students know this stuff as big-O and little-O
notation. For example, quicksorting twice as much data takes 2*log(2)
more time - virtually linear to the search data size. Bubblesort takes
4 times as long to sort twice as much data (n^2). That's going to run
out of control on big data sets.

RIGHT: Profile, profile, profile. If there's a performance issue, fire
up a profiler, and figure out where. Even if it really ends up being a
nickel-and-dime situation, where loads of tiny performance issues
killed you, the vast majority of your code is run only once, or very
rarely, while your app spends 95% of the time in 5% of your code (the
80-20 rule, on steroids). Let's say you really need to nickle-and-dime
your way out of your performance issue, then only by profiling do you
know where you should be spending the time rewriting your code (and
making it ugly in the process). More likely there's a specific thing
you're doing which is eating massive amounts of CPU, and profiling
will find it for you.

RIGHT: Cache expensive operations. Use Google Collections API to
easily create caches that don't get in the way of garbage collecting.
Or just use a simple LRU cache with LinkedHashMap.

RIGHT: Some things are just too complicated for one computer. Design
things so that its easy to shard your calculations if you expect that
the program will at some point in time need to handle data that's just
too much for one computer to bear, even if you handrolled perfectly
designed assembler. It's far cheaper to buy 10 budget boxes and wire
them up, than buy one mega server, and either option is waay cheaper
than spending 2 manyears rewriting code.

On Oct 12, 4:01 pm, Moandji Ezana <mwa...@gmail.com> wrote:

[Moandji Ezana]

Thanks a lot, Reinier!

I recognise a lot of that from Brian Goetz articles and his book, so I'll assume the parts I don't recognise are true as well. :)

Moandji

[Reinier Zwitserloot]

Brian knows his stuff; feel free to assume he's right :)

[Michael Lorton]

"Do not believe anything you read or hear, no matter who says or writes it, not even if I say it, unless it conforms to your own reasoning."
-- Buddha

[Moandji Ezana]

The Brian Goetz article I was thinking of is Urban Performance Legends. His entire Java Theory and Practice series is invaluable.

On Mon, Oct 12, 2009 at 11:56 PM, Michael Lorton <mic...@circuspop.com> wrote:

"Do not believe anything you read or hear, no matter who says or writes it, not even if I say it, unless it conforms to your own reasoning."
-- Buddha

To me, this sounds like "Don't believe anything, unless you agree with it," which is a bit strange. My interpretation is perhaps too naïve.

Moandji

[Michael Lorton]

The alternative is "Believe things although you don't agree with them", which is worse.  I guess Siddh's recommended heuristic is "If someone generally worth listening to says something you agree with, believe him; if he says something you don't agree with, investigate further."

M.

[Andrew Wiley]

I hate to interrupt the philosophical discussion, but is this feature
still being considered? I would find it very useful.

[Andrew Wiley]

I hate to interrupt the philosophical discussion, but I'm interested
in a @Delegate annotation as well. Is this feature in the works? Is it
technically feasible?

[Reinier Zwitserloot]

Yes, it's technically feasible, yes, it's a great feature, and yes, it
is in the works.

[Michael Lorton]

"And there was much rejoicing." ( http://www.youtube.com/watch?v=enSYlCEz5VI )

The closure stuff, cool though it is, would mean I would have to learn a whole new syntax, effectively a new language, and get in return a better implementation of what anonymous classes do just fine now.

Delegates, builders, those kind of little fidgety things, I can start using immediately and get rid of pages and pages of boilerplate that just collect bugs.

M.

"He is really not so ugly after all, provided, of course, that one shuts one's eyes, and does not look at him."
-- Oscar Wilde

[Andrew Wiley]

On Mon, Oct 26, 2009 at 9:19 PM, Reinier Zwitserloot <rein...@gmail.com> wrote:

Yes, it's technically feasible, yes, it's a great feature, and yes, it
is in the works.

Thanks, and sorry for the double post. Google said my session had timed out the first time, but apparently it went through anyway.

[Moandji Ezana]

One thing I'm still wondering about is how short "short-lived object" is. Is a typical web request (from app server to DB and back) long or short, in the JVM's eyes?

Reading some interesting slides on the topic of garbage collection and tuning, they say that objects are removed from eden space when it is full. Is that the only measure of long- vs. short-lived?

Moandji

[Reinier Zwitserloot]

A typical web request, _IF_ all the output data ends up in a buffer
first (and there's no blocking going on) is probably short-lived, but
once you start hitting the DB with your connection pool, which will
probably block somewhere, it depends. If your object escapes the
thread, it'll probably end up being on the long lived side of the
spectrum. These are guesstimates. Nothing beats empirical data when it
comes to performance tuning :)

On Nov 17, 9:48 pm, Moandji Ezana <mwa...@gmail.com> wrote:
> One thing I'm still wondering about is how short "short-lived object" is. Is
> a typical web request (from app server to DB and back) long or short, in the
> JVM's eyes?
>
> Reading some interesting slides on the topic of garbage collection and

> tuning<http://www.slideshare.net/caroljmcdonald/java-garbage-collection-moni...>,