New compiler-plugin available for the Scala wags

[Kevin Wright]

The autoproxy compiler-plugin for scala is now available to view on
github:
http://wiki.github.com/scala-incubator/autoproxy-plugin


My goal is to take some of the boilerplate pain out of object
delegation, using a code-generation approach similar to project lombok
on Java.

It's still a bit prototypey at the moment, and is currently based on
the SVN trunk of Scala.
To help with this, an ant script is available to build the plugin.

My aim is to track Scala release candidates and have the plugin
"officially" released at the same time as Scala 2.8

[Reinier Zwitserloot]

Interesting that you called it proxy. We have a similar plan for
lombok (used to be at the top of the list, but interesting co-
operation with Michael Ernst and the Checker Framework, as well as the
closures announcement, has pretty much made the lombok agenda null and
void. We're still sorting through things).

We were planning on calling it @Delegate though, and not @Proxy. We
chose @Delegate mostly because the term fits (all methods of the type
of the @proxy/@Delegate object are copied over, and the
implementations are _delegated_ to the object marked @Delegate),
there's some precedence (C# uses this term as well), and proxy has
other connotations in java (mostly involving dynamically creating an
implementation of an interface, with all methods pointing to something
akin to a method_missing kind of thing).

Intrigued as to why you went with @proxy. Is that a more common term
in the functional programming world, or something specific to scala?

--Reinier Zwitserloot

[Kevin Wright]

Funnily enough, that question has also been raised on the wave I
started for the project;
it's public, see here:
https://wave.google.com/wave/?pli=1#restored:wave:googlewave.com!w%252Ba3G_NBOLC


I guess that this is slightly a scala thing, I went for a verb instead
of a noun as this feels more natural to me when working in a
functional style.

My other choices were @with and @compose. @with had to be ruled out
as this clashes with a reserved keyword and @compose lost as @proxy is
shorter (being concise was important to me)

Also in my mind was that the synthesized methods are the delegates,
not the object being proxied.

[hlovatt]

Kevin,

I tried to look at your wave but don't have a wave account, it says
you need an invite. Any chance of inviting howard...@gmail.com.

Thanks in advance,

-- Howard.

On Nov 23, 10:59 pm, Kevin Wright <kevin.lee.wri...@o2.co.uk> wrote:
> Funnily enough, that question has also been raised on the wave I
> started for the project;

> it's public, see here:https://wave.google.com/wave/?pli=1#restored:wave:googlewave.com!w%25...

[hlovatt]

Kevin,

Can't see any source at the URL you give.

-- Howard.

On Nov 23, 11:39 am, Kevin Wright <kevin.lee.wri...@o2.co.uk> wrote:

[Patrick Wright]

Glad to see someone is working on this. In a Java implementation, I
would expect that Bar, which delegates to Foo, would have implemented
Foo, so that the clients of Bar would know what methods were available
on it, e.g. what contract it fulfilled. In Java, this would be like
implementing Comparable then delegating the comparison to a helper
class that works via reflection. I believe Lombok takes this approach
(saw some slides to that effect at Devoxx); when your class implements
an interface, you can have the methods in that interface implemented
by another class which you delegate to. I think that's a good practice
to follow, as in essence one is saying, "I implement this interface,
and it's none of your business how I do it".

I've wanted something like this for a while now, and am glad someone
is working on it, but there are two problems I don't see how you can
get around:
- synthetic methods of this sort cause debugging and stack traces to
not match the source file; for example, if the target object (the
receiver of the delegation) is null, you will receive an NPE inside
synthetic code
- I believe you can't safely publish the "this" (e.g. Bar) from inside
its constructor, meaning you can't do (in Bar's constructor) foo = new
Foo(this); this ends up (in my mind) reducing the usability of the
technique. See JCIP, chapter 3, "Sharing Objects", "Safe Construction
Practices"

In order to avoid the NPE issue, I believe one should enforce that the
field is a val, and is assigned a non-null value on construction.

Also, as far as terminology, I believe Josh Bloch calls this
"forwarding" (see Effective Java, Second Edition, item 16).


Regards
Patrick

[Graham Allan]

> I believe you can't safely publish the "this" (e.g. Bar) from inside
> its constructor, meaning you can't do (in Bar's constructor) foo = new
> Foo(this); this ends up (in my mind) reducing the usability of the
> technique. See JCIP, chapter 3, "Sharing Objects", "Safe Construction
> Practices"

Is it such a black-and-white scenario? I would have thought there were a few
caveats with which publishing 'this' in the constructor is safe. So within
either object's constructor the following should hold:

- foo = new Foo(this) is called last, after the state of the object has been
setup
- Foo does not publish Bar
- Foo does not mutate Bar
- Bar does not publish Foo within the constructor

I've not read the reference you gave though, so I expect I'm missing
something, but I'd be surprised if there were no conditions under which
passing 'this' into a composed object in the constructor is unsafe.

Kind regards,
Graham

[Patrick Wright]

> Is it such a black-and-white scenario? I would have thought there were a few
> caveats with which publishing 'this' in the constructor is safe. So within
> either object's constructor the following should hold:
>
> - foo = new Foo(this) is called last, after the state of the object has been
> setup
> - Foo does not publish Bar
> - Foo does not mutate Bar
> - Bar does not publish Foo within the constructor
>
> I've not read the reference you gave though, so I expect I'm missing
> something, but I'd be surprised if there were no conditions under which
> passing 'this' into a composed object in the constructor is unsafe.

It would be best to read Java Concurrency in Practice for a full
response. I think the answer is: there is risk involved if you publish
"this" from within the constructor, namely that (as far as I
understand it), the Java Memory Model will not guarantee that the
fields of the object are all set before the constructor ends, so the
receiver of "this" can observe a partially-constructed object. The
problem then is that, even if the risk were small, the potential
errors that could arise would be so confusing, and so hard to debug or
reproduce, that it's not worth the risk.

They specifically say that making the call the last call in the
constructor won't help; I believe this is because the operations can
be reordered.

It may be possible to get around this by forcing a flush using a
volatile or some kind of fence, but that could also be expensive if
the class if otherwise thread-bound and you are creating new instances
in a tight loop.

I'm no expert in this area, this is just one "best practice" I've
picked up along the way.

Regards
Patrick

[Kevin Wright]

I'll try and answer all the points in one mail :)

The source is here: http://github.com/scala-incubator/autoproxy-plugin
And the wiki here: http://wiki.github.com/scala-incubator/autoproxy-plugin
The tabs at the top of the page should help you...


I did also consider @forward in addition to @with, @delegate and
@proxy
The same logic applies as for delgate, I wanted a verb instead of a
noun
(so it would have to be @forwardto - no so pretty)
Plus I wanted to keep it nice and short. :)

My thinking is that "the delegates forward to the object that is
proxied", the the actual delegates (or forwarders) are the synthesised
methods and not the target object.



Also a few notes on where Scala differs from vanilla Java, things make
a bit more sense knowing this:


- all vals (immutable references) are by default made public final
(though you can override the public if you wish)

- vals and vars are actually implemented as a private field, plus
accessor methods
So:
var x = "Hello World"
becomes (in equivalent java)
private String x = "hello World!";
String x() { return x; }
String x_=(String x$ ) { x = x$; }

This isn't quite accurate, as the generatted getter doesn't require
parens to call, and '=' isn't valid in a Java method name (it's
actually encoded as "$equals"), but it does illustrate the principle.
Surprisingly, '$' is valid in an identifier, but strongly discouraged
in handwritten code because it risks clashing with generated stuff.
Also, member variables and methods occupy the same namespace in scala.

- classes have one primary constructor, other constructors can be
written but must delegate to the primary
This means that you have a guaranteed construction path for variables
that simply must be created,
there's no way to sidestep by having a second constructor - this helps
avoids a few nasty problems

The primary constructor is all the code between the opening and
closing braces of the class definition, except for method
definitions. Variables (typically) declared here get lifted to the
status of class members.

- if you need to pass 'this' to the object that you're forwarding to,
scala can help with lazy vals
i.e.

class Foo {
@proxy lazy val bar = new Bar(this)
//bar is initialised when first accessed
}

- traits...
A Scala trait is internally generated as both a class and an
interface.
This means that if Bar is a trait then it's also an interface, and
that interface can also be synthetically added to Foo's superclasses.
[in more depth: The class contains just static methods that all take
an additional 'self' argument, so they know the instance they're
operating on. Instances of the trait (usually some other class plus a
mixin) then have generated methods that forward to the static
implementations - this allows for compound types such as "Cube with
Colour" where all methods from the Colour interface are implemented in
"Cube with Colour" as forwarders.]

[Roel Spilker]

Project Lombok will not use reflection. It is a compiler plugin and will just generate the code to delegate the call to a field.

Roel

[Kevin Wright]

Another issue to contend with is what should happen if Bar is final
and possibly doesn't even have an interface.
What if instead of Bar I was proxying a String, for example? I still
want it to do *something* useful!

In this case, Foo obviously can't be used as-a String. What I plan to
do here is have Foo implement any interfaces that are available, so in
this case Foo would be useable as a CharSequence and would
additionally have any extra methods made available from the String
type - it just wouldn't expose these via a named interface.

Things like Comparable and Serializable could also make for an
interesting challenge. My current thinking is to not proxy these
interfaces, but require that they be explicitly declared if necessary.

> On Nov 24, 9:21 am, Graham Allan <grundlefl...@googlemail.com> wrote:
>
>
>
> > > I believe you can't safely publish the "this" (e.g. Bar) from inside
> > > its constructor, meaning you can't do (in Bar's constructor) foo = new
> > > Foo(this); this ends up (in my mind) reducing the usability of the
> > > technique. See JCIP, chapter 3, "Sharing Objects", "Safe Construction
> > > Practices"
>
> > Is it such a black-and-white scenario? I would have thought there were a few
> > caveats with which publishing 'this' in the constructor is safe. So within
> > either object's constructor the following should hold:
>
> > - foo = new Foo(this) is called last, after the state of the object has been
> > setup
> > - Foo does not publish Bar
> > - Foo does not mutate Bar
> > - Bar does not publish Foo within the constructor
>
> > I've not read the reference you gave though, so I expect I'm missing
> > something, but I'd be surprised if there were no conditions under which
> > passing 'this' into a composed object in the constructor is unsafe.
>
> > Kind regards,

> > Graham- Hide quoted text -
>
> - Show quoted text -

[Patrick Wright]

Hi Kevin

Thanks for your reply.

1) I think the plugin should enforce, at least with a runtime check/
assertion, that the value for a delegate field is non-null. I would
rather receive that exception than an NPE inside a synthetic method.
The lazy val trick sounds like a decent approach to publishing the
"this" reference.

2) My point about Bar and Foo re: interfaces is that I think if we say
Bar implements (or extends) Foo, then we expect Bar must implement all
of Foo's methods. If Bar just happens to implement Foo's methods by
delegating to a Foo, then it is not a Foo, except when using duck-
typing. Most of the cases where I wanted to use this sort of
delegation was where I had to implement an interface for which there
was already a reasonable implementation I could delegate to.

What I'm not sure of is what this would feel like in practice.
Presumably Bar could actually implement one or more of Foo's methods
even if delegating to Foo. But then the API of Foo is partly
implemented by concrete methods on Bar, and partly by synthetic,
hidden methods on Bar which delegate to Foo. That would seem non-
obvious when reading the source code for Bar. What's your experience
so far--do you find this confusing when using your compiler plugin?


Patrick

[Kevin Wright]

On Nov 24, 10:40 am, Patrick Wright <pdoubl...@gmail.com> wrote:
> Hi Kevin
>
> Thanks for your reply.
>
> 1) I think the plugin should enforce, at least with a runtime check/
> assertion, that the value for a delegate field is non-null. I would
> rather receive that exception than an NPE inside a synthetic method.
> The lazy val trick sounds like a decent approach to publishing the
> "this" reference.

The problem is that I can't do both together, as soon as I reference a
variable to test for null then if it's lazy it has to be initialized
at that point, which is still in the constructor...

You'll also find that null references are MUCH less common in Scala
than in Java, thanks to the Option class:
http://www.scala-lang.org/docu/files/api/scala/Option.html
http://blog.danielwellman.com/2008/03/using-scalas-op.html
http://www.artima.com/forums/flat.jsp?forum=276&thread=203082

> 2) My point about Bar and Foo re: interfaces is that I think if we say
> Bar implements (or extends) Foo, then we expect Bar must implement all
> of Foo's methods. If Bar just happens to implement Foo's methods by
> delegating to a Foo, then it is not a Foo, except when using duck-
> typing. Most of the cases where I wanted to use this sort of
> delegation was where I had to implement an interface for which there
> was already a reasonable implementation I could delegate to.

Agreed, if the object that I'm proxying has interfaces, then I'll
proxy them as well.
Proxying traits really does help here as the compiler has already
provided the necessary interface.
But I'm still going to have the plugin make a best effort to do
something useful, even if when interfaces aren't available.

> What I'm not sure of is what this would feel like in practice.
> Presumably Bar could actually implement one or more of Foo's methods
> even if delegating to Foo.

Exactly, a synthetic method is only generated if a method of the same
name and signature doesn't already exist.

> But then the API of Foo is partly implemented by concrete methods on Bar,
> and partly by synthetic, hidden methods on Bar which delegate to Foo.

> That would seem non-obvious when reading the source code for Bar.

It's pretty similar to working with the static mixins and self-types
that already
exist in Scala, where a constructed object can be pulling in
implementations
from multiple places. The difficulty in reading code is not really
any harder
than working with traits. I'm also including sensible position data
in the synthesised
methods, so that if something does go wrong then the debugger will
point to the
@proxy as the location of the error.

On the other hand, the reduction in boilerplate code only serves to
make
code more readable, and the synthesized methods are very trivial
so there's no real guess work involved.

> What's your experience so far--do you find this confusing when using
> your compiler plugin?

Actually, I'm finding the opposite, things are being simplified and
I'm sure that
if some of the compiler source I'm working with had used this
technique then
I would have come to understand it faster. The annotation very
clearly reflects
my intent as a programmer, something which isn't always immediately
obvious
when reading long stretches of source code.


> Patrick

[Graham Allan]

Having had a look at JCIP I think this is one of the cases where it is
possible to break the rule and still be safe, but as you say the potential
for hard-to-trace errors means you'd probably want to avoid it.

Thanks Patrick.

~ Graham

[Kevin Wright]

Of course, if you want to simply create an object and then forward to it, you're better off with standard mixins

instead of:

  class Foo {

    @proxy val bar = new Bar

  }

  val foo = new Foo

I would use:

  val Foo = new Foo with Bar

This requires that Bar is a trait, but that's also the recommended practice for using @proxy

If bar is being supplied by a factory method instead of being directly constructed, then @proxy becomes more relevant.

--

You received this message because you are subscribed to the Google Groups "The Java Posse" group.
To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[Kevin Wright]

correction:

  val Foo = new Foo with Bar

should read

  val foo = new Foo with Bar

[Kevin Wright]

New article on the plugin here:

http://www.artima.com/weblogs/viewpost.jsp?thread=275135

[Graham Allan]

I know you asked for feedback through Artima, but I don't have an account and
I wanted to share something.

I enjoyed the article, however I think there's information missing that is
likely to be useful. I've discussed with a few people the concept of
functionality like this for Java, and inevitably they always ask, what if
there's a conflict?

So, by your example, what would happen if you had the following:

trait A {
def foo() = println("a.foo")
def bar() = println("a.bar")
def baz() = println("a.baz")
}

trait C {
def foo() = println("c.foo")
def bar() = println("c.bar")
def baz() = println("c.baz")
}

class B(@proxy a : A, @proxy c : C) {
def bar() = println("b.bar")
}

val a = new A
val c = new C
val b = new B(a,c)
b.baz() // what happens here?


Please excuse any poor syntax, I hope you get the idea.

So personally, I would hope the Autoproxy plugin to report a compile error at
the line:
class B(@proxy a : A, @proxy c : C)

...and require that a conflicting method would have to be resolved by
explicitly choosing one of the proxies to delegate to.


But there's no mention of the policy for this in the article, which I think
would be helpful, and from my experience is probably something that people
will ask.


HTH,
Graham

[Kevin Wright]

On Thu, Nov 26, 2009 at 5:06 PM, Graham Allan <grundl...@googlemail.com> wrote:

I know you asked for feedback through Artima, but I don't have an account and
I wanted to share something.

I enjoyed the article, however I think there's information missing that is
likely to be useful. I've discussed with a few people  the concept of
functionality like this for Java, and inevitably they always ask, what if
there's a conflict?

 "the concept of functionality like this for Java"

Interesting, the number of times I've thought this to myself about different features.

The answer is invariably that Scala already has it...

 

So, by your example, what would happen if you had the following:

trait A {
 def foo() = println("a.foo")
 def bar() = println("a.bar")
 def baz() = println("a.baz")
}

trait C {
 def foo() = println("c.foo")
 def bar() = println("c.bar")
 def baz() = println("c.baz")
}

class B(@proxy a : A, @proxy c : C) {
 def bar() = println("b.bar")
}

val a = new A
val c = new C
val b = new B(a,c)
b.baz() // what happens here?


Please excuse any poor syntax, I hope you get the idea.

So personally, I would hope the Autoproxy plugin to report a compile error at
the line:
class B(@proxy a : A, @proxy c : C)

...and require that a conflicting method would have to be resolved by
explicitly choosing one of the proxies to delegate to.


But there's no mention of the policy for this in the article, which I think
would be helpful, and from my experience is probably something that people
will ask.

The current policy is that the first declaration wins.

So the final structure of B will contain:

All methods from B plus

All methods from A not matching any existing signature from B

All methods from C not matching any existing signature from A or B

This is actually by design, as B should be able to explicitly provide any methods that would otherwise be delegated.

If the demand is there and this throws up problems for people, then I can always emit a compiler warning for possible conflicts.

 

HTH,
Graham

> New article on the plugin here:
> http://www.artima.com/weblogs/viewpost.jsp?thread=275135

[Graham Allan]

> The current policy is that the first declaration wins.
> So the final structure of B will contain:
> All methods from B plus
> All methods from A not matching any existing signature from B
> All methods from C not matching any existing signature from A or B
>
>
> This is actually by design, as B should be able to explicitly provide any
> methods that would otherwise be delegated. If the demand is there and this
> throws up problems for people, then I can always emit a compiler warning
> for possible conflicts.

I'm not the best to cast judgement on this, I've only really dipped my toe
into Scala. With the little experience I have being in Java, the idea of
ordering in a definition affecting the type seems new (that's single
inheritance I guess). Correct me if I'm wrong but there is precedent for a
type's behaviour changing based on order (i.e. stackable traits), so it would
be fitting for Scala (at least, it does while looking through the narrow
blinkers of a lack of experience).

~ Graham

[Kevin Wright]

Part II now available:

http://www.artima.com/weblogs/viewpost.jsp?thread=275588

It explains a bit more about what's possible