GWT Enhancement

[jdoubleu]

Are there any plans to build out the reflection capabilities?
Specifically, the ability to reflect on instances at runtime, discover
and invoke methods dynamically. Even the basics would be very useful.

-Jon

[Scott Blum]

There aren't any plans to do that, because it would totally destroy all possibility of optimizing.  This isn't a small thing... a script might get ten times bigger if we couldn't prune unused methods.  However, you can accomplish almost everything you'd want to do with reflection via deferred binding and generators.  It's essentially "compile time reflection".  You reflect at compile time and then generate code to do what you need done at run time.  I'd suggest looking into that if you want to do reflection like stuff.

[mP]

You add extra annotations that include/exclude all of a class and its
methods in some form of meta data structure. This might help if users
want to use reflection on their value objects. They would have to
realise that using reflection anywhere else would fail due to class
meta data missing.

[jdoubleu]

I hadn't really considered the impact on code-size optimization. I
understand the issue there. I'm guessing that as of now, the compiler
just compiles those classes/methods that are referenced explicitly from
the entry point classes.

To mP's point tho, it does seem like one could use annotations or
compiler options of some kind to give hints that certain
classes/methods should be compiled despite the fact that they aren't
referenced directly. a basic introspection api or an impl of the
existing reflect package could consider only compiled classes.

Certainly, I could accomplish most or all of what I want to do with
code generators but all things being equal, reflection would be nicer
in most cases.

[Joel Webber]

All,

I certainly understand the strong desire that many have to use runtime reflection in GWT client code.  It's become a common pattern in a lot of Java code lately, and it probably feels a bit strange to no longer have it.  At the same time, Scott's point about code optimization is an extremely strong one.  Code size and optimization are critically important in the generated Javascript, and every decision we make that could negatively impact this must be taken *very* seriously.

I also am a little wary of "hint" solutions that open the door to hard-to-debug runtime errors.  I'm not saying that Miroslav's solution is not viable, but I have to admit that I'd be a little uncomfortable adding a new set of runtime failure states.

Scott described generators as 'compile-time reflection' earlier.  I'd be the first to admit that this stuff is not as clearly documented as it could/should be, but it really does give us an enormous amount of leverage to solve problems that are normally solved through reflection.  Maybe we should consider how we can make this facility easier to use (or at least provide helpers for common code-generation cases)?  I think it would be a good idea to start by considering specific use-cases and backing into useful facilities from there, rather than starting with reflection as a requirement.

joel.

[Jon Wells]

Jeol, thanks for your response.

When "generators" were recommended to me, I didn't realize the Scot was
making reference to an actual GWT feature, instead I thought he meant
code generation in general.

I come across uses for reflection and some of the other dynamic
features of the language pretty frequently, but my most common need is
for some kind of dependency injection container like thing. As it is,
I'm doing most of my wiring of client-side components manually which is
ok, just a little tedious (and ironically its a lot of code).

Perhaps a simple DI container would be a nice place to start. I can
live without it, but it would be cool.

Jon

[Scott Blum]

On 12/24/06, jdoubleu <jon....@gmail.com> wrote:

Certainly, I could accomplish most or all of what I want to do with
code generators but all things being equal, reflection would be nicer
in most cases.

To which I'd have to say, all things are not equal. :)  Reflection may arguably be nicer for the developer, but code generation is a lot nicer for the end user.  This project has a very strong bias towards the users.  So much so, that we want to make it easy to for developers to do things that will be good for the users and difficult to do things that will be bad for them.  I think lack of reflection is a perfect example of this.

One thing I'd like to point out, however, is that code gen actually has a lot of advantages for the developer when applied correctly.  With reflection, the possibility of a runtime error is omnipresent.  There's no escaping it, because you're interacting with code in a data-driven manner.  With code gen, most programming errors will be caught at compile time.  You generate some code which is then compiled into the rest of the app.  The fact that it compiles provides a degree of static checking that what you generated is correct.

Scott

[Jon Wells]

> To which I'd have to say, all things are not equal. :) Reflection may
> arguably be nicer for the developer, but code generation is a lot nicer for
> the end user.

I can live without reflection, but my end users are concerned with the
having the most featureful and high-quality applications in the most
timely manner. They don't really care about whether reflection is used
or not. To the extent that I'm experienced with using reflection and
would be aware of the code-size implications (of which there are little
or none if it were implemented correctly), I can be a little nicer to
my users using reflection.

I understand the desire to keep the toolkit simple and the code-size
performant, and on that basis perhaps it doesn't make sense, but I
don't buy the argument that reflection is somehow too "dangerous" to be
made available. I can't think of a major open source project that
doesn't make use of it in some way.

-Jon

> One thing I'd like to point out, however, is that code gen actually has a
> lot of advantages for the developer when applied correctly. With
> reflection, the possibility of a runtime error is omnipresent. There's no
> escaping it, because you're interacting with code in a data-driven manner.
> With code gen, most programming errors will be caught at compile time. You
> generate some code which is then compiled into the rest of the app. The
> fact that it compiles provides a degree of static checking that what you
> generated is correct.
>
> Scott
>

> ------=_Part_5714_327085.1167843855564
> Content-Type: text/html; charset=ISO-8859-1
> X-Google-AttachSize: 1543
>
> On 12/24/06, <b class="gmail_sendername">jdoubleu</b> &lt;<a href="mailto:jon....@gmail.com">jon....@gmail.com</a>&gt; wrote:<div><span class="gmail_quote"></span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> Certainly, I could accomplish most or all of what I want to do with<br>code generators but all things being equal, reflection would be nicer<br>in most cases.<br></blockquote></div><br>To which I&#39;d have to say, all things are not equal. :)&nbsp; Reflection may arguably be nicer for the developer, but code generation is a lot nicer for the end user.&nbsp; This project has a very strong bias towards the users.&nbsp; So much so, that we want to make it easy to for developers to do things that will be good for the users and difficult to do things that will be bad for them.&nbsp; I think lack of reflection is a perfect example of this.
> <br><br>One thing I&#39;d like to point out, however, is that code gen actually has a lot of advantages for the developer when applied correctly.&nbsp; With reflection, the possibility of a runtime error is omnipresent.&nbsp; There&#39;s no escaping it, because you&#39;re interacting with code in a data-driven manner.&nbsp; With code gen, most programming errors will be caught at compile time.&nbsp; You generate some code which is then compiled into the rest of the app.&nbsp; The fact that it compiles provides a degree of static checking that what you generated is correct.
> <br><br>Scott<br><br>
>
> ------=_Part_5714_327085.1167843855564--

[Scott Blum]

On 1/3/07, Jon Wells <jon....@gmail.com> wrote:

I can live without reflection, but my end users are concerned with the
having the most featureful and high-quality applications in the most
timely manner.  They don't really care about whether reflection is used
or not.  To the extent that I'm experienced with using reflection and
would be aware of the code-size implications (of which there are little
or none if it were implemented correctly), I can be a little nicer to
my users using reflection.

What I'm saying is that they care to the extent they are downloading a 200k script as opposed to a (say) 1 MB script, and that the application runs quickly.  I don't have hard data, but it's been observed that IE (for example) runs slower the more JavaScript you've parsed.

That being said, if you've figured out a way to implement reflection without impacting code-size, I'd love to hear it.

Scott

[Jon Wells]

> That being said, if you've figured out a way to implement reflection without
> impacting code-size, I'd love to hear it.

I'm not an expert in the inner workings of the compiler, etc. so I
won't be very precise, but here are some observations:

* If you consider only the code that will execute at runtime,
reflection must not necessarily increase the code size, in fact it
should reduce the overall code size. obviously, the difficulty is in
compiling the necessary code and as much as possible only the necessary
code.

* I'm assuming right now that the compiler works by recursively
compiling all of the classes/methods referenced from the entry point(s)
called out in the module.

* My code is generally divided into a series of frameworks and plugins
where plugins are supplied to frameworks to form applications. This is
an abstract use case for reflection. All the code that is needed at
runtime could be compiled using recursive compilation if I were able to
specify a handful of "root" classes to the gwtcompiler where each
framework has a some root classes, each plugin has some root classes,
and each entry point is a root class.

* Alternately, I could use annotations on a class or method to ensure
that it be compiled. Maybe I could just mark the "root" classes in my
module with a special annotation?

* If there were some facility to load classes/methods with the caveat
that I am responsible for ensuring that they have been compiled. The
same caveat of recursive compilation exists with javac -- that is to
say that if i compile A.java and A depends on B depends on C which
dynamically loads D, D will not be compiled.

* If there were some Dependency Injection facility built-in, then
perhaps the compiler could compile classes and do dependency injection
based on a Spring applicationContext.xml file?

These are just some ideas.

Jon

[Jon Wells]

Another possibly crazy idea for implementing reflection while limiting
code size would be to do some sort of dynamic script loading kinda
similar to the AppletClassLoader where scripts for classes are loaded
on demand from the server. Not really sure if this is possible, just a
thought.

[Scott Blum]

On 1/3/07, Jon Wells <jon....@gmail.com> wrote:

* If you consider only the code that will execute at runtime,
reflection must not necessarily increase the code size, in fact it
should reduce the overall code size.  obviously, the difficulty is in
compiling the necessary code and as much as possible only the necessary
code.

Can you put this idea into more concrete terms?  Imagine this psuedocode:

String className = getTextFromUser();
String methodName = getTextFromUser();
Class.forName(className).findMethod(methodName).invoke();

In order to handle this correctly, you'd have to not dead strip ANY class or ANY method in the entire accessible code base.  How can you know at compile time what the runtime values of the strings will be?
 

* My code is generally divided into a series of frameworks and plugins
where plugins are supplied to frameworks to form applications.  This is
an abstract use case for reflection.  All the code that is needed at
runtime could be compiled using recursive compilation if I were able to
specify a handful of "root" classes to the gwtcompiler where each
framework has a some root classes, each plugin has some root classes,
and each entry point is a root class.

*  Alternately, I could use annotations on a class or method to ensure
that it be compiled.  Maybe I could just mark the "root" classes in my
module with a special annotation?

Something like this might work, but this is very different from general-purpose reflection.  And even assuming I could use some kind of metadata scheme, which would be incompatible with Java anyway, it would still necessarily increase the code size.  Right now all of the identifiers get obfuscated into very small strings to conserve space; I'd have to put in full, long name identifiers for anything you might want to reference through reflection.

* If there were some facility to load classes/methods with the caveat
that I am responsible for ensuring that they have been compiled.  The
same caveat of recursive compilation exists with javac -- that is to
say that if i compile A.java and A depends on B depends on C which
dynamically loads D, D will not be compiled.

That seems like a recipe for some nightmarish debugging situations... where the behavior of your app depends on the particular optimizations that have been performed.

Some of your ideas actually sound pretty similar to some of the JavaScript API discussions we've been having.  Being able to access Java stuff from JavaScript in a controlled manner might solve some of the same sort of problems you're looking to solve.

Scott

[Jon Wells]

Scott Blum wrote:
> On 1/3/07, Jon Wells <jon....@gmail.com> wrote:
> >
> > * If you consider only the code that will execute at runtime,
> > reflection must not necessarily increase the code size, in fact it
> > should reduce the overall code size. obviously, the difficulty is in
> > compiling the necessary code and as much as possible only the necessary
> > code.
>
> Can you put this idea into more concrete terms? Imagine this psuedocode:
>
> String className = getTextFromUser();
> String methodName = getTextFromUser();
> Class.forName(className).findMethod(methodName).invoke();
>
> In order to handle this correctly, you'd have to not dead strip ANY class or
> ANY method in the entire accessible code base. How can you know at compile
> time what the runtime values of the strings will be?
>

This makes me think about the ways in which I use and have seen
reflection used. Generally, there's some framework that is reflecting
on some objects. Sometimes the framework knows exactly the methods the
objects implement like in the case where some object has a collaborator
that it interacts with via a "service provider interface" and the
implementation is loaded and injected at runtime. An example of this
would be a dependency injection container.

Sometimes the framework knows a little about the objects its reflecting
on (in terms of the methods they implement) like in the case of JUnit
where JUnit reflects on test* methods.

And on the very far end of the spectrum there are frameworks/aspects
that know little or nothing about the classes they're reflecting on
like an "object command shell" or code like your example above.

Not sure if I'm providing much useful information, just some
observations about typical usages of reflection. And a dependency
injection container (a la Spring's XmlBeanFactory) could provide some
of the dynamic-ness w/o implementing the Java API for reflection. If
I had a dependency injection container, I think my script size could be
smaller than without.

>
> * My code is generally divided into a series of frameworks and plugins
> > where plugins are supplied to frameworks to form applications. This is
> > an abstract use case for reflection. All the code that is needed at
> > runtime could be compiled using recursive compilation if I were able to
> > specify a handful of "root" classes to the gwtcompiler where each
> > framework has a some root classes, each plugin has some root classes,
> > and each entry point is a root class.
>
>
> * Alternately, I could use annotations on a class or method to ensure
> > that it be compiled. Maybe I could just mark the "root" classes in my
> > module with a special annotation?
>
>
> Something like this might work, but this is very different from
> general-purpose reflection. And even assuming I could use some kind of
> metadata scheme, which would be incompatible with Java anyway, it would
> still necessarily increase the code size. Right now all of the identifiers
> get obfuscated into very small strings to conserve space; I'd have to put in
> full, long name identifiers for anything you might want to reference through
> reflection.

I'm thinking of using something AspectJ's pointcut expression language
to call out what methods need to be honored. I'm not exactly sure what
you mean by "not compatible with Java", but this language is used to
configure aspects for AspectJ and Spring's AOP stuff, so it works in
the Java realm.

As for the identifiers, I imagine that you could solve this problem
with some type of lookup table that maps the pretty names to the
mangled. Perhaps the map could be downloaded lazily if you're
concerned about the size.

>
> * If there were some facility to load classes/methods with the caveat
> > that I am responsible for ensuring that they have been compiled. The
> > same caveat of recursive compilation exists with javac -- that is to
> > say that if i compile A.java and A depends on B depends on C which
> > dynamically loads D, D will not be compiled.
>
>
> That seems like a recipe for some nightmarish debugging situations... where
> the behavior of your app depends on the particular optimizations that have
> been performed.

My pt was that this situation is already present when using Javac. I
worked for a company where we wrote applet games. In order to ensure
we only compiled and jarred what the applet needed, we compiled a few
"root" classes including the applet class itself (analogous to the
EntryPoint) and the root class of any SPI implementations. This worked
well for us because our codebase was well-organized and the alternative
was literally calling out every class file we wished to include. My
pt is if your codebase is a mess, yes I can imagine this creating some
difficult debugging situations, but if your codebase is well-organized
you can benefit from it.

>
> Some of your ideas actually sound pretty similar to some of the JavaScript
> API discussions we've been having. Being able to access Java stuff from
> JavaScript in a controlled manner might solve some of the same sort of
> problems you're looking to solve.
>

My head is in the sand when it comes to JavaScript, I need to do some
studying.

What about dynamic script loading? Not sure what the issues are here,
but was/is it being considered? I could imagine some classes/methods
being "stubbed" initially with the idea that if they were referenced
they would be downloaded. This sounds sophisticated I know, but it's
a thought.

Jon

> Scott
>
> ------=_Part_9051_11592306.1167938963980
> Content-Type: text/html; charset=ISO-8859-1
> X-Google-AttachSize: 3551
>
> On 1/3/07, <b class="gmail_sendername">Jon Wells</b> &lt;<a href="mailto:jon....@gmail.com">jon....@gmail.com</a>&gt; wrote:<div><span class="gmail_quote"></span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> * If you consider only the code that will execute at runtime,<br>reflection must not necessarily increase the code size, in fact it<br>should reduce the overall code size.&nbsp;&nbsp;obviously, the difficulty is in<br>compiling the necessary code and as much as possible only the necessary
> <br>code.</blockquote><div><br>Can you put this idea into more concrete terms?&nbsp; Imagine this psuedocode:<br><br>String className = getTextFromUser();<br>String methodName = getTextFromUser();<br>Class.forName(className).findMethod(methodName).invoke();
> <br><br>In order to handle this correctly, you&#39;d have to not dead strip ANY class or ANY method in the entire accessible code base.&nbsp; How can you know at compile time what the runtime values of the strings will be?<br>
> &nbsp;</div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">* My code is generally divided into a series of frameworks and plugins<br>where plugins are supplied to frameworks to form applications.&nbsp;&nbsp;This is
> <br>an abstract use case for reflection.&nbsp;&nbsp;All the code that is needed at<br>runtime could be compiled using recursive compilation if I were able to<br>specify a handful of &quot;root&quot; classes to the gwtcompiler where each
> <br>framework has a some root classes, each plugin has some root classes,<br>and each entry point is a root class.</blockquote><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> *&nbsp;&nbsp;Alternately, I could use annotations on a class or method to ensure<br>that it be compiled.&nbsp;&nbsp;Maybe I could just mark the &quot;root&quot; classes in my<br>module with a special annotation?</blockquote><div><br>Something like this might work, but this is very different from
> general-purpose reflection.&nbsp; And even assuming I could use some kind of
> metadata scheme, which would be incompatible with Java anyway, it would still necessarily increase the code size.&nbsp; Right now all of the identifiers get obfuscated into very small strings to conserve space; I&#39;d have to put in full, long name identifiers for anything you might want to reference through reflection.
> <br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">* If there were some facility to load classes/methods with the caveat<br>that I am responsible for ensuring that they have been compiled.&nbsp;&nbsp;The
> <br>same caveat of recursive compilation exists with javac -- that is to<br>say that if i compile A.java and A depends on B depends on C which<br>dynamically loads D, D will not be compiled.</blockquote><div><br>That seems like a recipe for some nightmarish debugging situations... where the behavior of your app depends on the particular optimizations that have been performed.
> <br></div><br></div>Some of your ideas actually sound pretty similar to some of the JavaScript API discussions we&#39;ve been having.&nbsp; Being able to access Java stuff from JavaScript in a controlled manner might solve some of the same sort of problems you&#39;re looking to solve.

> <br><br>Scott<br><br>
>

> ------=_Part_9051_11592306.1167938963980--

[mr_etc_nemesis]

can someone please post a pointer to to docs on this? thanks

[Scott Blum]

There really isn't good doc for this right now.  Any volunteers? :)

If you're wanting to get into this, I can suggest a few sources of info.

1) Search the user group for threads between claude.e.duguay and me regarding generators.

http://groups.google.com/group/Google-Web-Toolkit/search?group=Google-Web-Toolkit&q=author%3Aclaude.e.duguay%40gmail.com+generator&qt_g=1&searchnow=Search+this+group

2) Look at com.google.gwt.junit.rebind.JUnitTestCaseStubGenerator for a fairly simply example.

3) Look at com.google.gwt.user.rebind.rpc.ServiceInterfaceProxyGenerator for a very powerful example.

HTH,
Scott