Big GWT Application

[Alex]

I've been looking into GWT as a candidat for a pretty large AJAX
application and I can say I'm impressed.

What I can't seem to figure out is how scalable is it for large
applications? I noticed that all the client side of your code gets
compiled into a set of HTML files (with embedded script) that get sent
to the client all at once when the client loads your app.

Is my understanding correct? What happens when your application gets
bigger and bigger? Is there a clean way to divide it in modules or
something?

Thanks!

Catalin

[Nicholas]

I haven't built anything large yet so this might be completely wrong.

I would think you could split your applications functionality up into
multiple modules if you don't need it all loaded in the browser at
once.

If you consider something like a Point of Sale application. You could
have it all in one module with menus or tabs for navigation within the
module, or you could split up functionality into smaller modules and
have them on seperate HTML pages using some HTML links to navigate
between the pages. Maybe some standard HTML navigation with one page
for Inventory Control, one for Customer record maintenance, one for
entering new Sales. etc.

[claude]

I am very interested in this topic. I would like to see a mechanism in
which subsets of an application can be loaded within the primary
application.

Every significant, successful application development systems offers
something like this, be it dynamically loaded libaries or modules. This
is not a trivial thing to accomplish so I don't want to breathe onto
anybody's neck, but I'd love to encourage discussion on how this might
be done in GWT if I may.

[Ryan Dewsbury]

This would be, in my opinion, the greatest next feature GWT could have.

Something simple in the module file like this:

<late-bind class='com.rdews.client.part2.*'/>

...that would compile this code to a seperate js file and load it only
when it is used.

[VivekPrahlad]

Here's one possible way to break up large applications:

Break up your application into multiple modules, each of which would
have their own HTML page. For example if you have a screen where the
user changes settings, then that screen could be a candidate for a new
module.

Then, use IFrames to embed your modules in your application. The source
of these IFrames would, of course, point to the HTML file for a module.

This technique has worked quite well for me.

Regards,
Vivek

[Ryan Dewsbury]

That definitely would work.

Althought that is essentially two distinct clients. There is limited
communication between the two and the communication you do either has
to be through the server or through some custom javascript. For
example, if wanted to build something like netvibes.com which could
have possible thousands of plugins that the host app needs to
programatically deal with I would not want to have to load every
plugin at once.

One great advantage of GWT is that to the user there seems to be no
download or install. However as a clients features scale up the dowload
will become noticable and you will have to put some type of download
progress indicator to let the user know that its still working. Being
able to stagger the download would fix this. And a trasnparent solution
implemented within GWT would be the best for developers.

This type of optimization is inherent in regular websites where each
document is only loaded on request and that lets these regular websites
scale to massive sizes.

Out of any AJAX toolkit, GWT has the most promise to be able to help
build, maintain, and run large client applications.

[georgeuoa]

+1 for on-demand loading of code and possibly garbage-collecting it
when not referenced anymore

[claude]

I suspect the two implied elements of a dynamic module design are:

1) Interoperability with other modules, perhaps through declared
interfaces that are not name-mangled at compilation time.

2) Unloading of some kind to reclaim memory and resources.

[IamRobe...@gmail.com]

Better yet... a way compile two (or more) entry points together to
create 3 sets of files...

1. a set of files for entry point 1
2. a set of files for entry point 2
3. a set of files for shared code

GWT code gets compressed pretty well, but when you break the app into
multiple entry points you end up delivering a lot of the same code each
time.

I figure that pretty much every entry point is going to use DOM,
Widget, List/Vector, Map/HashTable, Panel, etc. If the common code
could be placed in a shared JS file it could really reduce the size of
each entry point JS file.

Rob

[Sam]

Yea I have asked for this feature before. I am not sure if it is
technical feasable although. If I call a javascript function how does
the engine know that function is loaded or not? It is not like you have
classloaders or and OS to search for shared libraries. And you would
have to search for shared js files across the net.

Maybe if the GWT compiler generated an index/lookup so that the js
client would know what js modules to load (and from where) when a
function is not found, but not sure if this is possible either.

-Sam

[claude]

It might be asking a lot to have the compiler do all the thinking for
us, but it certainly would be nice to have it generate shared modules.
This could get pretty complicated if you work with more than two
modules. Would you generate new shared modules for each intersection,
for example, or just the elements shared by all the modules? How would
you combine modules that were designed separately and maybe
precompiled?

I think a viable short term goal would be to support dynamic module
loading and if we need to manage which modules have which code
elements, and which interfaces are exposed, it would still be a great
start. The the notion of components, with clear interfaces and
well-defined dependencies, is the next obvious architectural step.

OSGi's model is pretty impressive and has become the foundation for the
whole Eclipse platform. There's probably no better showcase anywhere
for a relatively uncomplicated, scalable application component model.
OSGi is not directly applicable, but there's a lot to learn about the
fundamentals..

What's interesting about GWT is that it remains viable only as long as
the burden of download is lower than competing technolgies like Flash,
or even Applets. As applications grow in size, the compelling
proposition becomes weaker. So IMHO a componenization model, consistent
with the existing GWT design philosophy, is a necessary evolutionary
step toward a long-term, sustainable architecture.

I think the GWT team is very clever, so I'd love to hear from them
about what might be feasible in this context.

[David Goodenough]

Some work on this area has been done by the other Java to Javascript
problem (Java2Script), and as that is Open Source it might be worth
seeing how they have solved the problem and reusing their solution.

David

[IamRobe...@gmail.com]

claude wrote:
> This could get pretty complicated if you work
> with more than two modules.

I think this can be done. (of course I assume that someone from Google
is screaming just the oppisite as they read this)

In my head what I am thinking is that you would define multiple entry
points in your config. The compiler would graph method uses from each
entry point seperately. I would then compile any code used by more
than one module into the shared files. Perhaps you would be able to
set some threshold for this as an argument, like perhaps in an app with
10 modules you might require that the shared code must be used by at
least 3 modules.

The compiler would then create seperate directories in the www
directory, one for each starting point, and one for the shared code.

> Would you generate new shared modules for
> each intersection, for example, or just the elements
> shared by all the modules?

I think just one set would be best, but it doesn't really matter.

> How would you combine modules that were designed
> separately and maybe precompiled?

Perhaps you could do it with seperate modules by adding some new tag in
the config to link them, but it wouldn't work on precomiled code.

Rob

[Micha]

Hi Vivek,

> Then, use IFrames to embed your modules in your application. The source
> of these IFrames would, of course, point to the HTML file for a module.

I would be grateful if you could give a short practical example for the
use of the iframes to a dummy ;-)

Thanks,
Michael

[al0]

Ryan Dewsbury wrote:
> That definitely would work.
>
> Althought that is essentially two distinct clients. There is limited
> communication between the two and the communication you do either has

> to be through the server or through some custom javascript. ...

It is indeed for good - client code should represent only a view layer
on an application, the less logic is embedded in it (and the more
"local" this logic is) the better.

So I do not see any problems with this limited functionality, and, if
it is unsatisfactory for some purpose - it is a good indicator that
"normal" rich-client is needed instead of AJAX.

[claude]

> It is indeed for good - client code should represent only a view layer
> on an application, the less logic is embedded in it (and the more
> "local" this logic is) the better.

This is true of Model 2 web applications, but I think GWT qualifies as
a rich client environment. MVC is always applicable in the front end
but middleware models may also exist (ie: Domain Models on the business
layer). The rich client can do work like sorting, responding to
configuration changes, presenting rich, interactive views, often
without talking to the server.

One example is a file browser, which may present the file view as a
table, list, or icon view, all of which use the same data. Having to
reload pages or page segments is common in Model 2 apps, but in GWT,
you just want to swap views without another request to the server

Granted, this example fits the segmentation you are recommending but if
you want o present the same view in two iframes, you'll be loading it
twice. It would be nice if there were a formal module loading system.
Once a module is compiled and tested, it shouldn't require
recompilation to use in a bigger context.

[Sam]

Interesting concept.

Would be nice if someone on GWT would comment on the overall matter.

-Sam

[br...@google.com]

Sam wrote:
> Interesting concept.
>
> Would be nice if someone on GWT would comment on the overall matter.

This is an interesting thread. It is an area we're thinking about, but
getting the model just right will take some deep thought. Hopefully
GWT's RPC mechanism makes it easy enough to save and restore complex
state that breaking apps up into multiple pages isn't too unreasonable
as medium-term solution until we can figure out the Right Thing.

The debate about splitting out shared code comes up all the time, but
it can be a bit of red herring except in extreme cases (although we
want to hear about your situation if you disagree). If you have your
HTTP configuration set up correctly, you can make the first-time
download really fast and subsequent startup almost instantaneous.
Deployment Best Practices is an area that is admittedly overdue for
some documentation in the Developer Guide, but here are a few hopefully
useful suggestions:

1) Gzip compress your *.nocache.html and *.cache.html files. That's a
major reason they're html and not js -- there's an IE decompression bug
that happens on external js files that is avoided if you use html files
in an iframe.

2) Make sure that clients don't cache the *.nocache.html file, since
when you recompile, you don't want a cached *.nocache.html file
referring to a no-longer-extant *.cache.html file. Use either "Pragma:
no-cache" or "Cache-Control" in your HTTP response header (the
to-be-written doc should give you some examples you can copy/paste).

3) Make sure that clients do cache the *.cache.html. In fact, GWT is
designed so that clients can safely cache these files forever. You can
do this with a far-in-the-future "Expires" or a "Cache-Control" age
header in the HTTP response. You'll really be glad if you configure
your server this way, because not only does the client not need to
download the compiled script again, it doesn't even need to ask whether
it is fresh or not, which completely avoids an extra HTTP round-trip.

Just as a quick example, I gzipped one of the compiled files from
gpoker.com.

Before compression, the compiled output was 202 KB.
After compression, the compiled output was 62.5 KB.

gzipping this would cut the initial download time and bandwidth by a
factor of more than 3.

So, as it turns out, GWT's next wave of size and performance
improvements are already there waiting for you. This should buy you
enough improvement in both size and speed that by the time apps start
straining size/speed limits, we will have figured out some clever way
to make dynamic loading a reality.

-- Bruce

[Eric B]

Wow, I had no idea that browsers would uncompress files for you. Do
all browsers do this? Will this also work with files requested by
HTTPRequest? I have some large xml files whose size would be cut down
dramatically if I could compress them.

Please provide a quick example of the steps to do this, or link the
updated documentation (I couldn't find anything new about this).

Thanks,
Eric

[IamRobe...@gmail.com]

> Do all browsers do this?

Probably not *all*, but most likely all the ones you care about will,
including IE and FF. To take advantage of this you would typically
have the app/web server server do the compression. For example, if you
are running Apache you would use the mod_gzip plug-in. When the
browser requests a page it also tells the server what compression types
it supports, and the web server will either send a compressed or
uncompressed version depending on the client.

> Will this also work with files requested by HTTPRequest?

Good question. In theory it should, but I never tried.

> Please provide a quick example of the steps to do this

[compression options for Tomcat]
http://groups.google.com/group/Google-Web-Toolkit/browse_thread/thread/f995db0bd7d9db9

[mod_gzip for Apache 1.3]
http://sourceforge.net/projects/mod-gzip/

[mod_deflate for Apache 2]
http://httpd.apache.org/docs/2.0/mod/mod_deflate.html

If you are using an app/web server other than this check the docs.
This type of functionality has been around a really long time, so more
than likely what you are using will have support for it.

Rob

[claude]

br...@google.com wrote:
> Sam wrote:
> > Interesting concept.
> >
> > Would be nice if someone on GWT would comment on the overall matter.
>
> This is an interesting thread. It is an area we're thinking about, but
> getting the model just right will take some deep thought.

Thanks for taking the time to read and respond to this thread Bruce.

I agree that this requries depper thought and doing it right will
ensure longevity, which I think is paramount in this case. I wonder if
you would be willing to share some thoughts about what you may have
considered and possibly rejected or what you think is viable in the
future, with the understanding that these are not commitments.

I for one would welcome a dialog, such that the community may be able
to add value to the discussion and, perhaps, help you think through
some ideas.

A common understanding of the requirements seems valuable. I had listed
the ability to load (on-demand), unload, and interoperate (through some
kind of export mechanism) earlier. Do you see this as the fundamental
core of the topic or do you see it differently?

Also, I have come to understand that the term 'modules' is used for
both source and compiled modules, though the semantices differ
markedly. In the case of compiled modules, all dependencies are
linked-in - not so in source modules. Source modules are atomic units
with clearly defined dependencies (inherits). With the exception of
export APIs, these are effectively components. Compiled modules are an
altogether different animal, however.

I wonder if the coupling of 'module' terminology in the framework is
somewhat misleading. Perhaps there should be a clear distincton betwen
source and compiled modules, since different rules apply to each (with,
admitedly, a fair amounf of overlap, of course). It might be helpful if
the differences were articulated somewhere.

[Guy]

Hi Alex,

I think the issue you are raising is a very important one and I am an
architect of a new framework called Visual WebGui that has a different
approach that I believe to be more fit to the kind of applications you
are talking about. Basically we call the concept "Empty client" which
reflects the approach. VWG loads to the browser only the presentation
layer and the rest of the application is loaded dynamically using a
unique communication layer. You can view a demo here
(http://samples.visualwebgui.com/MainForm.wgx) which will show you what
can be done with VWG. Currently the framework only works on the .NET
framework but we are planning on a java version. By the way the
development process is done using WinForms like API and design time
capabilities and is planed to be swing like development on the java
platform. If you would like to read some more about the technology you
can refer to our site here http://www.visualwebgui.com.

Guy

[Chad Bourque]

Guy,

VWG looks very nice, but does it not support any browser other than IE? The sample page doesn't load in my FF.

Chad

--
There are two types of people in the world:
  * Those who need closure

[jac...@gmail.com]

Guy,

My FireFox couldn't load sample page.

Jackz08

[Guy]

FireFox support will be available during next week we are wrapping it
up on the FireFox support.

Guy

[ash]

guy,

well done!

some immediate feedback:
* needs some more stability in IE (i had ie crash on me)
* needs some widgets to support more keyboard driven actions for better
accessibility
* needs to support command history (ie. browser back/forward)
* one of the best aspects of your approach is that you have leveraged a
well known component model (ie. winforms) and leveraged the language
features (ie. Andres Hejlsberg' delegates, one of the things that he
brought from delphi to c#)
* another aspect that i like is that you have leveraged the vs build
environment. i like how you pulled off the swingwt trick with only
namespace differences. swingwt has gone one step up and lets your code
actually bind to swing but then using a precompilation ant task, it
switches everything.

ive been slowly working on something similar to this with my gems oss
project using gwt as the driving technology, so from experience,
_bloody_ well done!

> are talking about. Basically we call the concept "Empty client" which
> reflects the approach. VWG loads to the browser only the presentation
> layer and the rest of the application is loaded dynamically using a
> unique communication layer. You can view a demo here

can u elaborate on the approach... is it a dwr style approach?

> framework but we are planning on a java version. By the way the
> development process is done using WinForms like API and design time
> capabilities and is planed to be swing like development on the java

have you started design? what are the key architectural elements? are
you planning on extending eclipse's visual editor for the builder? what
is your roadmap and timeline?

btw im assuming the java version will be gwt based?

im asking these questions b/c if you do it well, you will put me out of
my misery<g>.

i've currently implemented a component model very similar to winforms
(modeled on delphi's visual component library with both visual and
non-visual components). ive also implemented widget/component
(de)/serialisation as json string, forms, drag&drop and working on
drag&dock.
my goal is simple, build feature rich, enterprise ria using gwt in rad
like fashion to promote better relationships and quicker feedback loops
with the customer. i find myself incrementally building this b/c
nothing else is addressing this problem. hence, the sooner you release
a gwt/winforms version of your product, the sooner i can stop. im
already up against key features that gwt does not support: reflection
and dynamic linked modules. it kinda sucks developing a builder tool in
gwt without introspection or reflection. heck, ive even gone to the
effort of investigating how to integrate java2script & gwt so that i
can have these features.

btw, if you choose to open source the gwt version of the product, i can
contribute and commit code. afaik the sooner this is done the better.

good luck. pls keep me posted.

rgds ash

[Tom Chen]

VWG looks nice. But after playing with a while in IE , got an IE error.

[Guy]

Hi Ash,

As for stability in IE it seems that it is related to specific versions
of IE and only to controls that are using the createPopup. We are still
investigating the issue. The system is running in production in many
places and it is not that often that we see this issue but we still are
enhancing as we go.

Support for next/back is part of our road map and we have a proof of
concept for this issue. Because of the WinForms module we plan on
adding an event on the Form class that will let the programmer react to
back/forward. First version of this feature will be available shortly.

As for the VS build and the "Empty Client" approach... We do not
override the building mechanism of MS build. The compiler is building a
100% pure .NET assembly. As opposed to GWT, we are a runtime
environment this means. This we are not generating JavaScript
code/HTML/etc. We have a patented communication layer that provides
means of incrementally updating the client from the server. Client
events are been queued and sent to the server that in turn runs the
user code, that in turn causes components to require update, which in
turn results in an update command that is sent to the client to update
specific browser parts including opening and closing windows (Which we
have implemented on FireFox using floating layers).

The "Empty Client" approach has many advantages including full runtime
OOP support and unlimited application size. Downloaded volume is not
affected by how many screens you have or how much logic do you have.
This means we are an excellent choice for enterprise scaled apps. We
also do not expose logic ,unwanted data or web services to the client
meaning we are extremely secured. You will only be able to perform your
actions quicker using HTTP injection but can not access unauthorized
data. Although the concept seems to require more server processing
power it actually reduces server processing as we do not construct and
destroy objects on every client request. We render small chunks of
update commands even when application gets bigger. Over all we do
increase server memory consumption a bit but we dramatical reduce
server CPU usage and generated band width which compensate our server
based computing approach in terms of scalability and responsiveness.

We have tested a Java/Swing based POC which was very successful and we
are planing on releasing a Java version in mid 2007. We have checked
checked to compatibility with MONO and GRASSHOPPER and we will soon
release compatible version allowing to deploy VWG applications on LAMP
environment. First version of FireFox supporting VWG was released this
week and it looks pretty stable. There are some behavior issues but it
seems to be working nice.

We would love to have you help in the Java version, you sound like a
serious developer with lots of experience in this field.

Sincerely,
Guy Peled

[sk]

Has anyone implemented this suggestion. I tried and these are the
problems I faced.

1. Both Firefox 1.0.7 and IE 6.0 don't send the 'accept-encoding'
field.
2. But I modified Tomcat3.1(there is a reason I am using very old
version) to send a .gz file if the requested file is not present. I
removed the original html version and replaced it with a .gz version.
My app server sends the .gz version of the file with 'Content-Encoding'
set to 'gzip'.
3. The file shows up fine in Mozilla but not in IE. IE throws up the
open/save dialog.

Did anyone else have any luck with this?

[ash]

sorry i missed your reply. i rely on the google groups interface to
report to me which threads have been responded to. i totally missed
this. come to think of it, ive missed a few more. hmm.

Guy wrote:
> We would love to have you help in the Java version, you sound like a
> serious developer with lots of experience in this field.

im sorry u got the wrong impression. im a total lunatic that is hoping
to pursue a career as a monk after developing a decent gwt application
:-)

on a serious note, pls let me know where your java poc impl is.

---
ive just noticed your ff impl today and took a peek inside with
firebug.

to me it seems like your doing something very similar to xml11. the
difference being that your currency is winforms controls and xml11 is
awt widgets. its basically the xserver concept with a different
implementation (see http://www.xml11.org/documentation/).

btw, have u and ur project been invited to participate in the open ajax
alliance like xml11. given that both are similar in my mind, u may be
able to gain more weight on views and keep those client heavy guys
honest.

rgds ash
http://www.gworks.com.au