One of the tougher aspects of programming a well functioning AJAX application is the asynchronous population of data into the application. The problem is that loading all data at initialization leads to poor performance due to long start-up times, while trying to juggle the population of the widgets over time can be complex. Detailed below is one possible solution.
The questions I want the most feedback on are
1) Is it worth doing?
Will we get enough bang for the buck by providing data widgets to justify the hassle of creating and maintaining another API?
2) Is there a better approach?
Below we propose a single solution to the problem. Are there better solutions out there? Are there server-side frameworks that should be kept in mind while designing our client-side solution?
---------------Design Goals----------------
Server agnostic
The widgets themselves should not care about the actual server technology or how it represents its data.
Seamless user experience
The UI should be responsive even while data is being requested from the server.
Easy to use
It should be relatively easy for a user to figure out how to use a data enabled widget.
Resuable Components
As the data management code will be somewhat complex ideally the same code could be reused by all the data enabled widgets.
---------------Design Overview---------------------
Each data-aware widget must generate unique data requests that are handed to a data broker for processing.
The data-aware widgets are responsible to creating the data request and processing the response objects associated with that data request.
The data broker is responsible for batching, fetching, creating a response object, and caching data requests.
To use a data-aware widget, you must supply it an object conforming to the data broker interface, detailed below. For convenience, GWT will supply an abstract data broker which will manage caching and batching of requests, leaving only the actual server communication for user implementation.
------------Proposed Widget List----------------
Data Tree:
A tree populated via data from a data broker.
Data Table:
A scrollable table populated via data from a data broker.
Data ListBox:
Creates a list of options via data from a data broker.
Suggest Popup:
A selected suggest popup where suggestions are populated from the server based on text strings.
Suggest Box:
Composite widget using the suggest popup and a single text box to create a Google style suggest box.
-------------- Proposed Data API ----------------
Data Broker:
Each Data widget is associated with a single Data Broker. The Data broker is responsible for gathering and processing data requests and using the returned information to populate data widgets. It knows how to take a string payload from the server and translate it into a response object.
The Data Broker is data-agnostic. It should only be able to communicate with the server, cache data responses from the server, and re-assemble server side response. It may use whatever technology is most convenient for the user, RPC, JSON, HTTPRequests, etc. Only the server response and the request object should be aware of the underlying data.
It is the also the responsibility of the Data Broker to batch requests from the client, and cache the responses, in order to reduce the number of round trips required.
/**
* DataBroker interface used by data- aware widgets.
*/
public interface DataBroker {
/**
* Request object to be sent to the data broker.
*/
interface Request {
/**
* Fills the current widget with the data from a response. Each request may
* have multiple response objects associated with it.
*/
void fill(Response response);
/**
* Request data.
*/
String getRequestData();
}
/**
* Response object to be given to the <code>fill </code> method of the
* request object.
*/
interface Response {
/**
* Gets the index of this response. For example, if this is the third
* response to a query, then the index would be 2.
*/
int getIndex();
List getValues();
}
/**
* Registers a <code>Request</code> object with this data broker.
*/
void get(Request q);
}
DataBroker.Request:
The request object is sent to the data broker by the data widget. The request should have a widget specific fill method. Bellow is a sample request that might be used to fill a data table row.
/**
* Data aware table
*/
public class DataTable extends FlexTable {
class RowRequest implements Request {
int row ;
int col ;
RowRequest( int row) {
this.row = row;
}
public void fill(Response results) {
List values = results.getValues ();
int end = col + values.size();
for (int i = col; i < end; i++) {
String value = (String) values.get(i);
setHTML( row, i, value);
}
col = end;
}
public String getRequestData() {
return DataTable. this.getName() + ".Row." +Integer.toString(row );
}
}
…
Data Response:
The response object is created by the data broker to be passed into the fill method of the request object. Each request may have multiple response objects returned by the server. It is the responsibility of the data broker to ensure that the fill method is called sequentially on all returned responses. Each response object contains a list of values and an index. So, for example, if a request generates four response objects, then the response objects will have indexes 1 through 4.
Typical Control flow for a single data request:
Data-aware widget:
Creates Request object.
Passes Request object to DataBroker.
DataBroker:
Collates Request objects together.
Sends Requests to server.
Gets results from server.
Creates Result objects from server response.
Pass Results objects to Request object in index order.
Request
Uses Results object to fill data in data-aware widget
P.S.
Thanks to Henry and Dan for all their help on this!
Scott Stirling
Framingham, MA
Put another way, there should be a core set of widgets in GWT that a
developer can use to build an average e-commerce app, with an eye
toward the basic fact that GWT is a way for me to code Javascript and
DOM in Java. There's no pretense of preserving interfaces post-compile
time.
Let me reiterate my point: I don't want a proliferation of GWT widgets
for every little feature. I want the core widgets to be powerful out
of the box. Sorting and data binding seem like reasonable core
features to me. I'm not looking to build a business coding GWT
widgets. I want to be able to build a decent web app with GWT alone.
Scott Stirling
Framingham, MA
Do you remember Delphi/C++Builder? There was set of just GUI
component, for manual operations. And there was set of subclasses that
add data-aware features. So, don't modify core widgets. If you need
data binding, create additional library.
On the other hand, there are GWT internal implementation design
choices to be made. You may have a point there. I don't actually have
an opinion I want to argue for at the moment to that point.
I am thinking about Emily's question of whether data binding is worth
it. I am thinking my answer is, why not try it? Create a branch if
necessary, or commit it to head as an optional feature.
Scott S.
Framingham, MA
I wondered the same thing myself! - Scott
I will gladly write up my current issues and concerns and put them up
for discussion. (As well as what I have so far in implementation).
On Dec 30, 12:01 am, "Miroslav Pokorny" <miroslav.poko...@gmail.com>
wrote:
> I think it should.
>
> Real tables rather than tables for layout, with columns always have headers
> which is probably why the distinction was made to include data cells (TD's)
> and table headers (TH's).
>
> On 12/30/06, Emily Crutcher <e...@google.com> wrote:
>
>
>
>
>
> > Are adding headers something that would be helpful to our users? It seems
> > like a reasonable extension to me, can anyone think of a reason it
> > should/should not be added to HTMLTable?
>
> > On 12/29/06, Scott Stirling <scottstirl...@gmail.com> wrote:
In my opinion, yes. I think the SWT/JFace analogy from Joel is a good
one. It's a good design to follow, but it's important to note that
JFace is provided by eclipse.org and makes the API a lot easier to use
in most cases. So, out of the box you have the choice between the
low-level SWT or higher level JFace (in most cases I would choose the
latter).
In addition, a lot has been talked about data binding in the Java GUI
world in the recent past. JFace included an experimental data binding
API in eclipse 3.2 and they've been working on the final version for
Eclipse 3.3. For Swing, there's the Beans Binding JSR[1]. So, it seems
to me that people have found the third-party binding libraries like
JGoodies useful enough that the toolkits decided to have it out of the
box. I've used the JGoodies bindings library in the past and it was
very useful for maintaining the data in sync between widgets and data
source.
Opera: while the order of the thead and the tfoot elements should not
matter with respect to tbody elements they do for Opera if they are
manipulated via JavaScript. (Order doesn't matter in raw HTML source
for opera.) For Opera to render thead and tfoot correctly after
updating the DOM with JavaScript they must be the first and last childs
of the table element.
Safari: With some DOM inserts to the middle of the table Safari doesn't
re-layout the table correctly and visually tbody elements will appear
on top of thead elements. The quick hack I found around this is to
quickly add and them remove an empty caption element to the table
element after your updates and Safari should re-render the table
correctly.
I'm not sure if those bugs will affect GWT's HTMLTable implementations
because they work with the expectations of the basic HTML 4 table model
plus one tbody element. But if they do show up, hopefully that will
save you a number of hours of debugging. The source may have bug work
arounds I cannot recall off the top of my head too.
HTML 4 Tables:
http://www.w3.org/TR/html4/struct/tables.html
GWT-Stuff:
http://code.google.com/p/gwt-stuff/
A table demo is linked on the right side.
Thanks Sandy! Do you know if legal ever got your contributors agreement? I know you had talked about it with Scott, but I don't know what the resolution was.
I have also almost completed an implementation of the full 4.01 table
spec. Can I ask how you handled events?
I haven't traced through removal of individual cells as of yet, but I
have about a mid-maturity cut that makes the table structural sections
explicit (and gets around the Safari and Opera limitations by ignoring
the row indexes unless in a specific section) but tends to break bits
of backward compatibility.
The following document was to be my post discussing my implementation:
http://docs.google.com/View?docid=dhtrqvrz_8fhxrnw
Also, the patches are sort of mid-lifespan (and may have some widget
event holes as well as various small formatting bugs), but usable in
general. I can gladly send them to anyone willing to try them out or
look at them.
Thanks!
David
On Jan 1, 10:53 am, "John Tamplin" <j...@google.com> wrote:
> On 1/1/07, Emily Crutcher <e...@google.com> wrote:
>
>
>
> > Thanks Sandy! Do you know if legal ever got your contributors agreement?
> > I know you had talked about it with Scott, but I don't know what the
> > resolution was.Yes, you can see in the
> CONTRIBUTORS<http://google-web-toolkit.googlecode.com/svn/trunk/CONTRIBUTORS>file.
These terms aren't official but it's how they were described when I
first read about them and they stuck in my brain.
I chose not to start with the GWT provided table widgets. HTMLTable and
it's implementations are good at being intuitive for the programmer
thinking about a grid but their design doesn't work for the advanced
HTML Table model and efficient DOM manipulation.
On Jan 1, 7:12 pm, "spaceLe...@gmail.com" <spaceLe...@gmail.com> wrote:
> Can I ask how you handled events?
Each row group and table row is a UIObject instance and in GWT a
UIObject can be the source of events but only Widgets can receive
events. So when an event fires for my table widget it's onBrowserEvent
method redispatches the event to the table row group object's
onBorowserEvent method which then send the event to the table row
object's onBorowserEvent method. The programmer can add event listeners
on each of those object types which can deal with events.
In my table, table cells (td and th) extend SimplePanel so the normal
GWT event system just works there like any other Widget.
> I haven't traced through removal of individual cells as of yet, but I
> have about a mid-maturity cut that makes the table structural sections
> explicit (and gets around the Safari and Opera limitations by ignoring
> the row indexes unless in a specific section) but tends to break bits
> of backward compatibility.
IMO, the HTMLTable in GWT just wasn't designed to have header/footer
rows and it isn't worth trying to cram that feature into it. To
maintain existing behavior for backwards compatibility you need to add
all sorts of logic to deal with corner cases all over the place. This
will hurt performance and takes away from the ease of use Grid and
FlexTable currently have. In the end I think it's a net loss.
If you're going to do header/footer in HTMLTable widgets then you
probably should just fake using special logic it instead of using table
row groups. For example things like row spans work differently once you
introduce row groups. In the simple table model a header cell with a
row span of 99999 would extend down the whole length of the table but
not cause the table to actually have 99999 rows effectively giving you
a header column. In the advance table model the row span doesn't extend
past the row group so your header column won't run down the side of the
table body row groups. If the header and footer are in a different row
group than the body rows then you'll get unexpected behavior for
setRowSpan.
...so you cheated ;) I struggled with this problem, what happens to
TableListeners under this model?
> To maintain existing behavior for backwards compatibility you need to add
> all sorts of logic to deal with corner cases all over the place. This
> will hurt performance and takes away from the ease of use Grid and
> FlexTable currently have. In the end I think it's a net loss.
Could you elaborate on this? I don't understand what corner cases you
are talking about that any given table in html doesn't already suffer
from.
space...@gmail.com wrote:
> On Jan 1, 10:33 pm, "Sandy McArthur" <sandy...@gmail.com> wrote:
> > Each row group and table row is a UIObject instance and in GWT a
> > UIObject can be the source of events but only Widgets can receive
> > events. So when an event fires for my table widget it's onBrowserEvent
> > method redispatches the event to the table row group object's
> > onBorowserEvent method which then send the event to the table row
> > object's onBorowserEvent method. The programmer can add event listeners
> > on each of those object types which can deal with events.
>
> ...so you cheated ;) I struggled with this problem, what happens to
> TableListeners under this model?
I don't use TableListener, it doesn't make sense for a couple of
reasons. First, the interface only specifies a row and column, but I
would need a row group parameter too. Since HTMLTable basicly uses one
tbody row group then for TableListener that parameter is implicitly
always zero. Second, HTMLTable is one Object that controls all the
elements from table to td. The ObjectListTable makes use of separate
Objects that manage one element each and you attach a mouse listener
where you want to receive events. If you only want mouse over/out
events fired for every third row you could express that and while I
haven't benchmarked it yet, I think it would perform better than firing
events for every row and ignoring two thirds of them.
> > To maintain existing behavior for backwards compatibility you need to add
> > all sorts of logic to deal with corner cases all over the place. This
> > will hurt performance and takes away from the ease of use Grid and
> > FlexTable currently have. In the end I think it's a net loss.
>
> Could you elaborate on this? I don't understand what corner cases you
> are talking about that any given table in html doesn't already suffer
> from.
You said yourself "The GWT DOM currently accesses rows in a table using
the rows array of a table or the body element passed in from a function
call. This makes the problem of adding headers and footers non-trivial
due to the fact that the tfoot row is always [thead(rows) + 1] or
[tbody index - 1] regardless of its display order." but this isn't true
with Opera, the tfoot has to be at the end else it won't render
correctly. A lot of the corner cases come from optimizing update
performance. For example appendChild is faster than insertChild. To add
a row to the end of the table in everything but Opera you can use
appendChild. In Opera you would want to use appendChild when there
isn't a footer for faster updates. Maybe you won't find across all the
quirks I did trying to get things to work in each browser the same way.
I have no problem removing the prepareCell overrride problem, it was
something that bothered me. I should be able to make sample
modifications to my implementation within the next day or two to try
this out and see if it makes sense.
The other problem that I see (that is actually pretty huge) is a clean
way to handle table events and determine which section is described in
the event.
Not to mention the Opera limitation that Sandy mentioned earlier...
I am also interested in the Handler pattern, I was actually thinking
earlier that my proposed solution deals with Tables as UI elements and
a logical grouping of Elements, but Sandy's is a little more like a
Form element, where the Table itself is just a logical distinction not
a 'physical' element - tables contain actual visible, eventable things
not the table itself. It seems like Sandy has a better handle on this
than I do, so perhaps he can speak to this.
On Jan 2, 2:51 am, "Sandy McArthur" <sandy...@gmail.com> wrote:
> For example appendChild is faster than insertChild. To add
> a row to the end of the table in everything but Opera you can use
> appendChild.
I dont really understand how append child is any faster regardless of
location in the physical table unless its always being appended to the
end. Shouldn't we be attempting at least to not blatantly violate the
"TFOOT is always before TBODY" structural rule in the DOM where
possible? If one references the sections and appends to them instead
of a very large tbody, then the append cost is as bad as it is
currently in tbody, but negligible in tfoot and thead sections.
I dunno. I wrote my table so the table row group is created and
populated with rows and then cells before it's attached to the DOM.
(The goal being that I help the browser do fewest incremental
re-layouts while updating a table widget until my code is done.
Honestly I'm not sure I actually achieved anything there because I
haven't tested ordering DOM updates the other way around.) So I've only
every add/removed/moved table row group elements.
I would suspect that if you are adding tr elements to a single tbody
element then since it's at a different level of the DOM it shouldn't
cause a problem. eg: DOM.appendChild(aTable.theTbody, aTr)
But if you use a tfoot row group for the footer then you'll probably
can get away with it being the last element of the table element for
all browser despite what the HTML spec says.
(BTW: does anyone know of a place to submit bug reports for Opera? I
tried to find one when I first stumbled across this quirk but didn't
see a place for that.)
The Safari rendering glitch I don't understand as much. It seemed to
only happen when you inserted row groups near the top of the table.
Sometimes the inserted rows would be rendered on top of the existing
header rows as if the header rows didn't exist. I think the problem
didn't happen if you were only appending row groups but I'm 100% sure.
When I figured out that removing a child element of the table element
caused it to be rendered correctly I added the hack to add/remove an
empty caption element and didn't explore this quirk further.
> If there are still Opera or other quirks we must deal with, then the
> solution will probably be to bite the bullet and create a HTMLTableImpl in
> order to specialize methods per browser. Check out TextBoxImpl for an
> example.
That is what I did.
Made a table with a handful of rows in a tbody, as well as a header
and footer layed out according to spec. Ran some code to dig until the
tfoot section was found and checked the locations that the DOM
expected the footer to be located in and changed the HTML therein;
interestingly, I found that Opera is fine (at least 9.10) when using
the row index directly (even tho its the structural not the display
index), but Safari is off by 1 every time. The Firefoxes (2.x) and IE7
were also all fine.
Why did I bother? Well, I got to the rowIndex by asking the tfoot
section for rows[0].rowIndex, so the absolute value within the group
holds the correct location in the overall table everywhere but in
Safari apparently.
Why do I care? If one is designing headers and footers by monitoring
row offsets structurally, it creates a special case for Safari, but no
others. If one uses row groups with absolute row indexes within the
groups, the special case disappears. This is particularly convenient
in the current HTMLTable model because the table itself is treated as
one giant tbody row group and only indexed by table in two places.
On 1/2/07, Sandy McArthur <sand...@gmail.com> wrote:
>
really only getCellElement appears to be table based, and
getDOMCellCount is just being used that way.
On 1/2/07, Emily Crutcher <e...@google.com> wrote:
On Jan 2, 3:16 pm, "David Evans" <spacele...@gmail.com> wrote:
> getCellElement and getDOMCellCount
>
> really only getCellElement appears to be table based, and
> getDOMCellCount is just being used that way.
>
> On 1/2/07, Emily Crutcher <e...@google.com> wrote:
>
> > Which two places use table instead of tbody for indexing rows? As that is
> > probably a bug.
>
> > On 1/2/07, David Evans <spacele...@gmail.com> wrote:
>
> > > I just ran a quick test in Opera (win), Safari, Firefox (mac & win) and IE
> > 7
>
> > > Made a table with a handful of rows in a tbody, as well as a header
> > > and footer layed out according to spec. Ran some code to dig until the
> > > tfoot section was found and checked the locations that the DOM
> > > expected the footer to be located in and changed the HTML therein;
> > > interestingly, I found that Opera is fine (at least 9.10) when using
> > > the row index directly (even tho its the structural not the display
> > > index), but Safari is off by 1 every time. The Firefoxes (2.x) and IE7
> > > were also all fine.
>
> > > Why did I bother? Well, I got to the rowIndex by asking the tfoot
> > > section for rows[0].rowIndex, so the absolute value within the group
> > > holds the correct location in the overall table everywhere but in
> > > Safari apparently.
>
> > > Why do I care? If one is designing headers and footers by monitoring
> > > row offsets structurally, it creates a special case for Safari, but no
> > > others. If one uses row groups with absolute row indexes within the
> > > groups, the special case disappears. This is particularly convenient
> > > in the current HTMLTable model because the table itself is treated as
> > > one giant tbody row group and only indexed by table in two places.
>
I couldn't reproduce the opera problem I came across before. Maybe 9.10
fixed or maybe I was smoking crack and didn't know it. I'll recheck it
out later.
> I'll keep an eye out for the Safari
> behavior, thanks!
I compiled a version that triggers the overlap bug in Safari. Load
http://sandy.mcarthur.org/gwt-stuff/bugs/safari/overlap/TestTable.html
in Safari and click "Add Person" a few times until the bug happens and
you'll see something like:
http://sandy.mcarthur.org/gwt-stuff/bugs/safari/overlap.png .
Thank you,
Scott Stirling
Framingham, MA
Have you been able to reproduce it with straight javascript?
this brings me to my next point (while on the Table topic); is
insertRow well supported? It seems like the 'digging' approach
currently used could get very slow for large tables. If the browser has
this implemented, shouldn't we be inserting rows using the mechanics of
the browser?
On Jan 2, 4:05 pm, "Sandy McArthur" <sandy...@gmail.com> wrote:
> Icompiled a version that triggers the overlap bug in Safari. Loadhttp://sandy.mcarthur.org/gwt-stuff/bugs/safari/overlap/TestTable.html
currently when a row is inserted into a table (by FlexTable in
particular) the following code walks to the parent element (in this
case the tbody element of a flextable)
var count = 0, child = parent.firstChild, before = null;
while (child) {
if (child.nodeType == 1) {
if (count == index) {
before = child;
break;
}
++count;
}
child = child.nextSibling;
}
parent.insertBefore(toAdd, before);
So let's say that the tbody element is to have a new row inserted at
position N, this code essentially runs the tbody row for row until it
gets to the Nth row (or the end if the index is N+1) and then runs
insertBefore. I am wondering if perhaps insertRow(rowIndex) from the
DOM makes more sense specifically for row insertion into tables...
I know Sandy has way more knowledge of javascript table performance
than I do, but it seems like the browser's implementation of insertRow
may (or honestly, may not) be better at this.
Maybe it would be faster if I made more use of JSNI but I've currently
limited myself to GWT provided APIs. What I did to address the speed
for large tables is, I have a List of tbody elements and I use the
insertBefore method to manipulate the DOM. This way I don't have to
iterate to find the row I want to insert at. I just look up that row
number in the List and insert a new one.
I did have to write a insertBefore method in JNSI but I hope that
method gets added to GWT's DOM class:
http://code.google.com/p/google-web-toolkit/issues/detail?id=448
I'm really just finishing up the second step in 1) make it work, 2)
make it work right, 3) make it fast. Maybe later I'll look into faster
methods when I have a need to speed things up.
Emily Crutcher wrote:
> That makes sense, it's another instantiation of the
> w3-counts-whitespace-nodes problem, so we have to manually skip over them on
> standards-based browsers.
If GWT code was what built up the table element's DOM, as opposed to
being from HMTL source, how would whitespace nodes get added to the
mix? And if whitespace nodes couldn't be added then wouldn't it be safe
to skip the nodeType check?
On 1/1/07, Emily Crutcher <e...@google.com> wrote:Thanks Sandy! Do you know if legal ever got your contributors agreement? I know you had talked about it with Scott, but I don't know what the resolution was.
Yes, you can see in the CONTRIBUTORS file.
I recently spent some time writing an MVC-style table that supports
sorting, paging and filtering, with similar features to what you get
from the extremecomponents.org table widget. It was a lot of work to
get it right.
One thing that I think is important is making sure the design can
support client-side table "decoration" in addition to displaying raw
server data. For example, I needed to allow extra widgets to be added
to a row (like checkboxes to select the row for group operations, or a
link/button to allow the row to be edited or deleted), that are not
strictly part of the server-side data model but are required for
interaction. In my design, there is a decorator layer where you can
augment the server data to add these additional items to the model that
is ultimately used to populate the FlexTable. I wanted to support
inline cell editing as well, but I don't yet know the best way to
design it.