RR: Data aware widgets
Emily Crutcher
One of the tougher aspects of programming a well functioning AJAX application is the asynchronous population of data into the application. The problem is that loading all data at initialization leads to poor performance due to long start-up times, while trying to juggle the population of the widgets over time can be complex. Detailed below is one possible solution.
The questions I want the most feedback on are
1) Is it worth doing?
Will we get enough bang for the buck by providing data widgets to justify the hassle of creating and maintaining another API?
2) Is there a better approach?
Below we propose a single solution to the problem. Are there better solutions out there? Are there server-side frameworks that should be kept in mind while designing our client-side solution?
---------------Design Goals----------------
Server agnostic
The widgets themselves should not care about the actual server technology or how it represents its data.
Seamless user experience
The UI should be responsive even while data is being requested from the server.
Easy to use
It should be relatively easy for a user to figure out how to use a data enabled widget.
Resuable Components
As the data management code will be somewhat complex ideally the same code could be reused by all the data enabled widgets.
---------------Design Overview---------------------
Each data-aware widget must generate unique data requests that are handed to a data broker for processing.
The data-aware widgets are responsible to creating the data request and processing the response objects associated with that data request.
The data broker is responsible for batching, fetching, creating a response object, and caching data requests.
To use a data-aware widget, you must supply it an object conforming to the data broker interface, detailed below. For convenience, GWT will supply an abstract data broker which will manage caching and batching of requests, leaving only the actual server communication for user implementation.
------------Proposed Widget List----------------
Data Tree:
A tree populated via data from a data broker.
Data Table:
A scrollable table populated via data from a data broker.
Data ListBox:
Creates a list of options via data from a data broker.
Suggest Popup:
A selected suggest popup where suggestions are populated from the server based on text strings.
Suggest Box:
Composite widget using the suggest popup and a single text box to create a Google style suggest box.
-------------- Proposed Data API ----------------
Data Broker:
Each Data widget is associated with a single Data Broker. The Data broker is responsible for gathering and processing data requests and using the returned information to populate data widgets. It knows how to take a string payload from the server and translate it into a response object.
The Data Broker is data-agnostic. It should only be able to communicate with the server, cache data responses from the server, and re-assemble server side response. It may use whatever technology is most convenient for the user, RPC, JSON, HTTPRequests, etc. Only the server response and the request object should be aware of the underlying data.
It is the also the responsibility of the Data Broker to batch requests from the client, and cache the responses, in order to reduce the number of round trips required.
/**
* DataBroker interface used by data- aware widgets.
*/
public interface DataBroker {
/**
* Request object to be sent to the data broker.
*/
interface Request {
/**
* Fills the current widget with the data from a response. Each request may
* have multiple response objects associated with it.
*/
void fill(Response response);
/**
* Request data.
*/
String getRequestData();
}
/**
* Response object to be given to the <code>fill </code> method of the
* request object.
*/
interface Response {
/**
* Gets the index of this response. For example, if this is the third
* response to a query, then the index would be 2.
*/
int getIndex();
List getValues();
}
/**
* Registers a <code>Request</code> object with this data broker.
*/
void get(Request q);
}
DataBroker.Request:
The request object is sent to the data broker by the data widget. The request should have a widget specific fill method. Bellow is a sample request that might be used to fill a data table row.
/**
* Data aware table
*/
public class DataTable extends FlexTable {
class RowRequest implements Request {
int row ;
int col ;
RowRequest( int row) {
this.row = row;
}
public void fill(Response results) {
List values = results.getValues ();
int end = col + values.size();
for (int i = col; i < end; i++) {
String value = (String) values.get(i);
setHTML( row, i, value);
}
col = end;
}
public String getRequestData() {
return DataTable. this.getName() + ".Row." +Integer.toString(row );
}
}
…
Data Response:
The response object is created by the data broker to be passed into the fill method of the request object. Each request may have multiple response objects returned by the server. It is the responsibility of the data broker to ensure that the fill method is called sequentially on all returned responses. Each response object contains a list of values and an index. So, for example, if a request generates four response objects, then the response objects will have indexes 1 through 4.
Typical Control flow for a single data request:
Data-aware widget:
Creates Request object.
Passes Request object to DataBroker.
DataBroker:
Collates Request objects together.
Sends Requests to server.
Gets results from server.
Creates Result objects from server response.
Pass Results objects to Request object in index order.
Request
Uses Results object to fill data in data-aware widget
P.S.
Thanks to Henry and Dan for all their help on this!
--
"There are only 10 types of people in the world: Those who understand binary, and those who don't"
Miroslav Pokorny
Wouldnt it be better not to extend but rather use composition. There is no reason to couple a DataTable with a FlexTable. Composition will work just as well and allows flexibility in allowing one to substitute a different table impl in the future. Eg by using inheritence one is forced to use a FlexTable when other table widgets may come along eg a sortable table, a table where one can reorder columns, a scrollable table etc. Im pretty sure there will be more table variations in the future.
mP
Scott Stirling
that. I am sometimes miffed at the parsimonious nature of GWT's
out-of-the-box widgets. Sorting and data binding could be core
features as far as I am concerned. The Java->Javascript compiler
could, in theory, elide or optimize unused code/features to no-ops at
compile-time.
Scott Stirling
Framingham, MA
Miroslav Pokorny
Emily Crutcher
The exact details of how data table would be structured would be worked out after the overall data-aware widget design is settled, but the top two design goals would be
a) Ease of use
b) Ease of extension
Thanks for the feedback! As including that sample in the way I did was definitely confusing. I'd also love feedback on the big question of "is it worth doing?" As that is the one I am most concerned with right now.
Scott Stirling
cross-browser Javascript subtleties and idiosyncracies into the core
widgets. Miroslav, I think the logic of your argument would support
breaking out cross-browser functionality into browser-specific
widgets, which is a step backward. The small blocks I want to
assemble in GWT are things like Button, Flextable/Grid, Label, etc.
Put another way, there should be a core set of widgets in GWT that a
developer can use to build an average e-commerce app, with an eye
toward the basic fact that GWT is a way for me to code Javascript and
DOM in Java. There's no pretense of preserving interfaces post-compile
time.
Let me reiterate my point: I don't want a proliferation of GWT widgets
for every little feature. I want the core widgets to be powerful out
of the box. Sorting and data binding seem like reasonable core
features to me. I'm not looking to build a business coding GWT
widgets. I want to be able to build a decent web app with GWT alone.
Scott Stirling
Framingham, MA
Konstantin.Scheglov
> Let me reiterate my point: I don't want a proliferation of GWT widgets
> for every little feature. I want the core widgets to be powerful out
> of the box. Sorting and data binding seem like reasonable core
> features to me. I'm not looking to build a business coding GWT
> widgets. I want to be able to build a decent web app with GWT alone.
Do you remember Delphi/C++Builder? There was set of just GUI
component, for manual operations. And there was set of subclasses that
add data-aware features. So, don't modify core widgets. If you need
data binding, create additional library.
Scott Stirling
composition matter? I think it's a "rathole" I want to be able to pass
by, truly.
On the other hand, there are GWT internal implementation design
choices to be made. You may have a point there. I don't actually have
an opinion I want to argue for at the moment to that point.
I am thinking about Emily's question of whether data binding is worth
it. I am thinking my answer is, why not try it? Create a branch if
necessary, or commit it to head as an optional feature.
Scott S.
Framingham, MA
Dan Morrill
I think a big issue here is one-size-fits-all-ness. I also think this is what Emily is aiming at, when she asks whether anyone thinks this would even be useful.
Let me just point out the elephant in the room: there are a million and one ways to manipulate your data on your server. You might be using an O-R tool like Hibernate, or you might be using a stream or feed oriented mechanism like Google's GData API, or you might be using straight-up SQL-style transactions, and so on. Each of these million and one server techniques will conceivably imply or demand concessions from the client side widget implementation, in this case GWT.
So the question is: is it even possible to build an "intelligent widget" system for GWT that will meet a suitably large portion of users' needs? Remember that we are server-agnostic, so we don't have the option of specifying a particular server technology and binding ourselves to it.
To put some hypothetical numbers to my question, suppose it takes 200 person-hours to implement such a widget system. What percentage of GWT users will even care at all? Will this feature be used by 50% of GWT users, or only 5%? Beyond that, if only 33% of GWT users are even interested in the functionality, it could very well be that the solution proposed here is only practically useful to a third of *those* users, meaning that we could very easily spend 200 person-hours implementing a solution that is only useful to 10% of GWT users. The question of course isn't whether those 10% of users are "worth" 200 person-hours, but rather whether we could have spent those 200 person-hours on a feature which benefits a larger percentage of users.
So, those are the questions on my mind. To summarize them:
- Do we even have a hope of successfully building a generally-useful, server-agnostic intelligent widget mechanism?
- If we do, is Emily's proposal headed in the right direction? Why/why-not?
- Dan
Miroslav Pokorny
Probably off the point but why is there no api to set headers eventually creating a TH with text or widget ( setHeader( column, widget/text) for any of the Table based widgets eg Grid, HTMLTable, FlexTable instead of doing it thru DOM methods or JNSI?
Joel Webber
In GWT, we've tried to maintain a similar dichotomy. The core widgets should give you what you need to get stuff on the screen and handle basic events. There's no notion of MVC patterns, no attempt to enforce any particular design patterns, in fact. The intention was for them to give you the minimum set of functionality, on which you can build more complex frameworks to your heart's content.
In this particular case, I think it's very appropriate to build data binding as a framework that builds on top of the basic widget set, and could be easily used with additional widgets from other libraries. I seriously doubt anyone will ever find "one data binding framework to rule them all", and even if they did, there would still be people who don't need one at all.
Overall, I like this proposal -- but it would be really nice to hear if anyone has any suggestions that would make it better? Think it sucks? Even better, if you have real-world use-cases you've run into it would be great to hear how well this proposal would or would not apply to them.
Thanks,
joel.
Scott Stirling
> Probably off the point but why is there no api to set headers eventually
> creating a TH with text or widget ( setHeader( column, widget/text) for any
> of the Table based widgets eg Grid, HTMLTable, FlexTable instead of doing it
> thru DOM methods or JNSI?
I wondered the same thing myself! - Scott
Emily Crutcher
Miroslav Pokorny
Real tables rather than tables for layout, with columns always have headers which is probably why the distinction was made to include data cells (TD's) and table headers (TH's).
mP
space...@gmail.com
I've actually been working on doing just this for the last week. The
problem isn't as trivial as one might expect, particularly due to the
assumptions under which the current HTMLTable were written ie. always
TBODY.
I will gladly write up my current issues and concerns and put them up
for discussion. (As well as what I have so far in implementation).
On Dec 30, 12:01 am, "Miroslav Pokorny" <miroslav.poko...@gmail.com>
wrote:
> I think it should.
>
> Real tables rather than tables for layout, with columns always have headers
> which is probably why the distinction was made to include data cells (TD's)
> and table headers (TH's).
>
> On 12/30/06, Emily Crutcher <e...@google.com> wrote:
>
>
>
>
>
> > Are adding headers something that would be helpful to our users? It seems
> > like a reasonable extension to me, can anyone think of a reason it
> > should/should not be added to HTMLTable?
>
> > On 12/29/06, Scott Stirling <scottstirl...@gmail.com> wrote:
ijuma
> I'd also love feedback on the big question of "is it
> worth doing?" As that is the one I am most concerned with right now.
In my opinion, yes. I think the SWT/JFace analogy from Joel is a good
one. It's a good design to follow, but it's important to note that
JFace is provided by eclipse.org and makes the API a lot easier to use
in most cases. So, out of the box you have the choice between the
low-level SWT or higher level JFace (in most cases I would choose the
latter).
In addition, a lot has been talked about data binding in the Java GUI
world in the recent past. JFace included an experimental data binding
API in eclipse 3.2 and they've been working on the final version for
Eclipse 3.3. For Swing, there's the Beans Binding JSR[1]. So, it seems
to me that people have found the third-party binding libraries like
JGoodies useful enough that the toolkits decided to have it out of the
box. I've used the JGoodies bindings library in the past and it was
very useful for maintaining the data in sync between widgets and data
source.
Emily Crutcher
Sandy McArthur
table model (row groupings: thead, tfoot, tbody) and while it's not
fully tested yet and still has a little more evolving to do, two
implementation details I've encountered are:
Opera: while the order of the thead and the tfoot elements should not
matter with respect to tbody elements they do for Opera if they are
manipulated via JavaScript. (Order doesn't matter in raw HTML source
for opera.) For Opera to render thead and tfoot correctly after
updating the DOM with JavaScript they must be the first and last childs
of the table element.
Safari: With some DOM inserts to the middle of the table Safari doesn't
re-layout the table correctly and visually tbody elements will appear
on top of thead elements. The quick hack I found around this is to
quickly add and them remove an empty caption element to the table
element after your updates and Safari should re-render the table
correctly.
I'm not sure if those bugs will affect GWT's HTMLTable implementations
because they work with the expectations of the basic HTML 4 table model
plus one tbody element. But if they do show up, hopefully that will
save you a number of hours of debugging. The source may have bug work
arounds I cannot recall off the top of my head too.
HTML 4 Tables:
http://www.w3.org/TR/html4/struct/tables.html
GWT-Stuff:
http://code.google.com/p/gwt-stuff/
A table demo is linked on the right side.
Emily Crutcher
John Tamplin
Thanks Sandy! Do you know if legal ever got your contributors agreement? I know you had talked about it with Scott, but I don't know what the resolution was.
--
John A. Tamplin
Software Engineer, Google
space...@gmail.com
I have also almost completed an implementation of the full 4.01 table
spec. Can I ask how you handled events?
I haven't traced through removal of individual cells as of yet, but I
have about a mid-maturity cut that makes the table structural sections
explicit (and gets around the Safari and Opera limitations by ignoring
the row indexes unless in a specific section) but tends to break bits
of backward compatibility.
The following document was to be my post discussing my implementation:
http://docs.google.com/View?docid=dhtrqvrz_8fhxrnw
Also, the patches are sort of mid-lifespan (and may have some widget
event holes as well as various small formatting bugs), but usable in
general. I can gladly send them to anyone willing to try them out or
look at them.
Thanks!
David
On Jan 1, 10:53 am, "John Tamplin" <j...@google.com> wrote:
> On 1/1/07, Emily Crutcher <e...@google.com> wrote:
>
>
>
> > Thanks Sandy! Do you know if legal ever got your contributors agreement?
> > I know you had talked about it with Scott, but I don't know what the
> > resolution was.Yes, you can see in the
> CONTRIBUTORS<http://google-web-toolkit.googlecode.com/svn/trunk/CONTRIBUTORS>file.
Sandy McArthur
* "Advanced HTML Table model" I mean the use of table row groups
(thead, tfoot, tbody) introduced in HTML 4.
* "Simple HTML Table model" I mean the use of tables where the table
row (tr) elements are children of the table element. Modern browsers
promote this to advanced model by introducing one tbody to hold all the
tr elements.
These terms aren't official but it's how they were described when I
first read about them and they stuck in my brain.
I chose not to start with the GWT provided table widgets. HTMLTable and
it's implementations are good at being intuitive for the programmer
thinking about a grid but their design doesn't work for the advanced
HTML Table model and efficient DOM manipulation.
On Jan 1, 7:12 pm, "spaceLe...@gmail.com" <spaceLe...@gmail.com> wrote:
> Can I ask how you handled events?
Each row group and table row is a UIObject instance and in GWT a
UIObject can be the source of events but only Widgets can receive
events. So when an event fires for my table widget it's onBrowserEvent
method redispatches the event to the table row group object's
onBorowserEvent method which then send the event to the table row
object's onBorowserEvent method. The programmer can add event listeners
on each of those object types which can deal with events.
In my table, table cells (td and th) extend SimplePanel so the normal
GWT event system just works there like any other Widget.
> I haven't traced through removal of individual cells as of yet, but I
> have about a mid-maturity cut that makes the table structural sections
> explicit (and gets around the Safari and Opera limitations by ignoring
> the row indexes unless in a specific section) but tends to break bits
> of backward compatibility.
IMO, the HTMLTable in GWT just wasn't designed to have header/footer
rows and it isn't worth trying to cram that feature into it. To
maintain existing behavior for backwards compatibility you need to add
all sorts of logic to deal with corner cases all over the place. This
will hurt performance and takes away from the ease of use Grid and
FlexTable currently have. In the end I think it's a net loss.
If you're going to do header/footer in HTMLTable widgets then you
probably should just fake using special logic it instead of using table
row groups. For example things like row spans work differently once you
introduce row groups. In the simple table model a header cell with a
row span of 99999 would extend down the whole length of the table but
not cause the table to actually have 99999 rows effectively giving you
a header column. In the advance table model the row span doesn't extend
past the row group so your header column won't run down the side of the
table body row groups. If the header and footer are in a different row
group than the body rows then you'll get unexpected behavior for
setRowSpan.
space...@gmail.com
On Jan 1, 10:33 pm, "Sandy McArthur" <sandy...@gmail.com> wrote:
> Each row group and table row is a UIObject instance and in GWT a
> UIObject can be the source of events but only Widgets can receive
> events. So when an event fires for my table widget it's onBrowserEvent
> method redispatches the event to the table row group object's
> onBorowserEvent method which then send the event to the table row
> object's onBorowserEvent method. The programmer can add event listeners
> on each of those object types which can deal with events.
...so you cheated ;) I struggled with this problem, what happens to
TableListeners under this model?
> To maintain existing behavior for backwards compatibility you need to add
> all sorts of logic to deal with corner cases all over the place. This
> will hurt performance and takes away from the ease of use Grid and
> FlexTable currently have. In the end I think it's a net loss.
Could you elaborate on this? I don't understand what corner cases you
are talking about that any given table in html doesn't already suffer
from.
Sandy McArthur
general purpose as GWT's HTMLTables. ObjectListTable is very much about
rendering rows or groups of rows from data stored in a List. When that
list is manipulated then the table does what it needs to do to reflect
those changes. I think that type of logic is rewritten too many times
when it's external to the table implementation.
space...@gmail.com wrote:
> On Jan 1, 10:33 pm, "Sandy McArthur" <sandy...@gmail.com> wrote:
> > Each row group and table row is a UIObject instance and in GWT a
> > UIObject can be the source of events but only Widgets can receive
> > events. So when an event fires for my table widget it's onBrowserEvent
> > method redispatches the event to the table row group object's
> > onBorowserEvent method which then send the event to the table row
> > object's onBorowserEvent method. The programmer can add event listeners
> > on each of those object types which can deal with events.
>
> ...so you cheated ;) I struggled with this problem, what happens to
> TableListeners under this model?
I don't use TableListener, it doesn't make sense for a couple of
reasons. First, the interface only specifies a row and column, but I
would need a row group parameter too. Since HTMLTable basicly uses one
tbody row group then for TableListener that parameter is implicitly
always zero. Second, HTMLTable is one Object that controls all the
elements from table to td. The ObjectListTable makes use of separate
Objects that manage one element each and you attach a mouse listener
where you want to receive events. If you only want mouse over/out
events fired for every third row you could express that and while I
haven't benchmarked it yet, I think it would perform better than firing
events for every row and ignoring two thirds of them.
> > To maintain existing behavior for backwards compatibility you need to add
> > all sorts of logic to deal with corner cases all over the place. This
> > will hurt performance and takes away from the ease of use Grid and
> > FlexTable currently have. In the end I think it's a net loss.
>
> Could you elaborate on this? I don't understand what corner cases you
> are talking about that any given table in html doesn't already suffer
> from.
You said yourself "The GWT DOM currently accesses rows in a table using
the rows array of a table or the body element passed in from a function
call. This makes the problem of adding headers and footers non-trivial
due to the fact that the tfoot row is always [thead(rows) + 1] or
[tbody index - 1] regardless of its display order." but this isn't true
with Opera, the tfoot has to be at the end else it won't render
correctly. A lot of the corner cases come from optimizing update
performance. For example appendChild is faster than insertChild. To add
a row to the end of the table in everything but Opera you can use
appendChild. In Opera you would want to use appendChild when there
isn't a footer for faster updates. Maybe you won't find across all the
quirks I did trying to get things to work in each browser the same way.
Emily Crutcher
I'd be really curious to find out all the problems either of you see/have had with retrofitting HTMLTable with simple headers and footers. Keeping offsets dependent on whether the table has headers and footers for row indexes is certainly one of them. What are the others?
Also, for the HTMLTable scope, can anyone think of a reason where having a simple one row header and footer would not be sufficient? One mantra we use for HTMLTable is to keep it as simple as possible, and setHeader(col)/setFooter(col) seems, right now, the simplest API.
David Evans
Since HTMLTable was written assuming body as the target for all operations, it was simply a matter of modifying the code to pass a row group section to define what row/col you are referring to. This semi-breaks and semi-maintains backward compatibility, primarily due to the fact that prepareCell and prepareRow have to be modified in subclasses, but all other operations can be re-implemented in HTMLTable as functional passthroughs to the table's tbody - this way anyone not interested in headers and footers doesn't really need to think about them at all.
Emily Crutcher
David Evans
I have no problem removing the prepareCell overrride problem, it was
something that bothered me. I should be able to make sample
modifications to my implementation within the next day or two to try
this out and see if it makes sense.
The other problem that I see (that is actually pretty huge) is a clean
way to handle table events and determine which section is described in
the event.
Not to mention the Opera limitation that Sandy mentioned earlier...
Emily Crutcher
David Evans
getting in the way of fun!) I have been struggling with this decision
for events for a while, which is why i asked about events from Sandy.
I made changes to SourcesTableEvents such that an implementation is
expected to maintain the section from which the event was generated
(which actually allows subclasses to have clearing capability or event
routing).
I am also interested in the Handler pattern, I was actually thinking
earlier that my proposed solution deals with Tables as UI elements and
a logical grouping of Elements, but Sandy's is a little more like a
Form element, where the Table itself is just a logical distinction not
a 'physical' element - tables contain actual visible, eventable things
not the table itself. It seems like Sandy has a better handle on this
than I do, so perhaps he can speak to this.
On Jan 2, 2:51 am, "Sandy McArthur" <sandy...@gmail.com> wrote:
> For example appendChild is faster than insertChild. To add
> a row to the end of the table in everything but Opera you can use
> appendChild.
I dont really understand how append child is any faster regardless of
location in the physical table unless its always being appended to the
end. Shouldn't we be attempting at least to not blatantly violate the
"TFOOT is always before TBODY" structural rule in the DOM where
possible? If one references the sections and appends to them instead
of a very large tbody, then the append cost is as bad as it is
currently in tbody, but negligible in tfoot and thead sections.