Widgets 3.0, an idea...
Jorge Vargas
feedback, overall it was good so I'm posting it to tg-trunk list to
get more feedback. Maybe I can turn this into something useful in this
weekend's sprint. A little note, I'm now like 50/50 on sharing the
config file, maybe having widgets.py and widgets.js be valid python
and javascript respectively is a better approach.
ok lets see where to start, I had an idea and I went ahead and coded
it. And I'm writing here to see if it's worth working more on it.
A couple of things to get you on the mood.
1- Data is code, code is data
2- I was watching this http://video.yahoo.com/watch/630959/2974197,
not sure how that's related.
3- ever since embraced WSGI, every problem starts looking like a dict
The basic idea is to have on file to rule them all. So you will have a
widgets.py and a widgets.js didn't you said one file? yes it's the
same file, it's just that python doesn't likes imports that don't end
with .py and this will scare me away <script type="text/javascript"
src="js1.py">
for example this: (you can ignore the last one that was just a test
for empty values)
my_widget = {
'js' : ['js1','js2'],
'css' : ['css1','css2'],
'jsource' : ['console.log("my_widget");'],
'body' : '<h1>Hello</h1>',
}
my_widget2 = {
'js' : ['js1','js4'],
'css' : ['css4','css2'],
'body' : '<h1>Hello</h1>',
}
my_widget3 = {
}
The first thing to note is that the above file is valid python,valid
javascript, valid json and even valid yaml (some people say)
The second is that js and css are supposed to be paths, while jsource
and body "code".
Third, the first 3 are supposed to be added to the <head> while the
last one should also be a list, actually a dict, and should be
somewhat analogous to tmpl_context, in fact in a real implementation
this should be added as pylons.c.something and is what the template
author will use to display the widget in his layout.
At this moment it has two implementations, server and client side.
The server is the sane implementation, it basically parses widgets.py
and passes the results to string.Template which splits out
main-py.html which is browser rendable.
The client is more a "let me see if this is possible" than a real
proposal it starts with main-js.html which includes a single main.js,
which is basically an implementation of dynamic JS includes (that's
scary :p)
Now keep in mind I kind of hate JS and it hates me back, or in other
words I suck at JS and it sucks right back at me. Oh and this only
works on FF but it's pure JS. It has several bugs like adding the same
file twice and doing the wrong thing with body but as I said this is a
toy. Alberto pointed out dojo does something like this and we could
look into it.
So as you can see both api implementations are the less common
denominator, and they have around 100LOC each, minus comments, and
whitespace this is really just 3-4 functions. Ideally the python part
should be ported to a mako function as well as a genshi function and
for the JS we should have jquery,dojo,etc. and then have something
like wsgiref to make sure they are all api compatible.
What's the big win? let me start with the WHY?
First of all Toscawidget is great, I really enjoy it but I don't think
it's the hammer for all problems.
it excels at the forms part, tw.forms is total awesomeness, specially
with changes of requirements (by clients), not to mention how great
rum is.
but it fails at the "JS wrappers" part. it fails because of several things
1- having to wrap things, I really don't see the point of getting a
python value into a class, then into an instance, then to the
template, then to JS code, so the client can pick it from the page and
send it again into JS
2- it tries to hide you the JS, which is a mistake for some cases, no
offense but what's the point of tw.jquery it's even more code to do it
in python than just go to /templates and write it.
3- if you are going to use a wrapper, then lets do it in the same
language js_function WTF???, which is exactly what all JS libraries
are, wrappers!
4- a ton of files for a simple function, this morning I wrote (or
tried to write) a toscawidget for wymeditor (code on hg, release ann
on tw list) only the basic widget is 40 lines of code, yes most of
that are comments but the original is 5 lines, plus 2-3 for imports[1]
is much less and more readable, more importantly adding a
configuration is just one loc[2] while in TW that will be one for the
default, another for the updated value, a check in the init, and a doc
.... as for the files I know most of them are bolierplate to make
everything integrate nicely but to a newbie that's scary, and to a
maintainer is boring.
5- no one understands the one global instances and render time scopes,
it's a great feature and clever design. But people just don't get it
and try to set values to it on the controller and just call it as
${widget()} on the template. I have answered that question a lot on
IRC and the mailing list
6- the bungle is bad. The most recent example of this is the drop of
the JS wrappers from webhelpers, everytime a new version of the JS
library is release someone has to go over the python version and
add/remove the values mentioned in point 4, this is busy work and
totally annoying which results in what webhelpers did, also if you
take a look at [3] you will see most of the wrappers are either
outdated or have had little activity since the first time they where
wrote.
7- TW are hard to debug, there is just way too much magic going on, so
reading someone elses code is not nice, this contributes to 6.
and I'll stop now because I'm starting to talk about things that
aren't really solved by this tool.
Bottom line TW is great for forms and static traditional html stuff.
but it breaks with the "new wave" of single page apps and intensive
xhr usage, in which case some of us, at least I do just ditch it all
together and go back to the plain old 3 public/javascript, public/css and
/templates.
so back to the solution.
- we'll have one file or many that will define widgets. in a
namespaced way, (the current implementation uses both python and
javascript global space, for simplicity rather than design), I'm
inclined to use JS "namespaces" which in python will be a dict with
all the widgets with proper keys, again think WSGI, this will be lazy
loader and cached for performance. And will be the source of the
tmpl_context objects I talked about before, so in essence it will be
key='myproject.widgets.something' or 'jquery.something' value=python
dict, usable by the templating engine.
- we'll reuse or reimplement the JS/CSS injection from toscawidgets,
that's a great tool.
- all paths will be from the public directory unless they are
absolute, of course this will be configured variable for usage outside
tg2
- no packages, no setuptools, no .js inside egg, a widget will be
simply one dict,
- we should add a dependencies parameter so for example if widget 'b'
uses widget 'a' then it will be something like b = {'body' = '${a}',
'depends' = ['a']} and of course we'll need to code all the possible
edge cases, if done right this could be delegate to __import__, no
clue on the JS implementation.
- the preferred html generator will be webhelpers HTML package from
0.6, it's just too easy and very pythonic, much better than any
templating language IMO, and keep in mind the widget is something
build by programmers, there is a bottom comment on this.
- (this one is still in flux) the "body" of a widget could be a
string, a list, a generator, a node or a dict. the first 3 will be
interpreted as raw output and will be wrapped in webhelper's literal()
and send as one big string to the template engine. a node will be in
case of xml templates to be inserted in the tree, and if it's a dict
it will behave like TurboGears, but in a different namespace to
prevent overwriting the users variables. perhaps pylons.c.newwidgets
:)
- As I said above widgets.py/.js is valid python/js/json/yaml, ideally
this should stay that way if things get complicated then I think we
should keep it as json if it becomes a config file on it's own then
yaml. or have a format key in the dict.
- the system will only load the widgets the user wants, that is no
auto discovery or other magic, you should explicitly ask for all
widgets you want in a call to widgets_to_page (horrible name, must be
changed), which should be called in tg2 at the root of root.py and it
should be read only.
- Like widgets today, they can accept parameters but have no defaults.
if you want defaults you should overwrite/extend it.
- all references to external files should depend on their type. if
they are to be included in urls (js/css files) path notation, if they
are REAL python files should use dot notation.
- defining an external template file should be a special case, in
which case I vote for path notation starting from /templates (again
configurable for non-tg2 environments)
- if done right the same widget should be able to be rendered either
client or server side, or have both a client and serverside component,
which at the moment I don't see the usecase.
- I haven't though of overwriting/extending a widget. maybe you could
clone the dict and overwrite the proper values or just copy it over to
your project and modify it.
- neither of having more than one instance in a page and ids crashing.
I think that's it, attached is a compresses mercurial repo.Which isn't
available anywhere else for now.
Sorry for the long email, let me know what you think.
Regarding webhelpers.html, I'm not sure how well it does on
performance but the api is really nice, and it is a lot better than
"hand coded" html in templates stored somewhere, deep inside
site-packages.
it lack a "pretty print" option so it's generated code is rather ugly,
Mike Orr suggested to me a while back that using something like
beautiful soup on the output will be the best, which is fine for final
pages but I think a solution could be contributed, even if it's only
turned on for debugging. we should work on it.
[1] http://trac.wymeditor.org/trac/wiki/0.5/Integration
[2] http://trac.wymeditor.org/trac/wiki/0.5/Customization
[3] http://toscawidgets.org/hg
Michael Brickenstein
I read your post now, several times:
I know, that every answer, I can give now, to such a long, well
thought mail,
can't really give it the appreciation, it deserves.
Basically, I think that you have some valid points.
In particular, I think, TW is great, but not the way to write
a rich client application.
But if you need something for that, is it really a TurboGears
problem: TG can be used excellently as server side component of a rich
client app.
The widget problem for this kind application: isn't it solved better
by JS widget libraries like dojo's Dijit?
http://dojotoolkit.org/book/dojo-book-0-9/part-2-dijit-0
Michael
Alberto Valverde
TW, as a port of TG widgets, shares the scope of being a tool to create
*server-side* components (eggs) which bundle markup generating logic,
css, javascript and a thin interface between python and js to initialize
the client-side part of a widget. These widgets are more suitable for
pages which don't *require* javascript to be usable but that can be
progressively enhanced with its use (eg: the calendar widget from
tw.forms...)
It wasn't designed to build "one-page" rich client apps or handle module
dependencies on the client so it naturally falls short on this tasks. In
fact, I've considered once considered factoring out the resource
injection functionality from TW into a smaller lib to feed js "modules"
to the client, with the possibility of compressing them on production
(see the half-baked attempt at [1]), and in the future bundle them into
less files to optimize load time. But then I (finally) discovered dojo :)
What you're describing here, as Michael has pointed out, sounds very
much like what Dojo already does. Like how it handles loading modules
(dojo.require) on demand from the client side and the toolchain it
provides to "pack" everything into less files, including the html
templates used by dijit widgets (which can even be written in Django
template language and processed on the client!) , to optimize load time
on production. What has really impressed me is that using the optimized
packages, if planned well into the build-chain, doesn't require any
change in the existing code (eg: to conditionally include one file or
another, etc...)
To sum up, I think these are great ideas but following them will
probably lead to reinventing the dojo wheel. Incorporating them into TW
doesn't make much sense either IMHO since pure-javascript rich client
apps were never really TW's scope. I think Dojo or ExtJS or others are
much better suited for this task.
Alberto
[1] *http://toscawidgets.org/documentation/ToscaWidgets/deploy.html*
Daniel Fetchinson
>> I know, that every answer, I can give now, to such a long, well
>> thought mail,
>> can't really give it the appreciation, it deserves.
>>
>> Basically, I think that you have some valid points.
>> In particular, I think, TW is great, but not the way to write
>> a rich client application.
>>
>> But if you need something for that, is it really a TurboGears
>> problem: TG can be used excellently as server side component of a rich
>> client app.
>> The widget problem for this kind application: isn't it solved better
>> by JS widget libraries like dojo's Dijit?
>> http://dojotoolkit.org/book/dojo-book-0-9/part-2-dijit-0
>>
> My thoughts exactly.
Right. I for one don't use TW at all any more, every client side GUI
component is handled by ExtJS and I'm sending the GUI definition stuff
in JSON from the server, i.e. the whole GUI is "dynamical". The html
only has a couple of lines of <div>'s and that's it. I actually think
this is the way of the future for rich client side applications.
Cheers,
Daniel
--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
Toshio Kuratomi
I think Jorge's idea has one nice feature in that area. You can define
a widget in one file that is then realizable from both python and
javascript. With that ability, it's easier to create an html-driven
website from the server for people who don't have javascript enabled
browsers due to accessibility. The html presentation can then be
manipulated, destroyed, and recreated by a javascript layer that uses
the exact same widget definitions.
To prevent re-inventing the wheel, I'd love to see a way to parse the
dojo widget definitions from a turbogears server and generate those
initial static pages that way.
As you and Alberto have said, this seems outside the present goals of
TW, though.
-Toshio