important about ajax forms

[mdipierro]

There has been a lot of discussion in the past about forms that submit
via ajax and may or may not refresh the entire page. It is also useful
to be able to break html pages into "modules" or "plugins" or
"components" each with its own model, view, controller in such a way
that they communicate both serversize (by sharing session and
database) and clientsize (one boxed component should be able for
example to refresh the entire page or trigger a flash).

I have prototype application that does this.

http://www.web2py.com/examples/static/web2py.app.events.tar

It uses jquery publisher subscriber mechanism. All the code is in a
new web2py_ajax and a class call jDiv (similar to Rails Partial but
more powerful in my opinion) which I could include in html.py

It allows you to write code like this:

def index():
return dict(partial1=jDiv('click me for text','mycallback1'),
partial2=jDiv('click me for flash','mycallback2'),
partial3=jDiv('click me to redirect','mycallback3'),
partial4=jDiv('click me for form','mycallback4'))

def mycallback1():
return 'hello world'

def mycallback2():
return jDiv.flash('this is a test') # flash on the container page

def mycallback3():
return jDiv.redirect('http://www.yahoo.com') # redirects entire
page

def mycallback4():
form=FORM('your name:',
INPUT(_name='name',requires=IS_NOT_EMPTY()),
INPUT(_type='submit'))
if form.accepts(request.vars):
return jDiv.flash('hello '+form.vars.name)
return form

Can you figure out what it does?
Not that the page is never reloaded. Only parts (partials, jDivs) of
the reloaded. Each jDiv lives in its own container, has one action,
can have a view, and can talk to each other.

This may require some more thought.

Comments?

Massimo

[Yarko Tymciurak]

I think this is not structured _enough_.

Looking at what has already been done in this area (see jsr portlets;  I do not remember how dotnetnuke does the same with it's "page components" - but that is definitely worth analyzing).

What we need is some general way that a

def index():  return something()

May be assigned to a portlet / window (say on a home page - with a "read more" action), and

when read-more is selected, that same index() function may get it's own entire page.

The point is this "partials" should not be knowledge within index(), rather index should be encapsulated in something that handles this.

Keep thinking about this.....

- Yarko

[Yarko Tymciurak]

another way to state this:  partials are language from one very (too) narrow use case.

The general case is that page CONTAINERS manage output, and decay to one container per page (or the partials sort of idea that were mentioned).

Now - controller logic is just engineering logic - implementation of solution / output;

Containers, and the protocol to connect output (URI) to a container is view logic...

This is important, because this is how outupt to one page (for example for PyCon) can be collected from controller outupt from 2 frameworks (django or web2py)....

And this is where this concept shows the boundaries that are appropriate (the other end of the boundary - even if you never want to combine output from elsewhere, the point is still  that level of separation is desireable).

Come to think of it, another place to look is what do Yahoo Pipes do?   That is 3 places to look, and gather ideas:  Yahoo Pipes, jsr Portlets,  .Net / what dotnetnuke uses to implement application containers...

This last piece is where web2py can start:  think about how to have "portlets" that can (for example) be connected to web2py application output.

I think this is more appropriate to think of as a general Python solution to put in place....

- Yarko

[Yarko Tymciurak]

On Fri, Apr 3, 2009 at 4:35 PM, Yarko Tymciurak <yar...@gmail.com> wrote:

another way to state this:  partials are language from one very (too) narrow use case.

The general case is that page CONTAINERS manage output, and decay to one container per page (or the partials sort of idea that were mentioned).

Now - controller logic is just engineering logic - implementation of solution / output;

Containers, and the protocol to connect output (URI) to a container is view logic...

This is important, because this is how outupt to one page (for example for PyCon) can be collected from controller outupt from 2 frameworks (django or web2py)....

And this is where this concept shows the boundaries that are appropriate (the other end of the boundary - even if you never want to combine output from elsewhere, the point is still  that level of separation is desireable).

Come to think of it, another place to look is what do Yahoo Pipes do?   That is 3 places to look, and gather ideas:  Yahoo Pipes, jsr Portlets,  .Net / what dotnetnuke uses to implement application containers...

This last piece is where web2py can start:  think about how to have "portlets" that can (for example) be connected to web2py application output.

Note what this last concept does / says:  If I can have a screen container as a destination, then I can supply tools which are applications (however small) which a user / site can just install and connect - for example, to have a calendar on the page.   What that calendar ties to (now) could be done thru the calendar's admin interface --- instead of programming. Placing the calendar, and connecting it to a receptor/container too can be an admin functionality. This is an important shift.

Let's think about this carefully, study what's out there and come up with a starting, simple (but appropriately functional) proposal.

-y

[Yarko Tymciurak]

And not new - dotnetnuke  already does this (has for years).  It would just be new (?) to Python.  I know there are some aspects of this in Plone, but those I believe are rather less flexible than what I'm thinking of - I'm still thinking of one page, multi frameworks putputting to (still thinking about future of pycon site).

[Yarko Tymciurak]

Also see my.msn.com for concept of site / page elements that you can move around, and their content follows. and menu items that user (or concievably admin) can use to configure thru the container.

[mdipierro]

Yarko most systems just use iframes. You can put anything you want
into an iframe.

The problem I am trying to solve is turning existing forms into ajax
forms and do it in such a way that when the form has errors, it just
modified itself without reloading the page. If the form submission
succeeds the ajax response triggers an event outside the "box". I do
not want to use iframes because 1) I want to communicate with the
outside; 2) I want to inherit the CSS of the "box container".

I think this is a critical ingredient although not sufficient to
implement what you are suggesting.

[Yarko Tymciurak]

On Fri, Apr 3, 2009 at 10:33 PM, mdipierro <mdip...@cs.depaul.edu> wrote:

Yarko most systems just use iframes. You can put anything you want
into an iframe.

I'm not sure that's so, but regardless I think that misses the point (iframe is a dtd type, and can be the target of links on a page and so forth;  my.msn.com does not use any iframes anywhere, nor did dotnetnuke - although an iframe type was one of the container types, as I recall).  The point is to have a surrounding structure to accomplish what you are trying to do in controller code.  I think doing what you're doing in controller is fine if someone wants a quick solution to something they have to do "right now", but I have issue that it's not sufficiently general nor appropriately decoupled for more general use). Further, if the right structure were in place to do this other ways, there would be no need to do it this way, and I think other ways are better (and there is already evidence of others doing.... as I pointed out).

Anyway - I think the way I would like to see this go is look at other things that are doing "this kind of" thing, and study and catalog what they are dealing with and how, and then see where the overlap (or missing pieces) are.  Then think of a pythonic way to accomplish something that makes sense.

Doing this in the controller the way you have just looks troublesome, and even worse - precludes I think any clean extension.

Regards,

Yarko

[ceej]

I'm really liking this idea Massimo, I'm going to be using it in a
project I'm about to start and grow on it :)

Keeps using ajax very DRY.

[mdipierro]

I re-posted a slightly better version.

The new version work in this way:

say you have an action like:

def myform():
form=SQLFORM(...)
if form.accepts(....):
# do something
redirect(URL(....));
return dict(form=form)

You can turn it into a partial by replacing "redirect" with
"jDiv.redirect or "jDiv.flash"

def myform():
form=SQLFORM(...)
if form.accepts(....):
# do something
jDiv.redirect("done!!!");
return dict(form=form)

and create a view that DOES NOT extend the layout and does not have
HTML and BODY tags. Something like this will do:

{{=form}}

Then create a parent action and in the view embed this partial

{{=jDiv("click me to ajax the partial",URL(r=request,f="myform"))}}

Mind that a partial can be served by another application (within the
same web2py installation), can be a proxy to a different web-site and
can contain an IFRAME (not recommended but possible).

@Yarko. I agree that this that this is not yet a complete solution but
more of a hack. Nevertheless it lets you take forms you have already
created and turn them into ajax forms. It does not require any
modification in web2py nor any third party libraries (it only requires
the new layout and new web2py_ajax.html).

I have been looking but I cannot really find a detailed decsription of
how those other systems work.

[Yarko Tymciurak]

Massimo - you're awesome!  This is much better....

The problem is there _is_ no description of how those other systems work;  I looked (again - after years) at dotnetnuke code, and it is (in essence) like webkit in vb.... so I looked at pyjs again tonight  (and will want to experiment w/ that more fully).    The best description for what I want is actually NO description of "how" --- but just to work with those systems (you can get an admin to a test of at least dnn),  and catalog our own _abstraction_ of what those systems are doing.

I imagine the jsr portlets might be the best documented (but I have not yet looked at it).

Perhaps a few hours w/ each, using them and intuit-ing, cataloging "what kinds of things" they are doing, then collating our notes from several.... then we will have a sort of requirements and can start design  / prototyping from there.

I would VERY MUCH like to have the "write a little web2py app, and assign it to the screen container"  kind of situation that my.msn  and dotnetnuke implement....   or at least a python-ic version of the best of that essence...

another thing that occurs to me is that with this, the app will want to know something from the container (am I minimized? am I shown?) that is - register more than one behavior (admin/ auth user/ minimized / small portlet / full)...  iPhone and Android give some UI hints to where I'm going with this ---

An app in the environment of such a container class - could modify behavior based on something about where it's being displayed... so mobile would be part.... going the other way,   the "screen full of iconic chicklets"  that android and iphone have (which then expand into apps) might also be a useful index page for some applications on the full computer / full screen browser.

Anyway, the right structure will open up some new possibilities I expect.

Kind regards,

Yarko

[Joe Barnhart]

I think Yarko makes excellent points. I too am looking for the
natural way to express components in Web2py and the code proposed by
Massimo originally did not look general enough. The better job we do
at abstracting this and generalizing it, the more capable the web2py
platform will be.

One of the most interesting approches I have seen is "Seaside" -- the
Smalltalk web app engine. Seaside relies on "continuations" in
Smalltalk to keep state while the user interacts with different
applets on a page. It is unclear to me if continuations are needed to
make this work, or just an easier way to accomplish it. For one
thing, it lets the developer keep objects in his state without needing
to render them as strings and pass them through request and response
vars. (I think.) Here is a description of Seaside. It did "ajax"
before AJAX was invented.

http://scg.unibe.ch/archive/papers/Duca04eSeaside.pdf

As a postscript -- I'm not sure Python supports continuations in its
native form. It is one of the features added in Stackless Python,
however. It would be interesting to see Stackless with its tasklets
put to use in a Pythonic one-upmanship of Seaside's concepts.

Joe Barnhart

P.S. Python and Smalltalk are very analogous languages. Other than
syntax, it's almost like they were identical twins separated at
birth...

On Apr 3, 8:51 pm, Yarko Tymciurak <yark...@gmail.com> wrote:

[Iceberg]

I want to test this idea, but it seems the events.tar does not contain
that magic new web2py_ajax.html ? Did I miss something?

[AchipA]

Just an itsy bitsy note. How does this stuff handle sessions ? Joe B
has mentioned something that touches on this - web2py (and most
frameworks) use sessions to create an application level context. The
slight problem is if you introduce blocks that are supposed to work/
load in parallel, it will be a little awkward because of how currently
sessions work in web2py IIRC. At best, the blocks will load
sequentially, at worst you will end up with a deadlock.

Another problem is that the user can essentially 'fork' your
application by opening a page in a new tab/page, which can cause all
sorts of interesting interactions with self-aware independently
functioning (but state-wise not independent) blocks.

[Yarko Tymciurak]

On Sat, Apr 4, 2009 at 2:02 AM, Joe Barnhart <joe.ba...@gmail.com> wrote:

I think Yarko makes excellent points.  I too am looking for the
natural way to express components in Web2py and the code proposed by
Massimo originally did not look general enough.  The better job we do
at abstracting this and generalizing it, the more capable the web2py
platform will be.

One of the most interesting approches I have seen is "Seaside" -- the
Smalltalk web app engine.  Seaside relies on "continuations" in
Smalltalk to keep state while the user interacts with different
applets on a page.  It is unclear to me if continuations are needed to
make this work, or just an easier way to accomplish it.  For one
thing, it lets the developer keep objects in his state without needing
to render them as strings and pass them through request and response
vars.  (I think.)  Here is a description of Seaside.  It did "ajax"
before AJAX was invented.

http://scg.unibe.ch/archive/papers/Duca04eSeaside.pdf

This is somewhat interesting - says its history is from NeXT / Apple WebObjects...

Take a look at this Seaside / JQuery example:

http://seasidemo.blobworks.com:9090/seaside/tests/jquery/ajax%20and%20dom%20manipulation?_s=_2xHzedGqAg3wlNn&_k=PLJewPHQyiRAFQJM

.... in thinking about this, I thought of generators (do they have any place here?) - I've never seen this before:  generator expressions (like list comprehensions.. but different?)...

.... more later....

Thanks for this,

-Yarko 

[weheh]

I'm so very glad to see this thread. Frankly, I think partial page
updates targeted at an individual <div id='tag'> should be the default
behavior of web2py (and all web frameworks), not the other way. If you
want to refresh the entire page, just say so. I do not have any
concrete suggestions to help you do this. Only wanted to say that this
topic has been my desire for web2py since day 1, so please ace this
one the same way that you have aced web2py up 'til now. :^)

[mdipierro]

Check views/web2py_ajax.html it should have one more function when
compared to the normal web2py_ajax.html. It may look the same but it
is not.

[mdipierro]

If the "components" are all served by the same they are serialized
even if served via ajax. If they are served by different apps that do
not share a session then they are not serialized. There is no
possibility of deadlocks. Everything is parallelized if there you do
not use session.

Massimo

[mdipierro]

Thank you. Excellent paper. I will get back to you on this.

[AchipA]

Is then this a proper moment to suggest a

@nosession

decorator for controller functions to be able to elegantly
parallelize ?

[mdipierro]

session.forget()

is there already.

[mdipierro]

Note to self. Things to be fixed:

1) the action needs knowledge on whether it is in a container.
Solution: add a header that says so.

2) the action response needs to be able to reference other containers
in order to trigger actions there. Right now each container is
identified by a unique but random id and there is no way to guess it.
Solution: have the user specify the "name" of each container.

3) if a container contains a form that selft submit and then redirect
the content to a another action that also has a form. This second form
will not work properly. Solution 1: the programmer needs to specify
the [SQL]FORM(_action=) but this is a problem form legacy code that
you want to ajaxify. Solution 2: have the jDiv code automatically
determine the action from context.

4) right now every container has a link (that triggers the loading of
the content) and a div with the content. To what extent should this be
customizable? For example, you may want the container to have a top
bar with additional buttons. Should this customization be a feature of
the current container class (jDiv) or should the class be subclassed?
This opens a big can of worms since the "frame" of the components
needs to have some standard hooks in order to expose functions like
"reload the content", "close it", etc. (what else?).

can you think of anything else?

Massimo

[Yarko Tymciurak]

(literally on my way out the door ... don't know why I looked before shutting down.. but there's always a reason!)

On Sat, Apr 4, 2009 at 11:50 AM, mdipierro <mdip...@cs.depaul.edu> wrote:

Note to self. Things to be fixed:

1) the action needs knowledge on whether it is in a container.
Solution: add a header that says so.

Not only container, but container state (e.g. minimized) --- another way to describe this:  to be encapsulated by a container object, the action must ... probably be an object, and implement some minimal interface (set of methods / accessors).  What is the minimal and optional (but standard, defined) set?

 

2) the action response needs to be able to reference other containers
in order to trigger actions  there. Right now each container is
identified by a unique but random id and there is no way to guess it.
Solution: have the user specify the "name" of each container.

This could be door to injections...  there may be some insight in the way android allows applications to "register intents" which then a system finds to see if there is an appropriate intent, and provides a connection to the service...

This is a _little_ different, but the concept may be useful... I'll provide links to slides (they are public on android sites) that show the flow and how this works when I return in a week...

 

3) if a container contains a form that selft submit and then redirect
the content to a another action that also has a form. This second form
will not work properly. Solution 1: the programmer needs to specify
the [SQL]FORM(_action=) but this is a problem form legacy code that
you want to ajaxify. Solution 2: have the jDiv code automatically
determine the action from context.

this sounds like it might be a little like registering intent, and connecting with action which fulfills intent....  maybe...

 

4) right now every container has a link (that triggers the loading of
the content) and a div with the content. To what extent should this be
customizable? For example, you may want the container to have a top
bar with additional buttons. Should this customization be a feature of
the current container class (jDiv) or should the class be subclassed?
This opens a big can of worms since the "frame" of the components
needs to have some standard hooks in order to expose functions like
"reload the content", "close it", etc. (what else?).

Right - this is the interface definition I was speaking of earlier --- suggestion:   list everything you can think of at first, collate what kinds of things those things are later... like writing a book, this will get bigger first before it gets smaller (the smallest set of useful generalizations that fit the problem space)....   I think that is natural process, and "going with the flow" of that is a good thing ;-)

[AchipA]

Not disputing the functional aspect of session.forget, just saying it
would be more elegant, especially for people who use editors that
support folding.

[ceej]

I think there should defiantly be an option to have the contents
already loaded without having to click a link.

[mdipierro]

You can

jDiv('name',URL(....),timeout=0.01)

Massimo

[ceej]

Also a choice of what div to load the response in and maybe even what
multiple divs to effect on submit.

[mdipierro]

You can set a very small timeout.

[mdipierro]

I solved 1), 2) 3) below although I have not yet addressed Yarko's
comments.

The latest code is here:

http://www.web2py.com/examples/static/web2py.app.events.tar

And you can try it online here now:

http://www.web2py.com/events/default/index

Here is the source code of the controller posted:

def index():
import time
response.flash="test"
return dict(partial1=jDiv('click me for text or wait','mycallback1',
3000),

partial2=jDiv('click me for flash','mycallback2'),
partial3=jDiv('click me to redirect','mycallback3'),

partial4=jDiv('click me for form','page1'),
partial5=jDiv('click me to login','login'),
partial6=jDiv('click me to register','register'),
partial7=jDiv('click me to logout','logout'),
time=time.ctime())

### a partial object is a subset of the HTML
page
def mycallback1():
return 'hello world!'

### a partial can trigger an action such as a
flash
def mycallback2():

jDiv.flash('this is a test')

### a partial can trigger an action such as a page
redirect
def mycallback3():
jDiv.redirect('http://www.google.com')

def login():
return auth.login(onaccept=lambda form: jDiv.flash('welcome
'+auth.user.first_name))

def register():
return auth.register(onaccept=lambda form: jDiv.flash('try
login'))

def page1():
form=FORM('a=',INPUT(_name='a',_class='integer',
requires=IS_INT_IN_RANGE(0,1000)),INPUT(_type='submit'))
if form.accepts(request.vars):
session.a=int(form.vars.a)
redirect('page2')
return form

def page2():
form=FORM('b=',INPUT(_name='b',_class='integer',
requires=IS_INT_IN_RANGE(0,1000)),INPUT(_type='submit'))
if form.accepts(request.vars):
session.a+=int(form.vars.b)

redirect('page3')
return form

def page3():
return 'a+b=%s' % session.a

On Apr 4, 11:50 am, mdipierro <mdipie...@cs.depaul.edu> wrote:

[Joe Barnhart]

The syntax that looks like a list comprehension is really more like a
lambda:

html submitButton
onClick: (html jQuery ajax
script: [ :s | s << (s jQuery: #logger) html: DateAndTime
now ]);
with: 'Replace'.

The square brackets above enclose a "block" in Smalltalk. The
variable inside the block (s:) is passed in when the block is
evaluated. In this case it is sent the message "jQuery: #logger" and
the result of that message is sent the message "html: DateAndTime
now". From the context I can tell the s variable is an object that
obeys a Stream protocol and it is being sent messages that render html
onto the stream.

The really interesting thing about Seaside is not necessarily how it
does jQuery or other AJAX calls. It's that it can do "ajax like"
things in a browser with no AJAX at all. You can have several
different regions of your page work independently of each other, like
separate applications or parts of a single application. It is a
facinating environment and it kicks RoR to heck and back. In fact,
the ONLY weird or hard-to-accept thing about it is that it's based on
Smalltalk! <g>

-- Joe B.

On Apr 4, 7:27 am, Yarko Tymciurak <yark...@gmail.com> wrote:

>
> This is somewhat interesting - says its history is from NeXT / Apple
> WebObjects...
>
> Take a look at this Seaside / JQuery example:
>

> http://seasidemo.blobworks.com:9090/seaside/tests/jquery/ajax%20and%2...

[Brian]

These might be some additional requirements (let me know what you
think):

5) Make it still functional when javascript is turned off. If this
doesn't work, sites that require Section 508 compliance won't be able
to use jDIV.

6) Nested containers should work (might work already, I haven't tested
it)

I really like where this is going.

Cheers,
Brian

On Apr 4, 11:50 am, mdipierro <mdipie...@cs.depaul.edu> wrote:

[mdipierro]

5) may be though
6) I believe this works already.

[mdipierro]

Here is one more demo:

http://www.web2py.com/events/default/index

Here is the source code:

http://www.web2py.com/examples/static/web2py.app.events.tar

Look into default.py and the associated view.

Massimo

[Yarko Tymciurak]

This contains a brief description of intents on Android (as I promised):

http://www.openexpo.ch/fileadmin/documents/2008Zuerich/Slides/33_Printemps.pdf