formIO

[Jamie Hillman]

Hi,

I have been working on a form generation script similar to CatWalk but
for a slightly different purpose. Instead of being an 'admin interface'
for an adminstrator-like role to add and remove objects, formio is just
for saving time by auto-generating forms. You could use it for an admin
interface or to generate forms for the end user to use. Anyway, there's
a description of it at:

http://www.jamiehillman.co.uk/formio

In addition to auto-generating forms, it will automatically create
SQLObjects when the forms are posted. I haven't realeased any code as I
wasn't sure if it'd be useful to anyone. If people are interested i can
release the code.

Thanks,

Jamie

[Kevin Dangoor]

Hi Jamie,

This is a great bit of input. Between Ronald's CatWalk, Ian Bicking's
FormEncode ideas, bonono's CRUD interface, Robert Brewer's RESTful
recipe for CherryPy and our discussions at the sprint, we've got a
whole bunch of ideas for doing CRUD with TurboGears.

I'm steadily running out of time for this week. I'm actually on
vacation from this Friday through Thursday, which means that I will be
somewhat sporadically available. As soon as I can, though, I will try
to distill as many of these ideas into a coherent proposal.

Maybe it's just me being a stick in the mud, but I've generally not
been keen on putting presentation information directly into
SQLObjects. That muddies the MVC separation and also potentially makes
internationalization more difficult.

But your ideas certainly extend beyond their position in the
SQLObjects. Thanks for writing that document up!

So, TurboGears' fabulous 0.9 release (codename "Dilly") is looking
pretty firm to have authorization, simple CRUD and improved JSON
serialization at the very least. That will be a perfect stepping stone
to 1.0.

Kevin

--
Kevin Dangoor
Author of the Zesty News RSS newsreader

email: k...@blazingthings.com
company: http://www.BlazingThings.com
blog: http://www.BlueSkyOnMars.com

[Jamie Hillman]

Hi Kevin,

>Maybe it's just me being a stick in the mud, but I've generally not
>been keen on putting presentation information directly into
>SQLObjects. That muddies the MVC separation and also potentially makes
>internationalization more difficult.
>
>

I don't think you *have* to look at the 'annotations' I use for formIO
as presentation information. It can still be viewed as 'model' -
logical groupings of the fields of an SQLObject and longer names for the
fields. At least - I don't think it is so obviously view that it
violates MVC separation, but that's just my take on it - I seem to
remember Ian agreeing with you when I spoke to him about this a long
time ago. If you do enforce such a strict separation of model, view and
controller however, where else do you put this information? Personally
i'd accept muddying MVC seperation (if it is muddying it) if it meant
sticking to DRY. Moving the metadata necessary for generating these
views away from the core model means you have information about what is
essentially the data model in two or more places.

I thought about alternative places for this information when I wrote the
script, but couldn't really come up with anything I prefered over this
solution. Ordering and long names could be specified in seperate
objects, which would mean you could have different names and orderings
for different purposes, which may well be a more flexible solution. I
just like having one place for that all of that kind of information.

It is possible to generate the forms without this extra information of
course - using the ordering in the sqlmeta object of an SQLObject and
just the variable names for form prompts, but it's not as friendly.

Anyway, i'll stop rambling now!

>But your ideas certainly extend beyond their position in the
>SQLObjects. Thanks for writing that document up!
>
>

No problem, if you want to integrate any part of what i've done with
anything else or if you have better ideas for where to put the metadata
i'd be happy to help work on it.

Jamie

[Ian Bicking]

Hi; couple notes, some just general and possible applicable to CatWalk
as well. For datetime inputs my favorite widget is the
formerly named "Coolest Javascript Calendar Widget", but apparently they
lost the "Coolest". Sad, I liked that name. Anyway, it's here:
http://www.dynarch.com/projects/calendar/

artistHistory, artistDetail are clearly screaming out to be classes ;)
The analogous class in formencode.fields would be Layout. A general way
of using classes might be like:

class Artist(SQLObject):
_connection = hub

layout = Layout()

artistDetails = Layout(description='Artist Details')
layout.append(artistDetails)

name=StringCol()
name.prettyName="Artist Name"
artistDetails.append(name)

type=EnumCol(enumValues=['Solo','Group'])
type.prettyName="Artist Type"
artistDetails.append(type)

noofmembers=IntCol()
noofmembers.prettyName="Number of Members"
artistDetails.append(noofmembers)

...

But whether you append or use lists of string names, it doesn't matter
too much. Either or both seems fine -- one way or the other you'll have
to resolve them to the "real" column objects, which are only created
after class creation (and after the body of the class is executed).

> The next thing I need to do for the project i'm working on is provide
> error handling - this will basically involve regenerating the form
> with error messages next to any fields that were too long or of the
> wrong type. I also plan to update this page to show how joins work
> and to make support for joins more rich. At the moment the joined
> object always has 1 subform generated within the form. I'm going to
> implement support for multiple forms (for multiplejoins) and to have
> collapsible forms (like django!).

For larger forms I've been working on some Javascript (using MochiKit)
that roughly implements the WHAT-WG spec for repeating forms:
http://www.whatwg.org/specs/web-forms/current-work/#repeatingFormControls
-- I'll see if I can get it out in a rough form before I go.

Also, it would be useful to be able to give more validators that only
apply to the web form. Generally the model should have the most
permissive validation that you've ever had -- all current objects should
be "valid". But the actual form could be more restrictive.

I think it's generally more pain than its worth to try to do all the
steps at once -- error filling, filling in defaults, and filling in
possible request fields, plus form generation. But maybe not, hard to
know until you are finished.


--
Ian Bicking / ia...@colorstudy.com / http://blog.ianbicking.org

[Ian Bicking]

Kevin Dangoor wrote:
> This is a great bit of input. Between Ronald's CatWalk, Ian Bicking's
> FormEncode ideas, bonono's CRUD interface, Robert Brewer's RESTful
> recipe for CherryPy and our discussions at the sprint, we've got a
> whole bunch of ideas for doing CRUD with TurboGears.

Hopefully we can consolidate some of these things. Part of why I'm just
calling the formencode.fields and formencode.formgen stuff
"experimental" is because I'm not insistant that they have a future in
that form, or that they might fold into another library or have another
library folded into them.

Some features I would find useful in CRUD/form generation, which I've
tried to touch on in these libraries:

* Simple way to override and tweak. In the case of generating
formencode.fields.Field objects, you can poke and rewrite the objects in
an imperative style after the form has been generated. I think it's
relatively hard to pass in sufficient options to modify the generation
itself, except for very specific aspects (like the field names). Old
FormEncode form generation tried this, and it's just messy and annoying.

* Way to unpack a form and use it in a template.
formencode.fields.Layout *doesn't* do this well right now, but it
should. I'd like to be able to do:

<form>
<?python
layout = form.layout
?>
<table>
<tr><td>First Name</td><td>Last Name</td></tr>
<tr>${layout.fname}</tr><td>${layout.lname}</td></tr>
</table>
</form>

And stuff like that, in addition to generating the complete form with
fieldsets and all that.

* Allow fields to add validators. For instance, a secure hidden field
can use the validators.SignedString validator, and just be implemented
as a hidden field. Or a date input validator could unpack dates into a
dictionary of {'year': ..., 'month': ...}, and then unpack them.

* variabledecode is the bomb if you want to make complex forms, in part
because of that unpacking and packing step that can make a bunch of
fields look like a single field to the validator (and to the model
layer), and because it make repeating fields pretty easy to use.

* Decentralized rule building, using generic functions. They are like
adapters and registered views and all that, only they are really easy.
I think they are easier than even a fairly naive ad hoc adaptation
system, and more powerful. You can implement simple
if-this-attribute-is-present systems using a rule that tests with
hasattr(). This can work right down to the column layer. This is what
sqlformgen does (though only for two column types so far -- proof of
concept).


But when it comes down to it, there's a lot of grunt work and
experimental programming that has to go on, to see how this stuff works
in realistic situations. I think false starts should be expected, and
that's fine as long as there's no premature commitment (which makes it a
little questionable for TG 0.9, IMHO -- unless you have a plan for how
deprecated features will work).

[bon...@gmail.com]

Having been burnt so many times in the pass by various "code generator"
middle wares including ASP.NET, I have adopted a policy of only pass
the data/metadata around and let the programmer(be it js, python or
TSQL) do the last mile, in their problem domain.

What I have is like this:

1. data table created with SQL script
2. SQLObject can query the metadata from the database(field names, type
etc.)
3. My very crude python code would again query so.sql.metadata.columns
to get these information
4. adopting the mochiKit ajax_table example, the data is then passed on
to Kid as a dict(columns=[field names],rows=[[row data]])
5. kid is used to popular the form
6. mochiKit's ajax_table would do it again for the appearance and
behaviour.

There is no automatic generation of anything but just passing around
either list or dict and let the programmer at various level to decide
what to do with them. Say enhance the browser UI with dojo, when it
becomes mature.

[Jamie Hillman]

Hi,

> Hi; couple notes, some just general and possible applicable to CatWalk
> as well. For datetime inputs my favorite widget is the
> formerly named "Coolest Javascript Calendar Widget", but apparently they
> lost the "Coolest". Sad, I liked that name. Anyway, it's here:
> http://www.dynarch.com/projects/calendar/

Thanks, i'll have a look at that :-)

> artistHistory, artistDetail are clearly screaming out to be classes ;)
> The analogous class in formencode.fields would be Layout. A general way
> of using classes might be like:
>
> class Artist(SQLObject):
> _connection = hub
>
> layout = Layout()
>
> artistDetails = Layout(description='Artist Details')
> layout.append(artistDetails)
>
> name=StringCol()
> name.prettyName="Artist Name"
> artistDetails.append(name)
>
> type=EnumCol(enumValues=['Solo','Group'])
> type.prettyName="Artist Type"
> artistDetails.append(type)
>
> noofmembers=IntCol()
> noofmembers.prettyName="Number of Members"
> artistDetails.append(noofmembers)

> ...
>
> But whether you append or use lists of string names, it doesn't matter
> too much. Either or both seems fine -- one way or the other you'll have
> to resolve them to the "real" column objects, which are only created
> after class creation (and after the body of the class is executed).

I have these names resolving to the column *classes*, which seems to be
enough for my purposes. I need to look at formencode in detail - at the
moment I have html fragment templates for generating the actual html
forms - using formencode would be much nicer.

> For larger forms I've been working on some Javascript (using MochiKit)
> that roughly implements the WHAT-WG spec for repeating forms:
> http://www.whatwg.org/specs/web-forms/current-work/#repeatingFormControls
> -- I'll see if I can get it out in a rough form before I go.
>
> Also, it would be useful to be able to give more validators that only
> apply to the web form. Generally the model should have the most
> permissive validation that you've ever had -- all current objects should
> be "valid". But the actual form could be more restrictive.

Yeah that sounds good - this is probably something that wouldn't be best
placed in the SQLObject schema. But (in the case of formIO) the form
generation call could take a (chain of) validator(s) as an argument.

> I think it's generally more pain than its worth to try to do all the
> steps at once -- error filling, filling in defaults, and filling in
> possible request fields, plus form generation. But maybe not, hard to
> know until you are finished.

Building a framework that automates everything and copes with every case
sounds like extreme pain to me. Formio has saved me some time but when
it gets in the way, i've been tending to just not use it - rather than
extending it into a hideous monster framework (perhaps that's too much
time spent with Swing making me over-sensitive to such things). I think
that's an important thing to remember though - whatever may or may not
be integrated into turbogears need not solve every problem to be useful.


Jamie

[Jamie Hillman]

Hello again,

There's obviously a lot of thought gone into formencode, I haven't
looked at it for a while but sqlformgen sounds like a better basis for
this kind of functionality than what I have (and formencode a better
place to develop this kind of thing).

I'd like to convert what I have into a more formencode-centric version
and see how that looks, without stepping on whatever you have done
already.


Jamie

[Ian Bicking]

Jamie Hillman wrote:
> There's obviously a lot of thought gone into formencode, I haven't
> looked at it for a while but sqlformgen sounds like a better basis for
> this kind of functionality than what I have (and formencode a better
> place to develop this kind of thing).
>
> I'd like to convert what I have into a more formencode-centric version
> and see how that looks, without stepping on whatever you have done
> already.

Don't worry too much about what I've done -- I don't want my ideas to be
blockers for other people doing work on this, or for the perfect to be
the enemy of the possible. In a practical sense, it's probably most
useful to build concrete applications like CakeWalk, and then reflect on
what's in there, extract, refactor, etc.

That said, I think there's some good ideas in FormEncode, and some
Lessons Learned The Hard Way, and I think this work will be more
successful if it is at least informed by that. And ultimately I think
FormEncode is a good home for this stuff.

[Kevin Dangoor]

On 10/12/05, Ian Bicking <ia...@colorstudy.com> wrote:
> But when it comes down to it, there's a lot of grunt work and
> experimental programming that has to go on, to see how this stuff works
> in realistic situations. I think false starts should be expected, and
> that's fine as long as there's no premature commitment (which makes it a
> little questionable for TG 0.9, IMHO -- unless you have a plan for how
> deprecated features will work).

I should note that the idea with including some amount of CRUD in 0.9
is to put something out there and see if people think it's on the
right track for their needs. If it will take too long to do that
reasonably, or if people hate the feature as it appears in 0.9, it
will be tossed for 1.0. If, on the other hand, it makes it into 0.9
and people like the general direction, then it will likely get refined
a bit and then 1.0 comes out.

I do have thoughts in mind regarding deprecating features (post 1.0),
and you can be sure that major changes will be discussed on the list.

Kevin

[bon...@gmail.com]

I still think a cookbook/template/sample approach like the ajax_table
in MochiKit is better unless you want to gear every TurboGears user to
a particular model(RoR or to a less extend Nevow), which I assume was
not the initial intend.

[Kevin Dangoor]

On 10/18/05, bon...@gmail.com <bon...@gmail.com> wrote:
>
> I still think a cookbook/template/sample approach like the ajax_table
> in MochiKit is better unless you want to gear every TurboGears user to
> a particular model(RoR or to a less extend Nevow), which I assume was
> not the initial intend.

People will always be free to "roll their own", as far as CRUD goes...
it would be purely an addition to the framework.

One way to look at it is that it's like the "quickstart". Rather than
starting you off with a blank slate, it puts you part of the way
toward your final goal. The key is providing a straightforward,
non-magical way to get from point A (completely generated CRUD) to
point B (the CRUD that an individual application requires).

People seem to really like Django's admin interface, so it must not be
completely impossible to do CRUD that people like.

Kevin