Small report from Django/Rails meetup

[Robert Wittams]

So I went to the London Django/Rails meetup yesterday. In general a good
time was had - met Simon Willison, and some ThoughtWorks guys doing a
GreenPeace site with Django.

The general feeling from those using or considering Django (including
some rubyists) seemed to be "Release a 0.7 tarball, for the love of all
that is holy!"

It seems that quite some people just aren't comfortable with checking
things out of subversion. They don't particularly want backwards
compatibility, just a tarball. This would probably widen the number of
people willing to use/try out Django.

There seems to be some plan of releasing 1.0 in the near future. I'm not
convinced that is a great idea if backwards compatibility needs to be
preserved after this, as there are still a number of fairly major things
that do need work and thought, as well as practical experience.( eg
authorisation, schema transitions, etc). I personally do not know of any
pressing need to release a 1.0 .

It doesn't seem that being pre-1.0 has had a significant negative impact
on the hype surrounding certain other projects - but having no releases
does seem to be having some negative effect, however small, on us.

Any thoughts?

[Adrian Holovaty]

On 11/8/05, Robert Wittams <rob...@wittams.com> wrote:
> The general feeling from those using or considering Django (including
> some rubyists) seemed to be "Release a 0.7 tarball, for the love of all
> that is holy!"
>
> It seems that quite some people just aren't comfortable with checking
> things out of subversion. They don't particularly want backwards
> compatibility, just a tarball. This would probably widen the number of
> people willing to use/try out Django.

If all this means is automatically packaging a tarball every couple of
days/weeks, let's do it.

Adrian

--
Adrian Holovaty
holovaty.com | djangoproject.com | chicagocrime.org

[Jacob Kaplan-Moss]

On Nov 8, 2005, at 10:13 AM, Adrian Holovaty wrote:
> On 11/8/05, Robert Wittams <rob...@wittams.com> wrote:
>> The general feeling from those using or considering Django (including
>> some rubyists) seemed to be "Release a 0.7 tarball, for the love
>> of all
>> that is holy!"
>>
>> It seems that quite some people just aren't comfortable with checking
>> things out of subversion. They don't particularly want backwards
>> compatibility, just a tarball. This would probably widen the
>> number of
>> people willing to use/try out Django.
>>
>
> If all this means is automatically packaging a tarball every couple of
> days/weeks, let's do it.

I think we need to bite our lips, suck it up, and release a 1.0
version. It seems that we keep running into feeping-creaturism every
time we try to set a stake in the ground, and that's limiting
adoption. Obviously everyone is unhappy when their tickets aren't
included, but more tickets are being opened than fixed. I think we
need to accept that certain things might not get done, release a 1.0,
and move on.

[Calling it "0.7" would inaccurately represent how stable and usable
Django is, which is why I say 1.0.]

If it sounds OK, I'd like to start a 1.0 release branch and only
apply any outstanding bug fixes to it; moving feature requests/
patches to a 1.1 target. That way we can get a stable 1.0 out the
door and focus on 1.1 for feature improvements.

In terms of backwards-compatibility, it seems most of the big deal
architectural changes have already been done, and as long as we
provide VERY EASY upgrade paths from 1.0 -> 1.x I think we're OK.

So, any objections to starting a 1.0 bug-fix-only release branch?

Jacob

[Eric Walstad]

On Tuesday 08 November 2005 08:35, Jacob Kaplan-Moss wrote:
> I think we need to bite our lips, suck it up, and release a 1.0  
> version.

+1

A "stable" release would make those who are trusting my judgement in
choosing Django for a medium-large-ish project a little less nervous
(me, too).

[Adrian Holovaty]

On 11/8/05, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> If it sounds OK, I'd like to start a 1.0 release branch and only
> apply any outstanding bug fixes to it; moving feature requests/
> patches to a 1.1 target. That way we can get a stable 1.0 out the
> door and focus on 1.1 for feature improvements.
>
> In terms of backwards-compatibility, it seems most of the big deal
> architectural changes have already been done, and as long as we
> provide VERY EASY upgrade paths from 1.0 -> 1.x I think we're OK.

Here's what we should finish before this first 1.0 version:

* Transactions.
* New-admin branch.
* Reworking of the RSS framework (I believe there's already a patch,
which I can roll in).
* Documentation.
* Screencasts so fucking exciting you'll cry.

[Jakub Labath]

Eric has a point.
It would help to have a stable release as a selling point to my
customers/managers. Even in the case when I would in fact use the code
in svn for the actual work.:-)
Unfortunately world is all about perceptions.

[espen.g...@gmail.com]

I got to say that I really think what adrian says is the right thing to
do. Get the docs done, and merge the new-admin got to be top priority
in my opinion.

[Jacob Kaplan-Moss]

On Nov 8, 2005, at 11:08 AM, Adrian Holovaty wrote:
> Here's what we should finish before this first 1.0 version:
>
> * Transactions.

Agreed.

> * New-admin branch.

What's the state of this branch? Last time I tried it out -- a few
weeks ago, IIRC -- there seemed to be a bunch more work to be done.
I'd obviously LOVE to see this get rolled in, but if it's a "few
months" thing I don't think it would hurt to let it hit in 1.1.

> * Reworking of the RSS framework (I believe there's already a patch,
> which I can roll in).

Agreed.

> * Documentation.

What do you think we ought to document? RSS and the comments
framework come to my mind; anything else? We should come up with a
list and work out who will write what.

> * Screencasts so fucking exciting you'll cry.

(Sounds like something Rob would say <grin>)

These might be more difficult as they should ideally be taken *of*
1.0 (not before), so they might need to be something we do after 1.0
is done. Perhaps the best approach would be to make these when we
hit a 1.0 release candidate (after all the above is done).

To this list, I would add two things:

* A solution to the "OR" dilemma; #251 seems like an OK fix to me.
* Solidifying the middleware API and the "stacking" issue (I've got a
draft proposal of this including an explanation of why I did it the
way I did; I'll try to wrap it up and post it soon).

Jacob

[Adrian Holovaty]

On 11/8/05, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:

> What's the state of this branch? Last time I tried it out -- a few
> weeks ago, IIRC -- there seemed to be a bunch more work to be done.
> I'd obviously LOVE to see this get rolled in, but if it's a "few
> months" thing I don't think it would hurt to let it hit in 1.1.

I'd like to see a solution to "core=True" before 1.0 -- i.e., not
having to use that anymore. This goes beyond what new-admin offers,
and it would probably be a backwards-incompatible change (hence the
1.0 requirement).

> What do you think we ought to document? RSS and the comments
> framework come to my mind; anything else? We should come up with a
> list and work out who will write what.

* RSS
* Comments framework
* Authentication (already started; I need to finish)
* Admin-site documentation (already started on my laptop; I need to finish)
* Views (already started on my laptop; I need to finish)
* Finish tutorial
* How to contribute to Django, and how development works
* How to integrate third-party template systems

I'd prefer to do all of these myself, with the exception of the last two.

> These might be more difficult as they should ideally be taken *of*
> 1.0 (not before), so they might need to be something we do after 1.0
> is done. Perhaps the best approach would be to make these when we
> hit a 1.0 release candidate (after all the above is done).

Good call.

> * A solution to the "OR" dilemma; #251 seems like an OK fix to me.

Yes, we should fix this one. #251 still rubbed me the wrong way last
time I checked, but I'll give it some more thought.

> * Solidifying the middleware API and the "stacking" issue (I've got a
> draft proposal of this including an explanation of why I did it the
> way I did; I'll try to wrap it up and post it soon).

Looking forward to it...

[Pedro Furtado]

2005/11/8, Adrian Holovaty <holo...@gmail.com>:

>
> * RSS
> * Comments framework
> * Authentication (already started; I need to finish)
> * Admin-site documentation (already started on my laptop; I need to finish)
> * Views (already started on my laptop; I need to finish)
> * Finish tutorial
> * How to contribute to Django, and how development works
> * How to integrate third-party template systems

Why don`t we include Django-Ajax in 1.0? I agree that releasing it
sooner is good for the project but a nice ajax interface is the future
for all web applications. Don`t you agree this is an imprescindible
update?

[Jonathan Daugherty]

# Why don`t we include Django-Ajax in 1.0? I agree that releasing it

# sooner is good for the project but a nice ajax interface is the
# future for all web applications. Don`t you agree this is an
# imprescindible update?

I think that really goes into the "bells and whistles" category.
Admittedly, AJAX integration of some kind would be nice, but IMHO
there are lots of other bigger, more important things going on that
need to be considered.

--
Jonathan Daugherty
http://www.parsed.org

[Jeremy Dunck]

On 11/8/05, Adrian Holovaty <holo...@gmail.com> wrote:
> * Screencasts so fucking exciting you'll cry.

I'll stock up on gatorade.

[Jeremy Dunck]

On 11/8/05, Pedro Furtado <pedrol...@gmail.com> wrote:
> Don`t you agree this is an imprescindible
> update?

I dunno what imprescindible means. :)

[Maniac]

Jacob Kaplan-Moss wrote:

> So, any objections to starting a 1.0 bug-fix-only release branch?

No objection but a concern...

Some time ago I filed a ticket
(http://code.djangoproject.com/ticket/570/) about FormWrapper not
working for ForeignKey fields. It's rather basic functionality and I
consider it a major disfunction to be fixed for 1.0. But the ticket was
duplicated against 'new-admin' branch which fixes this bug.

Now I'm afraid that the whole 'new-admin' could be considered a large
dangerous feature for 1.0 and thus left out. If so then we should
extract any critical fixes (such as 570) from it and apply them to the
1.0 branch.

[Jacob Kaplan-Moss]

On Nov 8, 2005, at 11:30 AM, Adrian Holovaty wrote:
> I'd like to see a solution to "core=True" before 1.0 -- i.e., not
> having to use that anymore. This goes beyond what new-admin offers,
> and it would probably be a backwards-incompatible change (hence the
> 1.0 requirement).

That's a good point -- core=True sucks (just got bit by it again
today: don't make a radio button field to core or else once you've
set it you can never erase the row *sigh*).

Robert, is this something you were planning on including the new-
admin, or is it something that would need to be done once those
changes hit the trunk?

> * How to contribute to Django, and how development works
> * How to integrate third-party template systems
>
> I'd prefer to do all of these myself, with the exception of the
> last two.

I'll happily write those; I've got strong opinions about both of them :)

> Yes, we should fix this one. #251 still rubbed me the wrong way last
> time I checked, but I'll give it some more thought.

I also don't like #251 that much, but I'd take it over the scary
undocumented "_or" we've got now (and over raw SQL, which sucks).

Jacob

[Eugene Lazutkin]

Inline.

"Adrian Holovaty" <holo...@gmail.com> wrote
in message
news:6464bab0511080908p306...@mail.gmail.com...

On 11/8/05, Jacob Kaplan-Moss
<ja...@jacobian.org> wrote:

> * Transactions.

There are problems with transactions in caching --- sometimes database is
locked up for no reason without any errors. I didn't have time to look at it
but I am getting some bad side effects time to time. Of course, transaction
for data updates would be nice to have as well.

> * Screencasts so fucking exciting you'll cry.

+10! That is how it works nowadays. I don't know why they are so popular. I
gave up explaining this phenomenon. But confronted with data from my web
site I just admit that it is a fact of life.

Additional wish: multithreaded MySQL support. Unfortunately MySQL is much
more popular with hosting companies than Postgres. There is a patch for
MySQL. I use it for a long time without problems.

Thanks,

Eugene

[Ian Holsman]

personally I'd like to see the user-registration app get
documented/released as well.
what is the state of this?

--
I...@Holsman.net -- ++61-3-9877-0909
If everything seems under control, you're not going fast enough. -
Mario Andretti

[Jacob Kaplan-Moss]

On Nov 8, 2005, at 2:33 PM, Ian Holsman wrote:
> personally I'd like to see the user-registration app get
> documented/released as well.
> what is the state of this?

The registration app (as opposed to just the user module) is
currently part of our proprietary software package and I'm not sure
it's something we want to release; I'll talk it over with some people
internally and look into releasing it as django.contrib.registration.

Jacob

[Robert Wittams]

Ok, so this generated quite a bit of traffic.

I *really* don't think that 1.0 should be considered on the basis of
stability and usability of the implementation. It should be on the basis
of "is this a reasonable base to offer backwards compatibility for", ie
how much do we *know* is going to have to change.

These are the things that definitely need to happen before we end up
with a commitment to back compat nightmares:

1) Get rid of core fields * ( this blocks a surprising amount of stuff).
2) Sort out whether magic model modules are really a good idea. Adrian
was talking about this, I also talked to Simon about this yesterday at
the meetup. I am in favour of getting rid of them.
3) Some kind of story on authorisation that works for people other than
newspapers, and isn't an enforced ACL mire. *
4) Transactions
5) Real subtyping as discussed a few weeks ago *
6) Proper schema checkpointing and transitioning *
7) Extract the admin history thing from manipulators into a seperate app*

I was planning to do the * items in a branch before 1.0, with the
explicit aim of doing every breaking change in one lump before 1.0 so as
to minimise the number of times people have to mess with their models.

The reason I haven't done any of these yet is because new-admin is
supposed to be backwards compatible model wise. I could change this, but
it makes checking for regressions much harder.

I have no idea why we have to marry the idea of releasing a tarball to
the number 1.0, but the number 1.0 is already explicitly married to
backwards compatibility in the eyes of most users. IMO there are still a
number of areas that need a fair amount of work.

I would be much happier just releasing a tarball with no guarantees. I'd
even be happier about releasing nothing than rushing 1.0. This is
exactly the scenario I'd hoped to avoid with my message, emphasising
that right now, people seem far more concerned about just having a
tarball out there than they do about version numbers or backwards
compatibility.

So when we say "release a 1.0, and move on.", we are IMO making an
explicit back compat commitment, and so moving on is just what we will
be precluding.

------------------
As to the state of new-admin at the moment, it is fairly complete
functionality wise, and stability wise I do not believe it is
significantly worse than trunk. I do still have a very small number of
bits to templatise - date_hierarchy in the change_lists being the main
offender.

I'm still not satisfied with the api of auto generated forms - there is
some duplication between formfieldwrappers and the tower of bound
classes. I think that I will leave this to stew for a while, and fix it
properly in the backwards incompatible branch - I have a feeling the
clean solution to this may rely on getting rid of core fields. This is
why I have not documented it at all in NewAdminChanges.

[hugo]

>I also don't like #251 that much, but I'd take it over the >scary
>undocumented "_or" we've got now (and over raw SQL, >which sucks).

How about another solution to expression combination: wrapping query
expressions (the field__exact and friends) optionally in Query()
objects that implement __or__ and __and__ - that way you can do
Query(field__exact=blah)||Query(field__exact=blubb).

I did something like this for my active storage framework (a rather
weird database thingy - don't ask) and it worked quite nice. You would
need to extend the get_list() and friends to check for a parameter that
isinstance(Query) and than "compile" the query expression to SQL.

bye, Georg

[hugo]

>I have no idea why we have to marry the idea of releasing a tarball to
>the number 1.0, but the number 1.0 is already explicitly married to
>backwards compatibility in the eyes of most users. IMO there are still a
>number of areas that need a fair amount of work.

I would say: release a 0.9 version now (or in the near future) and give
a clear roadmap (that's what the trac feature is for ;-) ) of what will
go into 1.0 before release. That way people have some "quite stable
but not 1.0" tarball to play with, with a defined
compatibility-breakage ahead, but one they are warned about. That would
give us a bit more time to do the 1.0 right.

bye, Georg

[Adrian Holovaty]

On 11/9/05, hugo <g...@hugo.westfalen.de> wrote:
> I would say: release a 0.9 version now (or in the near future) and give
> a clear roadmap (that's what the trac feature is for ;-) ) of what will
> go into 1.0 before release. That way people have some "quite stable
> but not 1.0" tarball to play with, with a defined
> compatibility-breakage ahead, but one they are warned about. That would
> give us a bit more time to do the 1.0 right.

This sounds great to me. Thoughts, Jacob and everybody else?

[Jeremy Dunck]

On 11/9/05, Adrian Holovaty <holo...@gmail.com> wrote:
>
> On 11/9/05, hugo <g...@hugo.westfalen.de> wrote:
> > I would say: release a 0.9 version now (or in the near future) and give
> > a clear roadmap (that's what the trac feature is for ;-) ) of what will
> > go into 1.0 before release. That way people have some "quite stable
> > but not 1.0" tarball to play with, with a defined
> > compatibility-breakage ahead, but one they are warned about. That would
> > give us a bit more time to do the 1.0 right.
>
> This sounds great to me. Thoughts, Jacob and everybody else?

This sounds something like what Robert originally suggested, with the
version number changed. Which is to say, +1. :)

[Jacob Kaplan-Moss]

On Nov 8, 2005, at 3:41 PM, Robert Wittams wrote:

> Ok, so this generated quite a bit of traffic.
>

Ha -- thanks for kicking this off; it needed to be discussed.

> I *really* don't think that 1.0 should be considered on the basis of
> stability and usability of the implementation. It should be on the
> basis
> of "is this a reasonable base to offer backwards compatibility
> for", ie
> how much do we *know* is going to have to change.
>

I agree that this is a noble goal, and in a perfect world Django 1.X
would be completely compatible with Django 1.Y. However, the reason
I'm wanted to push out something called "1.0" is that Django's
*perceived* instability is hampering its adoption. The fact is that
people read "0.7" as "zero-point-missing-many-features" and shy
away. With enough eyeballs all bugs are shallow, and I want more
eyeballs (mmmm.... eyeballs....)

> These are the things that definitely need to happen before we end up
> with a commitment to back compat nightmares:
>
> 1) Get rid of core fields * ( this blocks a surprising amount of
> stuff).
>

Agreed (as I've said before).

> 2) Sort out whether magic model modules are really a good idea. Adrian
> was talking about this, I also talked to Simon about this yesterday at
> the meetup. I am in favour of getting rid of them.
>

Let's bring this up on a different thread since it's a big issue; my
10-second response is that this would take a *lot* of time, and
aren't there more important things to be doing?

> 3) Some kind of story on authorisation that works for people other
> than
> newspapers, and isn't an enforced ACL mire. *
>

This is a HUGE task; I think you know that. Coming up with a auth
system that doesn't get in the way of most people but has all the
power you could ask of it certainly is going to be difficult, and
finding consensus among developers on how auth "should" work could
easily turn into a holy war (the last time we discussed the topic the
tone turned nasty fairly quickly). Frankly, I'd guess that for over
90% of Django's users the current simple auth system is *too*
complicated for their needs. This isn't to say that I wouldn't LOVE
to see this happen, but simply that it, too, is a lot of work.


> 4) Transactions
>

Agreed.

> 5) Real subtyping as discussed a few weeks ago *
>

I could go either way on this; any way you slice it subclassing is
going to be a mismatch to a relational database, but the current
system is needlessly confusing and could be cleaned up.

> 6) Proper schema checkpointing and transitioning *
>

This is also a big deal. Again, a lot of what makes this difficult
is coming to a community consensus on what "proper" is. However,
this is certainly something that could be added without sacrificing
backwards compatibility; I'd prefer to take a pass on this and push
it to 1.1.

> 7) Extract the admin history thing from manipulators into a
> seperate app*
>

I'm not sure why you feel this is necessary, but it also seems it
could be done with a minimum of backwards-incompatibility (rename
django_admin_log and refactor manipulators slightly). Database
evolution between versions of Django is probably OK as long as the
APIs stay the same.

> I was planning to do the * items in a branch before 1.0, with the
> explicit aim of doing every breaking change in one lump before 1.0
> so as
> to minimise the number of times people have to mess with their models.
>

I really appreciate your intentions, Robert, but going solo on big
features like this is a bad idea from a community standpoint. Many
of the changes you highlight are likely to be controversial and we
*have* to discuss them before they'll make it into Django. For
better or for worse, I don't want this to be the type of open source
project that rolls out big architectural changes without discussing
them with the community.

Reading over what I've written so far, it seems that I'm shying away
from difficult tasks. Perhaps I am, but the fact is that the longer
we linger in a pre-release stage the more our potential community
slips away. I don't use the number "1.0" lightly, and I know that it
locks us in somewhat, but that very lockin is what draws developers
to stable code. Most of the big feature you propose are things that
I would give my left thumb to see added to Django, but at the same
time I don't want Django to become a project that lingers in
perpetual pre-release. If we decide that these features can't be
done in a backwards-compatible way and have to release a 2.0 six
months after 1.0, what's the harm in that?

I've added http://code.djangoproject.com/wiki/VersionOneFeatures as a
way of taking a straw poll on what features people feel would be
neccisary to create before we can release a 1.0; I've tried to
summarize the feelings from this thread, so if I mistook anyone's
opinion I apologize in advance.

Thanks,

Jacob

[Robert Wittams]

+1, don't really care about the particular version number, just the
implications that it has for most people. ;-)

[Robert Wittams]

> If
> we decide that these features can't be done in a backwards-compatible
> way and have to release a 2.0 six months after 1.0, what's the harm in
> that?
>

Thats fine, but that needs to be made absolutely clear. The default that
people are going to expect is that we are "satisfied" with the
interfaces in Django 1.0, and they will be the focus for development for
a while .

I have nothing against releasing 1.0, but only *if* it is made clear
that it is not a long term maintenance release. I see no actual benefits
in releasing something called 1.0 without a back compat guarantee - I
really don't share your idea that you can't get a big community without
a 1.0 release. Just look -gasp- at Rails.

So if the 1.0 release is just a "look at me, I'm usable" release, then I
would cut down what I would like done from my last message to core
fields, subtyping. The other stuff can wait as long as 1.0 can be
semi-abandoned quite quickly.

>> 7) Extract the admin history thing from manipulators into a seperate
>app*
>>

>I'm not sure why you feel this is necessary, but it also seems it
>could be done with a minimum of backwards-incompatibility (rename
>django_admin_log and refactor manipulators slightly). Database
>evolution between versions of Django is probably OK as long as the
>APIs stay the same.

The only reason this is here rather than in new-admin is that it relies
on core fields being removed to do it sanely. It wouldn't actually have
to be backwards incompatible.

[Wilson Miner]

+1 for hugo's suggestion. A tarball now gives us a "stable" version to point people to before we start knocking off backwards-incompatible changes. A finalized roadmap for 1.0 puts the target in sight. And 0.9 says "we're almost there". All in favor

[Eugene Lazutkin]

"Jacob Kaplan-Moss" <ja...@jacobian.org> wrote in
message news:5C836A25-DD02-407A...@jacobian.org...

>
> Reading over what I've written so far, it seems that I'm shying away from
> difficult tasks. Perhaps I am, but the fact is that the longer we linger
> in a pre-release stage the more our potential community slips away. I
> don't use the number "1.0" lightly, and I know that it locks us in
> somewhat, but that very lockin is what draws developers to stable code.
> Most of the big feature you propose are things that I would give my left
> thumb to see added to Django, but at the same time I don't want Django to
> become a project that lingers in perpetual pre-release. If we decide
> that these features can't be done in a backwards-compatible way and have
> to release a 2.0 six months after 1.0, what's the harm in that?

+1.

Waxing philosophical: what version do we have now? 0.0? I didn't think we
were doing 0.x stuff. I did think of it as 1.0 Beta NNN because our goal is
1.0. When we will do incompatible changes after 1.0, it is not 1.x Rev MMM
anymore --- it is 2.0 Beta MMM.

Django was useable the first day it was published. To me that was 1.0
version even it was not called like that. Okay, let's call it 0.0 and
release 1.0 now incorporating small changes and adding a few documents. Why
bother with 0.9 stuff? It is just a name. All big stuff mentioned in this
thread is going to be rolled in and released as 2.0 as soon as it is
finished.

Thanks,

Eugene

[swrainey]

I'm a developer trying to choose what framework to use. I would like to
give my perspective on the situation. I would personally like to see
0.9 released real soon with a roadmap of what is coming for the 1.0
release and what might not be backwards compatible. I can easily see
that this framework has been proven on some high traffic sites and I
feel it's stable. Rails hasn't released 1.0 and it certianly has had
developer buy in. I think releasing the 0.9 and then the 1.0 will make
you look active and agressive.

I also think ajax is is more important than some others think. This is
probably just because there are a lot of other things that truly are
more important.

Ajax is really hot right now and I could see loosing some developers
because it's not as on the forefront of the whole web 2.0 hyped up
junk. Ajax is more about usability than eye candy or at least it should
be. That being said. I know I can use ajax really easy inside of a
django project but will anyone choose another framework based on most
of the other ones having "ajax support".

I think what you guys are going for is wanting developers to think that
your project is stable, relevant, and has a rapidly growing community.

I think you guys have done a great job and I look forward to seeing
this project progress.

[Simon Willison]

On 14 Nov 2005, at 07:10, swrainey wrote:

> Ajax is really hot right now and I could see loosing some developers
> because it's not as on the forefront of the whole web 2.0 hyped up
> junk. Ajax is more about usability than eye candy or at least it
> should
> be. That being said. I know I can use ajax really easy inside of a
> django project but will anyone choose another framework based on most
> of the other ones having "ajax support".

For me "Ajax support" really is pure marketing fluff - as far as I'm
concerned EVERY web framework supports Ajax unless it does something
truly moronic like refuse to let you output documents that don't have
the standard header/footer template.

That said, I know my way around JavaScript and prefer to write it by
hand. I imagine there are many developers out there who don't and
prefer having the framework do the work for them. The Ajax support in
Rails is my least favourite feature, precisely because I like to have
full control over how my JS works - but it makes a lot of Rails
developers very happy indeed.

At the very least, it is useful to have your framework make a few
decisions/recommendations for you - things like which XMLHttpRequest
cross-browser abstraction to use. I mould tend to look towards
MochiKit for that kind of thing since it's more Pythonic than other
JavaScript libraries, taking a lot of its ideas from Python features.

One thing that would be very cool would be some built in support in
Django for outputting JSON, which is a really neat format for sending
data to and from the server via XMLHttpRequest. Maybe a custom
template tag or filter would be useful here.

I know the Ajax in Django discussion has been going on for a long
time, but maybe it's time to take a closer look at it now that we're
thinking about features for 1.0. After all, in the ultra competitive
world of Web Frameworks marketing is important.

Cheers,

Simon

[Ian Holsman]

On 11/14/05, Simon Willison <swil...@gmail.com> wrote:

> For me "Ajax support" really is pure marketing fluff - as far as I'm
> concerned EVERY web framework supports Ajax unless it does something
> truly moronic like refuse to let you output documents that don't have
> the standard header/footer template.

At the point of a developer deciding, it's *ALL* about marketing.
You have about 2-3 clicks for him to decide if the framework is worth
looking further into.

Some developers think ajax is hot stuff, and it will make their app
look sexy in front of their PHB, so they want it supported out of the
box.

Others think it is a waste of space, and they can do the job better by
rolling their own.

this is the real 'marketing' question here..

Who do you want to use django?

If you want django to be popular and in wide use you will need to add
ajax support for those who want it.
The 'trick' i guess is how do you design the framework so it can have
ajax support, and the ability to turn it off.

regards
Ian

[Robert Wittams]

One thing to note here: When we get rid of core fields, we are going to
want to use both ajax & drag and drop in order to implement ticket #13.
It would be nice to use some "consistent" framework to do this, rather
than picking random bits of javascript up from around the internet.

Another thing to consider is ajax (pre-submit) form validation. So we do
have a need for goals within django itself which would best be served by
bundling a consistent js library set : its not just for users.

I think that Mochikit is probably the best option, being fairly light
and pythonic.

It doesn't include drag and drop or many effects - but it should be
possible to port anything we need from eg dojo or scriptaculous, and
maybe some turbogears people will beat us to it ;-P ).

[Stephen Rainey]

I agree with you that it's pure marketing fluff and I guess you see that
was my point. It just got me to thinking when I was reading about
developer adoption. I do like your lightweight ideas. It might be good
to do something rather lightweight and then get some visibility on the
website about Django working with Ajax. Or maybe just try to find
another way to get some ajax visibility without building it into the
core framework.

I work with a partner who was checking out django and ruby on rails. And
his first comment was, it doesn't look like it supports ajax. To him
ajax is highly important not because of eye candy but because we use it
to make applications more usable. I was able to assure him that django
does support ajax.

This is from a pure marketing perpesctive. I guess what I'm saying is
that I just want to make sure that it doesn't get completely written off
because of that. Being a programmer I'm more interested in the technical
goodness, but I'd like to see this also address marketing concerns as
well to gain more community interest and developer adoption.

Just wanted to throw in a different perspective.

[James Bennett]

On 11/14/05, Stephen Rainey <ste...@clearfunction.com> wrote:
> I agree with you that it's pure marketing fluff and I guess you see that
> was my point. It just got me to thinking when I was reading about
> developer adoption. I do like your lightweight ideas. It might be good
> to do something rather lightweight and then get some visibility on the
> website about Django working with Ajax. Or maybe just try to find
> another way to get some ajax visibility without building it into the
> core framework.

I'm not sure that "AJAX in the core" would be a good thing for Django,
for a couple reasons:

1. It really doesn't seem like it'd do anything other than provide
buzzword compliance, and people who make development decisions based
on buzzwords are probably better left to be Some Other Framework's
Problem anyway.

2. It doesn't necessarily fit with what Django is. I've written a
couple times that the main difference between Rails and Django is that
Rails is stronger for pure "applications" and Django is stronger for
content-oriented systems. In Rails' case, an integrated AJAX library
with Ruby hooks is a good idea, because people will use it often
enough. In Django's case I'm not so sure that all sorts of AJAX
built-ins would be as generally useful; also, Django currently doesn't
get in the way of integrating AJAX, and I think that's plenty good
enough for most people's needs.

> I work with a partner who was checking out django and ruby on rails. And
> his first comment was, it doesn't look like it supports ajax. To him
> ajax is highly important not because of eye candy but because we use it
> to make applications more usable. I was able to assure him that django
> does support ajax.

If it had to be explained to him that Django can support AJAX, then I
can't help thinking that your partner doesn't actually understand
AJAX. I don't mean to insult, but it sounds like he only understands
frameworks which have a "push here for AJAX" button which hides all
the complexity (and avoids a lot of the thinking).

> This is from a pure marketing perpesctive. I guess what I'm saying is
> that I just want to make sure that it doesn't get completely written off
> because of that. Being a programmer I'm more interested in the technical
> goodness, but I'd like to see this also address marketing concerns as
> well to gain more community interest and developer adoption.

My opinion is that quality is better than quantity; Django is
attracting a number of very good developers (who understand AJAX and
the proper use thereof) and that, more than anything, will drive its
adoption.


--
"May the forces of evil become confused on the way to your house."
-- George Carlin

[Eugene Lazutkin]

Personally I am leaning towards Dojo over MochiKit.

Disclaimer: I contributed some code to Dojo so I may be biased. Upside: I
know what I am talking about in regards of Dojo.

The reasons are simple: flexible AOP-inspired event system, thought-out
widget packaging and management, versatile I/O. Additionally they have a
compression-packaging system, which allows to extract and use only needed
parts of the library.

While it is clearly possible to add missing pieces to MochiKit, size of
additions will outweigh MochiKit codebase. I think that MochiKit is small
(==mean and lean) because Bob likes it that way. I don't think he will be
eager to maintain our additions. Do we want to support them separately?

In case of Dojo, Django-specific part would be a thin layer on top of Dojo.

I think it is easier to add to Dojo from MochiKit what's missing. In fact
they are already doing it: MochiKit's creator Bob Ippolitto is in the list
of Dojo contributors, MochiKit supports dual licensing specifically to be
compatible with Dojo, some parts of Dojo is/will be lifted from MochiKit.

I think it is wise to talk to core Dojo guys (e.g., Alex Russell) about
Django Ajax and explain them what we need. They are accessible and
responsive. I am sure they will meet Django's requirements. Let's talk to
Bob too to understand his position on that.

Thanks,

Eugene


"Robert Wittams" <rob...@wittams.com> wrote in
message news:dla47v$ms4$1...@sea.gmane.org...

[Eugene Lazutkin]

I don't recall anybody proposing "AJAX in the core". Clearly Ajax should be
an optional feature. While Ajax is generating a lot of buzz lately causing
adverse reaction in non-marketing people, don't overlook it is potential to
improve usability of web sites. Ajax can be abused, but let's be pragmatic:
many people are capable to write good JavaScript code for Django apps, but I
feel that a lot of code would be the same, which goes against DRY principle.
As any indication of that you can look at requests in this list and in
Django-User for reuse of parts of Admin in their own applications. People
don't want to reinvent the wheel, they want to reuse the wheel. Same goes
for Ajax.

I think that the best idea is to start with Admin and identify what kind of
widgets would improve usability. Not low-level stuff, not technical oomps,
not visual wows, but plain simple high-level usability.

Thanks,

Eugene


"James Bennett" <ubern...@gmail.com> wrote
in message
news:21787a9f0511141116j2ec...@mail.gmail.com...

[James Bennett]

On 11/14/05, Eugene Lazutkin <eugene....@gmail.com> wrote:
> I don't recall anybody proposing "AJAX in the core".

I've seen it asked for more than once, particularly in the IRC
channel. People ask for Django to pick one AJAX library and integrate
it. To me, that's "AJAX in the core".

> While Ajax is generating a lot of buzz lately causing adverse reaction in non-marketing
> people, don't overlook it is potential to improve usability of web sites.

I don't feel that I am; I do feel that right now the vast majority of
people who are clamoring for AJAX are woefully uninformed about what
it is, what it does, and how best to use it. As a result, it ends up
not being used to enhance usability, but rather to provide lots of
flashy effects and transitions which may well have a negative impact
on overall usability -- many people are rushing to "AJAX-ify" their
applications without doing any sort of thinking about usability at
all, let alone testing to determine the impact.

Which, again, says to me that the best thing for Django is to let all
those people be Some Other Framework's Problem.

> As any indication of that you can look at requests in this list and in
> Django-User for reuse of parts of Admin in their own applications. People
> don't want to reinvent the wheel, they want to reuse the wheel. Same goes
> for Ajax.

Integrating AJAX into Django would mean settling on one toolkit to use
for AJAX effects, and building around that. This is inconsistent with
how Django handles other things, because for the most part Django is
agnostic about what tools you use; for example, if you don't like the
template system, you can use another. But this sort of flexibility
isn't easy (or, sometimes, even possible) to maintain with AJAX
toolkits.

[Stephen Rainey]

I agree with keeping the quality high. There are a few other decent
frameworks out there, but has a lot of people using it who don't really
have to think and just because of that community it turns me off of the
framework.

My example regarding the partner was a situation where the partner isn't
a developer. Which means that I get to make the decision anyway so the
example is probably not valid.

I don't like the marketing hype around ajax and I think people are out
there not using it appropriately at all. But I agree with eugene has
said in a later post about people are generally going to use it in sites
and applications and there may be some DRY concepts that can be applied.

I would not be for adding it to the core of django. I just got the
general feeling from a few people that they think all ajax is is
marketing fluff and had written it off. I'm not necessarily trying to
steer anyone in any direction. I'm completely new to django. I just
wanted to give some thoughts from someone who was out there having a
hard time evaluating different frameworks.

This is a great project and I look forward to working with it. Here are
a few things that has helped me decide to use this framework.

1. High quality and well put together
2. Intelligent development community.
3. Non competitive attitude of development community (willing to learn
from and share with other frameworks)
4. Stability
5. More sutiable for content heavy sites.
6. More freedom working with various peices of technology. Doesn't lock
in to templates, ajax libraries....

[Simon Willison]

On 14 Nov 2005, at 19:26, Eugene Lazutkin wrote:

> I think it is wise to talk to core Dojo guys (e.g., Alex Russell)
> about
> Django Ajax and explain them what we need. They are accessible and
> responsive. I am sure they will meet Django's requirements. Let's
> talk to
> Bob too to understand his position on that.

This is a very good idea. Alex Russell has expressed interest in
Django in the past. Dojo is a remarkable framework - all it needs to
take over the world is more detailed documentation.

Cheers,

Simon

[Eugene Lazutkin]

Inline.

"James Bennett" <ubern...@gmail.com> wrote
in message

news:21787a9f0511141230n59c...@mail.gmail.com...

On 11/14/05, Eugene Lazutkin
<eugene....@gmail.com> wrote:
>> I don't recall anybody proposing "AJAX in the core".

>I've seen it asked for more than once, particularly in the IRC
>channel. People ask for Django to pick one AJAX library and integrate
>it. To me, that's "AJAX in the core".

1) Let's keep different forums separate. I hope you responded to those
unnamed guys in the IRC channel.
2) My car has an integrated CD-player. Does it mean it is "in the core"? I
respectfully disagree with your definition of "in the core" as mere
"integrated".

>> While Ajax is generating a lot of buzz lately causing adverse reaction in
>> non-marketing
>> people, don't overlook it is potential to improve usability of web sites.

>I don't feel that I am; I do feel that right now the vast majority of
>people who are clamoring for AJAX are woefully uninformed about what
>it is, what it does, and how best to use it. As a result, it ends up
>not being used to enhance usability, but rather to provide lots of
>flashy effects and transitions which may well have a negative impact
>on overall usability -- many people are rushing to "AJAX-ify" their
>applications without doing any sort of thinking about usability at
>all, let alone testing to determine the impact.

This is a good argument against everything from cars to iPods. People can
(and will!!!) abuse everything. If usability requires "flashy", let's do
"flashy". I don't care how people would abuse it if it solves some real
problems. Just for the record: in my book "flashy" == "bad usability" in
99.99%.

>Which, again, says to me that the best thing for Django is to let all
>those people be Some Other Framework's Problem.

>> As any indication of that you can look at requests in this list and in
>> Django-User for reuse of parts of Admin in their own applications. People
>> don't want to reinvent the wheel, they want to reuse the wheel. Same goes
>> for Ajax.

>Integrating AJAX into Django would mean settling on one toolkit to use
>for AJAX effects, and building around that. This is inconsistent with
>how Django handles other things, because for the most part Django is
>agnostic about what tools you use; for example, if you don't like the
>template system, you can use another. But this sort of flexibility
>isn't easy (or, sometimes, even possible) to maintain with AJAX
>toolkits.

I would be happy, if Django supports many libraries, but I don't think we
can afford it. E.g., I can understand the need for 2 versions of Admin:
plain vanilla HTML, and Ajax version. If somebody wants to create 3rd
version of Admin, it is fine with me, but it looks like an overkill.

Basically all this Ajax thingy boils down to three different things:

1) Callbacks, when Django sends objects to the client. It is about the
eternal question: "how to expose server-side stuff to client side". It is
pretty much library-agnostic. One possibility is to add a special form of
Response object, which will serialize returned object in JSON (IMHO,
preferred way) or XML format. We may need to mark statically (similar to
META options), which fields of the model should not be serialized.
Additionally, it makes sense to provide some simple dynamic filtering of
object's fields before serialization.

2) A front end for manipulation of server-side objects from client-side
script. This is an opposite to callbacks: client sends objects to Django.
Django already provides unified methods for model objects (tutorials cover
most of them, details can be found here:
http://www.djangoproject.com/documentation/db_api/ and
http://www.djangoproject.com/documentation/models/). You can
create/modify/delete objects, retrieve related objects, search, and so on
using command line. It is quite simple to provide an automated support for
some safe subset of it using JSON/XML as input. This should be an optional
feature. Some field markup may be required.

3) Ready-to-use client-side components, which do GUI simple and ergonomic.
From the top of my head: client-side replication of a field group to
associate multiple subobjects with a master object; type-ahead style of
object selector; rich edit (it is partially solved by 3rd party editors);
drag'n'drop support for complex selections; menu widgets; client-side form
validation; and so on. #1 and #2 should be rich enough to provide an
adequate support for such components.

Notes:

- #1 and #2 do not require to settle for one library. They require to settle
for APIs and formats.

- People may like to have more than one implementation of #3 but I doubt it.
It should be possible for advanced users to implement their own custom GUI
using #1 and #2 and whatever Ajax library they desire to use in their own
custom application.

- Please note that "flashy effects and transitions" are not mentioned. :-)
If people want to implement "dancing words" or similar distraction, let them
do it. It is not up to us to police them, but it is up to us to support only
reasonable components.

Thanks,

Eugene

PS: In any case it makes sense to study the only existing Django Ajax
implementation: http://opensvn.csie.org/django_ajax/trunk/. You can see what
was done right, and what is wrong.

[Simon Willison]

On 14 Nov 2005, at 21:30, Eugene Lazutkin wrote:

> I would be happy, if Django supports many libraries, but I don't
> think we
> can afford it. E.g., I can understand the need for 2 versions of
> Admin:
> plain vanilla HTML, and Ajax version. If somebody wants to create 3rd
> version of Admin, it is fine with me, but it looks like an overkill.

If we use unobtrusive scripting there's no need for two versions of
the admin. Load up the regular admin, then have JS alter it a little
to add in any Ajax hooks.

There aren't that many critical use cases for Ajax in the admin. The
most obvious is live search, which Adrian/Jacob/I have been talking
about adding for the last year or so.

Cheers,

Simon

[James Bennett]

On 11/14/05, Eugene Lazutkin <eugene....@gmail.com> wrote:
> 1) Let's keep different forums separate. I hope you responded to those
> unnamed guys in the IRC channel.
> 2) My car has an integrated CD-player. Does it mean it is "in the core"? I
> respectfully disagree with your definition of "in the core" as mere
> "integrated".

By "AJAX in the core" I mean "Ships with an AJAX library and both
makes use of that library in built-in areas and exposes an interface
for the use of that library in user-defined areas". What is being
asked for here, for example, as far as AJAX effects in the admin
application, would be "AJAX in the core" by that definition.

> This is a good argument against everything from cars to iPods. People can
> (and will!!!) abuse everything. If usability requires "flashy", let's do
> "flashy". I don't care how people would abuse it if it solves some real
> problems. Just for the record: in my book "flashy" == "bad usability" in
> 99.99%.

Usability doesn't require "flashy". Usability requires "simple" and
"sensible". I also don't see any real-world "problem" which is
"solved" by AJAX, only common tasks which can sometimes be made
simpler by prudent use of AJAX.

> I would be happy, if Django supports many libraries, but I don't think we
> can afford it. E.g., I can understand the need for 2 versions of Admin:
> plain vanilla HTML, and Ajax version. If somebody wants to create 3rd
> version of Admin, it is fine with me, but it looks like an overkill.

As far as multiple versions of the admin, that'd be unnecessary if the
scripting was done properly. As far as committing exclusively to one
AJAX library, I'm not so sure; again, one of the things I really like
about Django is that you can easily "switch out" components when
there's something else you'd rather use.

> 1) Callbacks, when Django sends objects to the client. It is about the
> eternal question: "how to expose server-side stuff to client side". It is
> pretty much library-agnostic. One possibility is to add a special form of
> Response object, which will serialize returned object in JSON (IMHO,
> preferred way) or XML format. We may need to mark statically (similar to
> META options), which fields of the model should not be serialized.
> Additionally, it makes sense to provide some simple dynamic filtering of
> object's fields before serialization.

There's no need for a special form of Response object; AJAX effects
like these can already be done easily with the existing view/template
system.

> 2) It is quite simple to provide an automated support for

> some safe subset of it using JSON/XML as input. This should be an optional
> feature. Some field markup may be required.

Again, I don't see how this is something that can't be done with
Django right now.

> - #1 and #2 do not require to settle for one library. They require to settle
> for APIs and formats.

They don't require anything; once more, they can both be done right
now, as far as I can tell.

[Eugene Lazutkin]

Inline.

"James Bennett" <ubern...@gmail.com> wrote
in message

news:21787a9f0511141354u6ac...@mail.gmail.com...

On 11/14/05, Eugene Lazutkin
<eugene....@gmail.com> wrote:
>By "AJAX in the core" I mean "Ships with an AJAX library and both
>makes use of that library in built-in areas and exposes an interface
>for the use of that library in user-defined areas". What is being
>asked for here, for example, as far as AJAX effects in the admin
>application, would be "AJAX in the core" by that definition.

Okay. In this case all contrib is "in the core" as well: comments, markup,
redirects, syndication, and admin. I am fine with that. :-)

>Usability doesn't require "flashy". Usability requires "simple" and

I disagree. It really depends on the application. Visit your local nuclear
power station and ask guys, if they have "flashy" in the interface, and if
they really need it. :-) Almost all military installations will do too. ;-)

>As far as multiple versions of the admin, that'd be unnecessary if the
>scripting was done properly. As far as committing exclusively to one
>AJAX library, I'm not so sure; again, one of the things I really like
>about Django is that you can easily "switch out" components when
>there's something else you'd rather use.

It would be great, if it is possible. I am all for it. But I want to see a
proof of it first. I am not talking "generic" case, I am talking about
Admin. From my practice sometimes you have to generate some extra
information, which is not used in GUI, but required by scripts to do their
magic. It can be done in the form of declaration. Frankly, I don't see the
big difference --- some text is generated. The critical part is that content
depends on actual scripts. One more problem is going generic frequently
means GCD approach and I want to avoid that.

If you are talking about separation of declaration and implementation, it is
possible in most cases but still this separation can be misleading.

>There's no need for a special form of Response object; AJAX effects
>like these can already be done easily with the existing view/template
>system.

There is no need in Django --- everything can be done easily with existing
Python. :-) Oh! And there is no need in Python --- everything can be much
more efficiently coded in assembler. :-) We don't need no stinking
assembler, when we can program memory directly! Do you see my point? :-)

I am not talking about "can be done". I am talking about DRY and reuse of
common code across applications.

>Again, I don't see how this is something that can't be done with
>Django right now.

Again, I'm not talking about potential possibilities. See above.

>> - #1 and #2 do not require to settle for one library. They require to
>> settle
>> for APIs and formats.

>They don't require anything; once more, they can both be done right
>now, as far as I can tell.

No, they do require to declare their interfaces (API & formats). Otherwise
code reuse is not possible.

Yes, they can be implemented as soon as such interfaces are defined.

Thanks,

Eugene

[Jacob Kaplan-Moss]

Hey folks --

Man, it's fun having such smart and passionate people on this list!

I've been thinking about how Django might leverage DHTML^H^H^H^H^H
AJAX a lot recently, and I think the framework I'll sketch out below
should make a bunch of people happy.

As I see it, there are three layers necessary for doing Django-AJAX:

The first layer will be a package -- django.core.serializers, most
likely -- which exports a set of serializer/unserializer libraries.
These libraries will know how to convert Django objects to and from a
specific format; we should most likely have XML and JSON serializers
at first crack, and XML-RPC and SOAP serializers could be quite useful.

The second layer will be a set of views not unlike the
django.views.generic, except that they'll take and produce data in
one of the formats defined by the serializer and be nicely RESTful
(appropriate use of HTTP verbs and such). So you'll be able to do
something like::

urlpatterns = patterns('',
('^rpc/', include('django.conf.urls.rpc')),
)

and get slick web services for free.

The third layer will be a toolkit that abstracts those views into
Javascript. I'm afraid that picking a Javascript framework is going
to turn into a bikeshed argument, but as long as whatever we choose
can seemlessly hook into the CRUD views, we should be OK. The idea
is that in Javascript I should be able to do::

entries = rpc.get_list('blogs.entries', {'limit' : 15})

(or something) and have it Just Work™.

As I see it, these three bits should be written in a way that's
consistent with the rest of Django's philosophy: it should be very
easy to use all four together, but they should be loosely coupled so
that any layer could be used without the others. That way if you
don't like the JavaScript wrappers, you can always write your own.
Or, if you don't like the way Django does CRUD, you could roll your
own CRUD layer.

How does this sound to everyone?

Jacob

[Ian Holsman]

I was also thinking of something a whole lot more simplistic as well.
javascript-enabling some of the validators.

for example.. we have a URL field.. it would be nice if the javascript
did the same regex match that the server side did, so that people
wouldn't have to submit it in the first place, or have a calendar pop
up for a date box (it might already.. I hardly use dates)

This kind of thing would be relatively easy to 'plugin' i would
expect, and while not AJAX per se might make for a good first start.

and i agree with what Jacob said as well.

--Ian

[Simon Willison]

On 14 Nov 2005, at 22:37, Jacob Kaplan-Moss wrote:

> How does this sound to everyone?

Frickin' awesome.

How about an informal rule for this list: no more discussion of Ajax
without code to back it up :)

Cheers,

Simon

[Eugene Lazutkin]

Nah, it would take all fun from bashing. :-) What kind of code do want to
see for negative statements?


"Simon Willison" <swil...@gmail.com> wrote
in message news:8550BAAB-15E5-4DA4...@gmail.com...

[Jacob Kaplan-Moss]

On Nov 14, 2005, at 4:45 PM, Ian Holsman wrote:
> I was also thinking of something a whole lot more simplistic as well.
> javascript-enabling some of the validators.

I was planing on this being one of the web services views -- a
"validate" view that you would pass a new or modified object into
that would return any validation errors. That way the admin pages
could do live validation of all the fields (which was at one point
ticket #1 in our private trac install :)

Jacob

[Jeremy Dunck]

On 11/14/05, Eugene Lazutkin <eugene....@gmail.com> wrote:
>

> Nah, it would take all fun from bashing. :-) What kind of code do want to
> see for negative statements?
>

Failing tests.

[Eugene Lazutkin]

Looks good. It pretty much consistent with my arguments. Unfortunately the
third layer is the most complex one and it is not well defined.

"Jacob Kaplan-Moss" <ja...@jacobian.org> wrote in

message news:8FD7A7CD-B528-48DE...@jacobian.org...

Hey folks --

Man, it's fun having such smart and passionate people on this list!

I've been thinking about how Django might leverage DHTML^H^H^H^H^H
AJAX a lot recently, and I think the framework I'll sketch out below
should make a bunch of people happy.

[...cut...]

[Wilson Miner]

I think this all sounds great.

I also think Adrian's point in his post about AJAX support is very valid here. Any support for AJAX in Django should come from real needs in a real project.

As Rob pointed out in an earlier thread, once the new-admin changes are rolled in, the project to fix the edit_inline interface is going to require some basic AJAX ( e.g. drag-and-drop ordering, adding/deleting table rows, etc). And as Simon pointed out, the admin changelist is an obvious target for some basic AJAX magic (e.g. live search, live filtering)

I think the best way to go about adding AJAX to Django is to add it where it's needed and obviously useful in the admin app, build the necessary support layers along the way with a mind for abstraction, then extract those techniques to an extensible sub-framework that we can build on.

That way we have real tools from real needs, no theoretical "we might need this if" cruft, and a good repository of real world examples and use cases in the admin app. And thanks to Rob's work in the new-admin branch, all those examples and the initial functions will be prime for reuse and expansion in other apps.

On 11/14/05, Jacob Kaplan-Moss <ja...@jacobian.org > wrote:

Hey folks --

Man, it's fun having such smart and passionate people on this list!

I've been thinking about how Django might leverage DHTML^H^H^H^H^H
AJAX a lot recently, and I think the framework I'll sketch out below
should make a bunch of people happy.

As I see it, there are three layers necessary for doing Django-AJAX:

The first layer will be a package -- django.core.serializers, most
likely -- which exports a set of serializer/unserializer libraries.
These libraries will know how to convert Django objects to and from a
specific format; we should most likely have XML and JSON serializers
at first crack, and XML-RPC and SOAP serializers could be quite useful.

The second layer will be a set of views not unlike the

django.views.generic , except that they'll take and produce data in

one of the formats defined by the serializer and be nicely RESTful
(appropriate use of HTTP verbs and such).  So you'll be able to do
something like::

     urlpatterns = patterns('',
         ('^rpc/', include('django.conf.urls.rpc')),
     )

and get slick web services for free.

The third layer will be a toolkit that abstracts those views into
Javascript.  I'm afraid that picking a Javascript framework is going
to turn into a bikeshed argument, but as long as whatever we choose
can seemlessly hook into the CRUD views, we should be OK.  The idea
is that in Javascript I should be able to do::

       entries = rpc.get_list ('blogs.entries', {'limit' : 15})

[Eugene Lazutkin]

Failing usability tests?

if stuff_is_blinking then return YOU_FAIL!!! :-)


"Jeremy Dunck" <jdu...@gmail.com> wrote in
message news:2545a92c0511141501m69...@mail.gmail.com...

[Robert Wittams]

James Bennett wrote:
> On 11/14/05, Eugene Lazutkin <eugene....@gmail.com> wrote:
>
>>1) Let's keep different forums separate. I hope you responded to those
>>unnamed guys in the IRC channel.
>>2) My car has an integrated CD-player. Does it mean it is "in the core"? I
>>respectfully disagree with your definition of "in the core" as mere
>>"integrated".
>
>
> By "AJAX in the core" I mean "Ships with an AJAX library and both
> makes use of that library in built-in areas and exposes an interface
> for the use of that library in user-defined areas". What is being
> asked for here, for example, as far as AJAX effects in the admin
> application, would be "AJAX in the core" by that definition.

I've got to wonder here. If we have a need for AJAX functionality in the
admin, are we "allowed" to use it? If we have this need, are we
"allowed" to use a toolkit rather than mashing around IEs faults on our
own, and recreating countless bugs someone has already solved?

What purpose is served by a) banning AJAX from built in apps and b)
banning the bundling of any particular library?

I have no idea where you are coming from. The fact that some stuff best
performed with AJAX is wanted in the admin is clear (see ticket #13) . I
see no reason to ban particular approaches just to satisfy someones hype
paranoia. If you don't like something, you don't have to use it.

Robert

[Robert Wittams]

Wilson Miner wrote:
> I think this all sounds great.
>
> I also think Adrian's point in his post about AJAX support is very valid
> here. Any support for AJAX in Django should come from real needs in a real
> project.
>
> As Rob pointed out in an earlier thread, once the new-admin changes are
> rolled in, the project to fix the edit_inline interface is going to require

> some basic AJAX (e.g. drag-and-drop ordering, adding/deleting table rows,

> etc). And as Simon pointed out, the admin changelist is an obvious target
> for some basic AJAX magic (e.g. live search, live filtering)
>
> I think the best way to go about adding AJAX to Django is to add it where
> it's needed and obviously useful in the admin app, build the necessary
> support layers along the way with a mind for abstraction, then extract those
> techniques to an extensible sub-framework that we can build on.
>
> That way we have real tools from real needs, no theoretical "we might need
> this if" cruft, and a good repository of real world examples and use cases
> in the admin app. And thanks to Rob's work in the new-admin branch, all
> those examples and the initial functions will be prime for reuse and
> expansion in other apps.
>

I've got to say, this plan sounds like it would lead to *another*, half
featured AJAX library. Do we really want to reimplement everything? It
seems like the world needs fewer JS toolkits, not more...

Lets accept that other people are capable of waving the "extract from a
working application" magic wand, and take a look at what they found
rather than starting from scratch and recreating all their bugs.

[Eugene Lazutkin]

"Robert Wittams" <rob...@wittams.com> wrote in

message news:dlb684$h7h$1...@sea.gmane.org...

>
> I've got to say, this plan sounds like it would lead to *another*, half
> featured AJAX library. Do we really want to reimplement everything? It
> seems like the world needs fewer JS toolkits, not more...
>
> Lets accept that other people are capable of waving the "extract from a
> working application" magic wand, and take a look at what they found
> rather than starting from scratch and recreating all their bugs.

+1.

[Eugene Lazutkin]

"Robert Wittams" <rob...@wittams.com> wrote in

message news:dlb5md$fl5$1...@sea.gmane.org...

>
> I have no idea where you are coming from. The fact that some stuff best
> performed with AJAX is wanted in the admin is clear (see ticket #13) . I

Yep. Just try to reorder stuff without drag and drop. I did it using select
in some of my apps and

a) it's no fun to build it
b) users don't get it
c) major reorg (move from top to bottom) is looong for long lists
d) wastes a lot of vertical space to prevent scrolling

Let's admit that "no Ajax" means more limitations of GUI.

Thanks,

Eugene

[hugo]

Hi,

>I've been thinking about how Django might leverage DHTML^H^H^H^H^H
>AJAX a lot recently, and I think the framework I'll sketch out below
>should make a bunch of people happy.

my main gripe with your ideas is that you throw RPC style stuff
(XML-RPC and SOAP) in with the REST stuff. That just doesn't compute -
both are far too different in philosophy and structure to be really
compatible with each other. I think we need two different things for
webservices - support for RPC style APIs and support for REST style
APIs. And both can be mapped into the ideas of Django quite nicely, I
think.

I have written down my ideas in my trac:

https://simon.bofh.ms/cgi-bin/trac-django-projects.cgi/wiki/WebservicesForDjango

I first thought about posting it here, but it got longer and longer and
so I thought it's better to put it into one place and only post the
link ;-)

So my main problem is, RPC needs just one view func with registerable
callbacks (and a set of predefined callbacks for models) that sits on
one URI and dispatches on method name, while REST style APIs need a way
to map objects into the url space (so the dispatch is done by the
standard url pattern handling of Django).

Of course everything on that page is just an idea - and it's written
over a short period of time, so it's neither complete nor perfect. It
might even be that some things don't make sense at all (like how the
url space for REST APIs is structured - that's just an idea how
something might look like, as that gives a more concrete thing to talk
about).

bye, Georg

[hugo]

>Lets accept that other people are capable of waving the "extract from a
>working application" magic wand, and take a look at what they found
>rather than starting from scratch and recreating all their bugs.

+1

bye, Georg

[Eugene Lazutkin]

Lets make it happen. Who wants to be involved? What form of interaction is
preferred: IRC, e-mails, anything else?

AFAIK, Dojo is built on bones of many projects and directly sponsored by
real life applications like JotSpot (http://www.jot.com). Guys from WebWork
(http://www.opensymphony.com/webwork/) and Yahoo/Flickr (I think you don't
need URL for that :-) ) donated some code as well. Wicket
(http://wicket.sourceforge.net/) is using it. We are not talking about
academic ideas here. Maybe Alex will share his take on the whole
client-server issue.

Regarding documentation --- what you said is true. A lot of cool stuff
should be excavated from sources. Recently they started to spend more time
on documentation. It is going to be a major focus of one of upcoming
milestones.

Thanks,

Eugene


"Simon Willison" <swil...@gmail.com> wrote
in message news:3B9196E3-1420-4E15...@gmail.com...

[Wilson Miner]

That's a red herring. Nothing in that plan precludes using an existing AJAX library. If Dojo or Mochikit or whatever else out there does what we need it to for the admin, then the end result will be that we "integrate" that particular AJAX framework and build in some access points to make it easy for anyone to do the same in their app.

"Only do what we need" doesn't automatically mean "only use what we make from scratch".

[Robert Wittams]

Wilson Miner wrote:
> That's a red herring. Nothing in that plan precludes using an existing AJAX
> library. If Dojo or Mochikit or whatever else out there does what we need it
> to for the admin, then the end result will be that we "integrate" that
> particular AJAX framework and build in some access points to make it easy
> for anyone to do the same in their app.
>
> "Only do what we need" doesn't automatically mean "only use what we make
> from scratch".

Thats fine. I (and seemingly others) misinterpreted your post as "only
add the absolute minimum necessary for the bundled apps", which would
necessitate either starting from scratch or pulling apart existing JS
libraries.

If you just mean the support stuff on the python side - I agree that
there isn't all that much to provide there.

Robert

[Wilson]

How about starting a wiki page (here's a good place to start:
http://code.djangoproject.com/wiki/AJAX ) as a place to collect
everyone's input on requirements, suggestions and examples.

There was some discussion in IRC today that it would be useful to
collect examples of how people are already using AJAX (with or without
Django) as a starting point for deciding how best to facilitate similar
techniques in the framework. If there's a cowpath to be paved, let's
find it.

[Wilson]

I think Dojo looks great. Their rich text editor demo looks like it's
designed to plug in to the Django admin :)

Before we get too far into the "which Ajax framework should Django use"
discussion, I think it's important to lay the groundwork first.

It seems to me that the goal of any Ajax support in Django should be to
make it easier to integrate with any Ajax framework. If there is a
default Ajax framework coupled with Django (it may just be a matter of
which one is bundled and documented) it should be at least as easy to
use another framework (or frameworks) as it is to use other template
systems. Loose coupling, sensible defaults, etc.

So the real productive question at this stage isn't "which Ajax
framework", it's "how can Django make Ajax easier for app developers."
At that point, what framework you use or whether you roll your own is
up to you.

[Eugene Lazutkin]

Inline.

"Wilson" <wmi...@gmail.com> wrote in message
news:1132099139.5...@f14g2000cwb.googlegroups.com...

>
> I think Dojo looks great. Their rich text editor demo looks like it's
> designed to plug in to the Django admin :)

:) I am thinking to switch from TinyMCE to Dojo Rich Editor on my web sites.

> Before we get too far into the "which Ajax framework should Django use"
> discussion, I think it's important to lay the groundwork first.
>
> It seems to me that the goal of any Ajax support in Django should be to
> make it easier to integrate with any Ajax framework. If there is a
> default Ajax framework coupled with Django (it may just be a matter of
> which one is bundled and documented) it should be at least as easy to
> use another framework (or frameworks) as it is to use other template
> systems. Loose coupling, sensible defaults, etc.

+1. I would prefer Ajax-related additions to be a separate subsystem
(application, middleware, bunch of related classes, and so on), which should
be used only when app developer says so.

> So the real productive question at this stage isn't "which Ajax
> framework", it's "how can Django make Ajax easier for app developers."

Hear! Hear!

> At that point, what framework you use or whether you roll your own is
> up to you.

Right. I don't think that Django should have "AJAX in the core". Simple
no-nonsense HTML-only web side should be possible. I do think Admin would
gain a lot in terms of usability from going Ajax. I do think that
reinventing wheels is not necessary, when there are true and tried
libraries, which are supported and enhanced without our direct involvement.

Thanks,

Eugene

[Eugene Lazutkin]

Today I discovered that I missed very interesting Django/Ajax-related
discussion on #django. Specifically MochiKit-Dojo comparison was discussed.
I know it is hard to talk about Dojo, when documentation is lacking. Let me
present my take, based on some experience with both toolkits from
prospective of simple user. I'll try to be objective.

MochiKit.

1st and foremost: it has the best documentation of all Ajax toolkits I saw
(see http://www.mochikit.com/doc/html/MochiKit/index.html). Test collection
is comprehensive as well.

MochiKit builds on Python whenever it is possible. You can see Python in
many objects provided by the toolkit. E.g., Async module (it wraps XHR) is
built after Twisted and provides Deferred, which is very nice. Out of the
box it provides GET verb. You can do POST too, but it requires some manual
work: basically you should build post data by yourself and work directly
with XHR. Fear not, there are some useful string manipulation functions,
which can help. Iter module is derived from Python as well.

There is DOM creation API, which is nicely done too. The only potential
problem is performance, which can be critical, if you want to create big DOM
tree from your code. It is not a limitation of MochiKit, it's a limitation
of approach --- unfortunately modern browsers work faster with mundane
innerHTML. At least this is my experience.

MochiKit provides a comprehensive set of string formatters, which is very
handy too.

Despite some people's opinion, MochiKit has exactly 1 (one) visual effect:
rounded corners. IIRC it was donated and somehow got included. The lack of
visual effects is intentional: MochiKit focuses on data processing and DOM
creation. Basically it follows prototype.js strategy, which doesn't provide
visual effects at all (0/zero), relying on independent libraries like
script.aculo.us or OpenRico for visual glitter. I don't know, why Bob
decided to include rounded corners, but they are here, they work, they are
useful as part of visual design. MochiKit is fairly neutral, it may be used
with other visual effect libraries.

Another thing, which I like in MochiKit, is consistent JSON support, which
simplifies a lot of things. Basically JSON notation is a simplified
declarative JavaScript code, which can be "decoded" using
eval("("+json_data+")") (see http://www.crockford.com/JSON/index.html). It
makes programmer's life easier. Try to do it with XML. ;-) MochiKit makes it
easy to "read" JSON from server, and to serialize your JavaScript object to
JSON for server processing. I really dig it.

All modules are in separate files, so you can select what functionality you
need.

Support is done by Bob Ippolito --- the creator of MochiKit. Having a sole
developer/maintainer makes MochiKit more consistent. It's easy to write him
a letter or ask him in MochiKit support group. You can find him there, or in
Dojo group helping people, or in TurboGears answering questions --- yes,
some other major web framework has already selected MochiKit as part of its
stack.

Dojo.

Dojo covers much more than any other Ajax/JavaScript toolkit I know of.
Obviously size of Dojo's source code is large than other major toolkits. In
order to overcome this problem Dojo has elaborate modular compression /
packaging / linking / loading system. It works like that:

1) Every module declares its dependencies and what it provides
(import/export declarations).
2) It is possible to declare your dependency on a single file, all files of
a directory, or any mix of those.
3) When you load dojo.js, it loads very small bootstrap code, which can load
other modules dynamically.

This process is fully automated. If requested module has some unresolved
references, loader will continue the process. Once it is loaded, it will be
reused for subsequent requests. It means that if your web page doesn't use
some functions, they are not going to be loaded. Simple stuff loads almost
instantly.

Obvious downside of dynamic loading is a potential to produce a lot of small
requests, which can be a problem due to network delays. In order to combat
it, Dojo has a packaging system. It is trivial to specify, which modules are
required, and build a custom "profile" of Dojo. In this case your custom
dojo.js will include a loader _and_ requested modules. 5 profiles are
pre-built: Minimal, Browser I/O, AJAX, Event, Event + I/O, and The Kitchen
Sink. AJAX profile is about the size of MochiKit.

If you use modules, which are not included in the profile, they will be
loaded dynamically. So it doesn't really matter, which profile to use. The
difference is in loading time, which depends on application.

For one of my projects I created a custom build, which included just the
stuff I need. I discarded the rest to conserve memory, because I know that
the rest is never called. Basically all unused components can be easily
removed --- Dojo has true plug-and-play modular structure.

Optional source code compression can be used to reduce size as well.
MochiKit uses Dojo compression/packaging system.

dojo.io.bind (Dojo's XHR wrapper) is more elaborate than MochiKit's offering
communication-wise. It handles GET and POST using simple dicts to specify
what you want to send and how to handle it (see
http://dojotoolkit.org/docs/intro_to_dojo_io.html for code examples). It is
possible to specify an existing form, instead of actual content. In this
case the form (its fields) will be submitted using dojo.io.bind. I like this
feature because it allows to embed Ajax in existing apps without major
surgery. Multipart POST patch (written by yours truly) is pending. It can
handle files formed directly on the client using JavaScript as well as
regular data.

Another remarkable part of Dojo is the widget system. It allows to separate
your widget into (up to) three files: JavaScript module, optional HTML file,
and optional CSS file. HTML file and CSS files are templates, which are used
by widget system --- very Django-istic. In your template you define points
of attachment. No JavaScript code goes there.

It is very easy to use widgets. The simplest way is to mark up your widgets
in HTML file and Dojo will instantiate them for you. There are several ways
to do it. I prefer this way:

<div id="AreYouReallySureDialog" class="dojo-dialog">...</div>

In this case "dialog" widget is going to be instantiated. It can use
existing content, it may create its content dynamically, or Dojo may use
templates for substitution. BTW, widgets may contain another widgets. It is
possible to build a super-widget out of simple widgets.

From my experience it is easy to create completely self-contained widgets
with clean mark up. The best thing is you don't need to write spaghetti
code, and hook into events in different places. Finally I was able to make
HTML files, which are fully dynamic but don't have a single line of
JavaScript code or extensive mark up.

Dojo event system is another marvel. It's modeled after AOP concepts and
allows you to do a lot of nifty stuff. E.g., it is possible to hook your
event to ... calls to some object's method. In canonical AOP fashion you can
specify when do you want your hook to be called --- before, after, around.
It allows to order your pre- and post- processing in a flexible way.
Obviously all browser events are supported seamlessly. Multicast events are
supported seamlessly. Custom events with parameters can be created. All
technical event-related code are completely gone from application --- you
operate in terms of "connect" and "disconnect". See details here:
http://dojotoolkit.org/docs/dojo_event_system.html

I spent a paragraph writing about events because they are extremely
important for widget writing. Who tried to do it will understand me. Dojo
event system makes your code clean. From my experience with Dojo events
almost all widget code handles business logic instead of mundane plumbing.

Custom events are useful for messaging between components. It is a provision
for communication between different widgets. It makes it possible to make
widgets aware of neighbors.

Dojo supports a lot of other stuff: drag-and-drop, curve-based animation,
cross browser transparency, a lot of useful utilities, and so on.
Additionally Dojo has a library of widgets. I counted about 35 of them. Not
all of them are ready for prime time. I use Dialog a lot, played with Menu,
ContextMenu, Tree, InlineEditBox, RichText, SlideShow, SplitPane,
ColorPicker, and Time/Date pickers. They mostly work. :-) In any case
working with Dojo I didn't have a need for other libraries. It was a stark
contrast with, say, prototype.js --- I had to augment it with
script.aculo.us and behavior.js plus wrote quite a few of my own support
code.

Other notable things: some work was done on SVG. I didn't try it but
apparently you can use Dojo in SVG. It sounds interesting. Finally we will
have some nice charts, which will be a killer app for IT. In another
interesting development Dojo people established (semi-)official contacts
with Mozilla people. I hope it will improve both Dojo and Firefox.

Now lets talk what's bad about Dojo:

1) Documentation is not even close to MochiKit's one. Mainstream stuff may
be documented but I dig into source code a lot for less popular features,
like MD5 calculation. Fortunately a lot of simple test files are available
practically for all modules.
2) Not everything is properly tested. It means that when I release my code,
I have to test it with all available browsers to make sure it works as
intended. When it doesn't work, the culprit can be my code, but it can be a
bug in Dojo's code as easily.
3) Dojo supports XML and JavaScript but doesn't provide direct support for
JSON --- I have to use eval("("+json_data+")") "conversion, when I need it.

Dojo is improving on daily basis. Its support group is very much alive and
questions are answered by core developers in timely manner. SVN and Trac are
available, documentation is in ReST files --- the setup reminds Django a
lot. But they have more core developers. :-) It is nice to do open source
being sponsored. :-)

MochiKit and Dojo.

MochiKit and Dojo are friendly to each other. It is possible to use them
both. AFAIK, there is a pact between them, which allows MochiKit to use Dojo
parts, and Dojo can lift stuff from MochiKit at will --- Bob is all for
that.

Sometimes people talk about merging these two frameworks. Bob Ippolito said
that he doesn't mind eventual merging, but in the near term it doesn't look
feasible: while MochiKit has fewer coverage feature-wise, it is much more
mature and more stable. At the moment development focus of both toolkits is
different too.

I hope it helps. I will be glad to answer questions. If you want to contact
core developers, Bob Ippolito is responsible for MochiKit, while Dojo can be
represented by Alex Russell. I can help to facilitate contacts with Bob and
Dojo people, if it is needed.

Thanks for reading my rambling post,

Eugene

[Wilson]

I pulled some of the info from this thread and started a Wiki page:

http://code.djangoproject.com/wiki/AJAX

Please commence filling in the blanks and correcting my errors and
misrepresentations :)