Session/cookie based messages (#4604)

46 views
Skip to first unread message

Tobias

unread,
Sep 10, 2009, 7:00:45 PM9/10/09
to Django developers
In the spirit of Russell's No! Bad Pony! talk at DjangoCon I'd like to
start a conversation about cleaning up session messages enough to get
them into core or contrib. Alex suggested that the ticket (http://
code.djangoproject.com/ticket/4604) was not the right place for the
conversation so I'm starting one here.

There are a number of usable solutions out there but django-notify is
the most complete, polished one that I know of. I just contributed a
patch for combo/fallback storage, a version of which SmileyChris
integrated into trunk (of django-notify, not django). It's a solid
product with well-commented code and a good test suite.

What needs to be done to get this or something like it into contrib?

Tobias

Hanne Moa

unread,
Sep 19, 2009, 11:59:59 AM9/19/09
to django-d...@googlegroups.com
On Fri, Sep 11, 2009 at 01:00, Tobias <tobias....@gmail.com> wrote:
> There are a number of usable solutions out there but django-notify is
> the most complete, polished one that I know of.

I've just tested django_notify, so far it seems plenty good enough to
replace all my pass-on-message hacks (not to mention being much more
elegant than any of 'em) expect for not being able to set a message
php-style in the GET, this way:

http://example.com/?msg=Shiver+me+timbers

Whether being able to do that at all is desirable is another question
entirely ;)

Big bonus in my view: it's completely independent of the
django.contrib.auth-system.

> What needs to be done to get this or something like it into contrib?

Yes, this inquiring mind also wants to know.


HM

Russell Keith-Magee

unread,
Sep 20, 2009, 10:24:15 AM9/20/09
to django-d...@googlegroups.com

Since you were evidently at my DjangoCon presentation, you know that I
(and the core in general) have a pretty high level of reticence to
adding things to contrib - especially when the package can live quite
happily outside of core. So far, everything I have seen about
django-notify/djangoflash makes it seem like a completely standalone
package that will gain nothing from being part of core, other than
increasing the size of the tarball, and increasing the maintenance
load of the core team. One of the questions that needs to be answered
is "why should this be shipped with Django?"

Packages also need to meet the basic definition of django.contrib in
order to be accepted. Part of this definition is that it must be the
defacto standard implementation. In order to be the defacto standard
implementation of anything, it needs to be relatively stable and
feature complete. When you make statements like "I just contributed a
patch for combo/fallback storage", it leads me to believe that this
may not be the case - at least, not yet.

You also mention that there are a number of other implementations, but
you haven't really given a compelling survey or analysis of the
alternatives - you've just blessed one in particular. Why?

Another piece of the puzzle that is missing from my perspective is any
discussion of how session-based messaging interacts with the existing
contrib.auth messaging framework. IMHO, Django shouldn't ship with two
messaging APIs, so what is the integration point? Do we need a
messaging backend API? I haven't thought about this problem anywhere
near enough to offer a proposal, but I haven't seen any discussion of
this point either. A good solution for this problem would go a long
way to greasing the wheels.

Yours,
Russ Magee %-)

Chris Beaven

unread,
Sep 20, 2009, 6:13:32 PM9/20/09
to Django developers
> One of the questions that needs to be answered
> is "why should [a session based notification system] be shipped with Django?"

> Another piece of the puzzle that is missing from my perspective is any
> discussion of how session-based messaging interacts with the existing
> contrib.auth messaging framework. IMHO, Django shouldn't ship with two
> messaging APIs, so what is the integration point?

Having some kind of defacto cross-request notification system makes
sense; it's a very common usage pattern.
Attaching these kind of messages to a User instance is wrong: there is
not an enforced one to one correlation between a user and a session
[1], and you can't notify anonymous sessions.

The current contrib.auth messaging framework can't be removed or
replaced while keeping full backwards compatibility. A basic user
based messaging system still has it's uses, but it is not the best fit
for session notifications.
Side note: a pet peeve of mine is that contrib.auth messages are not
lazily loaded, which means you get a db hit every request [when using
a RequestContext] and lose your messages whether you use the messages
context var or not, but this is a side issue and should probably be a
ticket of it's own.

For some background, I initially started 4604 much more tightly
coupled with core, and one which backwards-compatibly worked with the
current user-based messaging system. Malcolm specifically requested
that this should be written as a stand-alone contrib app rather than
part of core, and that it should have minimal impact on existing code.

Russell Keith-Magee

unread,
Sep 20, 2009, 8:05:24 PM9/20/09
to django-d...@googlegroups.com

Sorry - perhaps I need to be more clear on my intent here. I'm
convinced of the importance of session-based messages - just not of
when and how a particular implementation should be added to trunk.

I agree that they are a useful idea. I agree with Malcolm that session
based messages should be a standalone app. I don't think they should
be tightly bound to core.

What I'm talking about is an orthogonal set of modifications that
would allow for _any_ messaging system (database backed, session
backed, or magic pony backed) to be used. This doesn't couple a
particular implementation of session-based messages to the core, but
it would allow for end-users to choose a session-based message
framework if they found one that was appropriate to their needs. In
the fullness of time, we may end up adding django-notify or similar to
contrib, but if we add the plugable backend we can defer that decision
until much later when implementations have stabilized and a clear
implementation winner has emerged.

If such an abstraction API isn't possible (and the differences between
the capabilities of the two certainly makes this likely), then that
makes it even more important that we choose the right implementation
to add to contrib. As it stands, I'm aware of at _least_ two
implementations (django-notify and djangoflash), but I have no real
feel for the implementation maturity or general level of community
acceptance of either. I'm not going to advocate adding anything to
contrib until I can be convinced that one particular implementation
really is the defacto standard implementation of the pattern.

Yours,
Russ Magee %-)

Chris Beaven

unread,
Sep 20, 2009, 11:30:17 PM9/20/09
to Django developers


On Sep 21, 12:05 pm, Russell Keith-Magee <freakboy3...@gmail.com>
wrote:
Thanks for your clarifications.

> What I'm talking about is an orthogonal set of modifications that
> would allow for _any_ messaging system (database backed, session
> backed, or magic pony backed) to be used. This doesn't couple a
> particular implementation of session-based messages to the core, but
> it would allow for end-users to choose a session-based message
> framework if they found one that was appropriate to their needs. In
> the fullness of time, we may end up adding django-notify or similar to
> contrib, but if we add the plugable backend we can defer that decision
> until much later when implementations have stabilized and a clear
> implementation winner has emerged.

django-notify is built on a pluggable backend architecture.
It comes with packaged with three backends, and custom backends are
also supported.

> As it stands, I'm aware of at _least_ two
> implementations (django-notify and djangoflash),

I agree, the decision is important to get right.

To clarify, django flash is actually more than just a session
notification backend, it introduces a whole new temporary (but cross-
request) variable scope.
Leah has a messaging implementation which uses django-flash:
http://github.com/leah/django-flash-status

So the question changes into, "are we after just a notification system
or do we want to have a more open (but arbitrary) system of a
temporary cross-request variable scope?"

Alex Gaynor

unread,
Sep 20, 2009, 11:33:58 PM9/20/09
to django-d...@googlegroups.com
Forgive me, but isn't a "temporary cross-request variable scope" just
sessions or cookies?

Alex

> >
>



--
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me

Tobias McNulty

unread,
Sep 21, 2009, 8:30:42 PM9/21/09
to django-d...@googlegroups.com
On Sun, Sep 20, 2009 at 6:13 PM, Chris Beaven <smile...@gmail.com> wrote:
Having some kind of defacto cross-request notification system makes
sense; it's a very common usage pattern.
Attaching these kind of messages to a User instance is wrong: there is
not an enforced one to one correlation between a user and a session
[1], and you can't notify anonymous sessions.

Exactly.  What makes this such a pressing issue for me is that the current user.message_set implementation directly encourages the (bad) habit of attaching messages to a user, without offering a more extensible alternative.  There are many situations in which the message_set implementation breaks down and reasons why a more robust standard is needed, and needs to be blessed by the core developers.  To mention a few:

1) it's not possible to create messages for anonymous users
2) if the same Django user is logged in twice on different computers simultaneously, they see each others' messages
3) as Chris mentioned, messages may get wiped even if they're not actually displayed to the user
4) high-load sites want to avoid the unneeded database or cache usage
5) since the built-in implementation is broken and no standard exists, reusable apps don't know what system to use and hence cannot rely on providing session feedback
6) it's debatably a more common problem than anything else in contrib: A site that doesn't allow users to login (hence no contrib.auth) may still need session-specific feedback.

Implementation-wise, it's also a very simple problem.  Sessions and cookies will cover 95% of the use cases at hand.  Both are necessary for a couple reasons: cookies may not be large enough to store all messages, and the session may be more expensive to use (and hence should be avoided if possible).  For the remaining 5%, it's incredibly easy to write a custom backend in at least one of the messaging/notification options out there.

In my mind what we need now is more eyes on the code and more feedback about the pros and cons of each option out there.  Perhaps something as simple as a wiki-based table would do.  I've previously summed up the criteria for what I think would make a solid notification engine and only found one app that did it, but that may have changed and there are probably options out there that I don't know about.  My post and the feedback I received can be found here:


On a side note, I talked with Simon about this at the sprints and he made the point that we also need a genuine, cryptography expert-approved cookie-signing utility in the core, which this messaging/notification app would ideally use.  The issue, as I understand it, is that we want to be certain pickling misc. objects and sticking them in a cookie is actually secure, and not rely on (with all due respect) [some random cookie signing implementation].

Tobias

Tobias McNulty

unread,
Sep 21, 2009, 9:08:15 PM9/21/09
to django-d...@googlegroups.com
On Sun, Sep 20, 2009 at 10:24 AM, Russell Keith-Magee <freakb...@gmail.com> wrote:
You also mention that there are a number of other implementations, but
you haven't really given a compelling survey or analysis of the
alternatives - you've just blessed one in particular. Why?

I started a wiki page comparing some of the different options out there:


Feel free to update with (your messaging backend here).

Tobias

veena

unread,
Sep 22, 2009, 11:17:34 AM9/22/09
to Django developers
Hi Tobias,
good idea with start a wiki page.

I'm not sure if we don't forget one issue.

How about same session (or same cookie sent by browser) with
simultaneously opened windows of one browser? Then message could
appear in different window not the right one where we invoke the
event. Is it a problem? Is only possibility to get of this issue that
flash app should add a query parameter into redirected url which would
identify the right window?

Vaclav




On Sep 22, 3:08 am, Tobias McNulty <tob...@caktusgroup.com> wrote:
> On Sun, Sep 20, 2009 at 10:24 AM, Russell Keith-Magee <
>

Tobias McNulty

unread,
Sep 22, 2009, 1:55:17 PM9/22/09
to django-d...@googlegroups.com
Vaclav,

I think this is less of an issue, because you'd have to switch to another tab and perform a second operation that generated feedback in the ~200 millisecond window of time between clicking a link and the new page loading.

If you need to support this functionality, you could write a storage to put the message directly in the URL query string, either by adding it to the request Location header for HttpResponseRedirect objects or by parsing the response output and sticking in the extra parameter for all URLs on the page, but this is obviously pretty ugly and in my mind not worth the effort.

Tobias

Russell Keith-Magee

unread,
Sep 22, 2009, 9:51:11 PM9/22/09
to django-d...@googlegroups.com
On Wed, Sep 23, 2009 at 1:55 AM, Tobias McNulty <tob...@caktusgroup.com> wrote:
> Vaclav,
> I think this is less of an issue, because you'd have to switch to another
> tab and perform a second operation that generated feedback in the ~200
> millisecond window of time between clicking a link and the new page loading.

Ah... the wonderful American perspective of the internet. :-)

As a proud resident of the antipodes, allow me to assure you that
200ms is not a representative sample of the time required to load a
page for those of us that don't live in the continental USA.

Let's consider Rackspace as a representative sample of a US based
server with plenty of bandwidth. As the crow flies, I'm about 17000 km
from a server in Texas. By simple laws of physics, it takes 60ms for
the signal to get from me to the server, and 60ms for the signal to
get back. That's 120ms, and I'm not even taking into account:

* The time spent going through the 14 routing points between me and
Rackspace's servers

* The fact that the cable between me and Texas doesn't follow the
same route as a crow

In reality, I get a ping time closer to 300 ms. And that's to a
high-end data center under ideal conditions - it can be much larger if
I'm dealing with low end providers.

Now add the time required to actually compute a response.

To put this in practical perspective - during DjangoCon, I heard lots
of people complaining about the speed of the hotel wireless network. I
didn't notice a serious change from what I use at home every single
day.

In this particular case, I think you're correct - I'm not especially
concerned about parallel cookie requests. However, there are plenty of
arguments that get made on the assumption that the internet is a form
of instant communication (<cough> database connection lag </cough>). I
want to drive home the point that Earth is a very large place, and no
matter where you are located, most of the world's population isn't
anywhere near you - and no amount of technology will fix problems
caused by the laws of physics.

Yours,
Russ Magee %-)

Tobias McNulty

unread,
Sep 22, 2009, 10:49:14 PM9/22/09
to django-d...@googlegroups.com
On Tue, Sep 22, 2009 at 9:51 PM, Russell Keith-Magee <freakb...@gmail.com> wrote:
In reality, I get a ping time closer to 300 ms. And that's to a
high-end data center under ideal conditions - it can be much larger if
I'm dealing with low end providers.
 
What?? 200 ms is the average quoted by Mr. Sproutcore himself!

No, in all seriousness, my apologies for making such a broad generalization about packet latency.  I could/should have said 500 ms or even a second; the argument and corresponding solution, if needed, remain the same.

Anyways..we digress.  I am marking this a non-issue in my own head - please pipe up if there's a case to be made.

Tobias

David Cramer

unread,
Sep 23, 2009, 7:17:19 PM9/23/09
to Django developers
I'm a bit late in here, and it seems we reinvented a wheel as well
(even tho we did this about a year ago), but recently just OS'd our
simply notices system [1]. I'm also +1 for including something like
this in trunk rather than using the current user messages. I had a
brief look at django_notify and its a similar approach, but it seems
much more configurable than what we wrote.

[1] http://github.com/dcramer/django-notices

On Sep 22, 9:49 pm, Tobias McNulty <tob...@caktusgroup.com> wrote:
> On Tue, Sep 22, 2009 at 9:51 PM, Russell Keith-Magee <freakboy3...@gmail.com

Tobias McNulty

unread,
Oct 10, 2009, 1:19:09 PM10/10/09
to django-d...@googlegroups.com
In hopes of jump starting this conversation once again, I made some
significant changes to the wiki page that try to sum up the state of
the conversation so far in a better way:

http://code.djangoproject.com/wiki/SessionMessages

Things that still need to be discussed/done:

* Coming to consensus on what 3rd party app we actually choose to
extend/modify to fit into Django

* What to do with the existing user message API (see
http://code.djangoproject.com/wiki/SessionMessages#IntegratingwiththeExistingAPI)

* Review/add to the TODO list on the wiki
(http://code.djangoproject.com/wiki/SessionMessages#TODOOnceaSolutionisChosen)

Cheers,
Tobias
--
Tobias McNulty
Caktus Consulting Group, LLC
P.O. Box 1454
Carrboro, NC 27510
(919) 951-0052
http://www.caktusgroup.com

Tobias McNulty

unread,
Oct 10, 2009, 1:53:04 PM10/10/09
to django-d...@googlegroups.com
On Sat, Oct 10, 2009 at 1:19 PM, Tobias McNulty <tob...@caktusgroup.com> wrote:
> Things that still need to be discussed/done:
>
> * Coming to consensus on what 3rd party app we actually choose to
> extend/modify to fit into Django
>
> * What to do with the existing user message API (see
> http://code.djangoproject.com/wiki/SessionMessages#IntegratingwiththeExistingAPI)
>
> * Review/add to the TODO list on the wiki
> (http://code.djangoproject.com/wiki/SessionMessages#TODOOnceaSolutionisChosen)

I should have also added:

* Coming to consensus on a de facto standard API suitable for
inclusion in Django

I originally put it on the TODO list, but in retrospect the discussion
needn't wait till we pick a solution.

As a practical starting point, I copied the API for django-notify to
the wiki page. I like the general form of that API, but I'd be
slightly more happy with something like:

from django.contrib import messages

request.messages.add('message', messages.INFO)
# or
request.messages.add('message', classes=(messages.WARNING,))
# or
request.messages.error('message')

A la python logging, I think endorsing a specific set of message
classes or tags (but still allowing them to be extended) is A Good
Thing because it helps reusable apps provide more customized feedback
in a standard way.

David Cramer

unread,
Oct 10, 2009, 2:05:59 PM10/10/09
to django-d...@googlegroups.com
The proposal per your email is more or less how django-notices works.

Sent from my iPhone

On Oct 10, 2009, at 12:53 PM, Tobias McNulty <tob...@caktusgroup.com>

Tobias McNulty

unread,
Oct 10, 2009, 2:23:58 PM10/10/09
to django-d...@googlegroups.com
On Sat, Oct 10, 2009 at 2:05 PM, David Cramer <dcr...@gmail.com> wrote:
> The proposal per your email is more or less how django-notices works.

For comparison I added django-notices to the Available Options:

http://code.djangoproject.com/wiki/SessionMessages#AvailableOptions

Hanne Moa

unread,
Oct 10, 2009, 6:44:26 PM10/10/09
to django-d...@googlegroups.com
On Sat, Oct 10, 2009 at 19:53, Tobias McNulty <tob...@caktusgroup.com> wrote:
> I should have also added:
>
> * Coming to consensus on a de facto standard API suitable for
> inclusion in Django

The suggested API: how hard will it be to use different message-types
in addition to the syslog-like ones? I use django-notify now and the
types I use is nothing (for info), "error", and "help" (for
help/tips). I've also made django-notify optional in my reusables,
with a dummy-object if it is not installed. Locking them down with
class.attributes will find typos of course, but right now I don't have
to import anything just to use session-messages, I just do a
hasattr(request, 'notifications') .


HM

Tobias McNulty

unread,
Oct 10, 2009, 6:58:45 PM10/10/09
to django-d...@googlegroups.com
On Sat, Oct 10, 2009 at 6:44 PM, Hanne Moa <hann...@gmail.com> wrote:
> The suggested API: how hard will it be to use different message-types
> in addition to the syslog-like ones?

I think it should definitely be easy, if not actively encouraged. IMO
those constants should just be strings, or integers which can easily
be mapped to strings (like in Python logging). This would give you
the flexibility of passing in anything you want. django-notices does
it the latter way:

http://github.com/dcramer/django-notices/blob/master/django_notices/constants.py

I clarified that criterion in the wiki:

http://code.djangoproject.com/wiki/SessionMessages#Criteria

veena

unread,
Oct 10, 2009, 7:03:50 PM10/10/09
to Django developers
Today I was on very bad wifi connection. Constantly dropped. 20
seconds to load a page.
I save in admin in two tabs and got a notice in one tab from the other
tab.

But I agree, I defer this "bug" for the first release od django
messaging. I think django isn't now in right mood to add there some
functionality like adding of query parameters to response object by
some application. Maybe in future.

Vaclav


On Sep 23, 4:49 am, Tobias McNulty <tob...@caktusgroup.com> wrote:
> On Tue, Sep 22, 2009 at 9:51 PM, Russell Keith-Magee <freakboy3...@gmail.com

Tobias McNulty

unread,
Oct 10, 2009, 9:00:07 PM10/10/09
to django-d...@googlegroups.com
On Sat, Oct 10, 2009 at 7:03 PM, veena <vst...@gmail.com> wrote:
> Today I was on very bad wifi connection. Constantly dropped. 20
> seconds to load a page.
> I save in admin in two tabs and got a notice in one tab from the other
> tab.
>
> But I agree, I defer this "bug" for the first release od django
> messaging. I think django isn't now in right mood to add there some
> functionality like adding of query parameters to response object by
> some application. Maybe in future.

AFAIK this will become a non-issue with the advent of HTML5, as each
window/tab will have its own session.

Tobias

David Cramer

unread,
Oct 12, 2009, 10:21:48 AM10/12/09
to django-d...@googlegroups.com
I also don't think this problem is being addressed here. Yes you could
pass messages to the context, but you would lose the ability to
retrieve those variably. I believe storing it in the existing session
is the best appoach for django's builtin support.

Sent from my iPhone

On Oct 10, 2009, at 8:00 PM, Tobias McNulty <tob...@caktusgroup.com>
wrote:

>

Tobias McNulty

unread,
Oct 12, 2009, 10:39:24 AM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 10:21 AM, David Cramer <dcr...@gmail.com> wrote:
> I also don't think this problem is being addressed here. Yes you could
> pass messages to the context, but you would lose the ability to
> retrieve those variably. I believe storing it in the existing session
> is the best appoach for django's builtin support.

I'm not exactly sure what problem you see as not being addressed, and
I don't think you were arguing against this, but it has been noted
elsewhere that it's more desirable to store the messages directly in a
cookie, if possible, to avoid the extra database or cache hit.

It seems well worth it to me and there are solutions out there that
try to store the messages in a cookie and then fall back to the
session for longer (>4kb) messages.

Paul McLanahan

unread,
Oct 12, 2009, 11:42:45 AM10/12/09
to django-d...@googlegroups.com
On Sat, Oct 10, 2009 at 2:05 PM, David Cramer <dcr...@gmail.com> wrote:
> The proposal per your email is more or less how django-notices works.

For me, the best solution would be a combination of django-notify and
django-notices. django-notify for their fallback backend and the
interchangeable backend support in general, and django-notices for the
logger-like API. I especially like the NOTICE_LEVEL setting which
could easily be controlled via your DEBUG setting. I agree that custom
levels should be easy and encouraged, and the strings used even for
the built-in levels should be configurable.

Paul

Tobias McNulty

unread,
Oct 12, 2009, 11:50:48 AM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 11:42 AM, Paul McLanahan <pmcla...@gmail.com> wrote:
> For me, the best solution would be a combination of django-notify and
> django-notices. django-notify for their fallback backend and the
> interchangeable backend support in general, and django-notices for the
> logger-like API. I especially like the NOTICE_LEVEL setting which
> could easily be controlled via your DEBUG setting. I agree that custom
> levels should be easy and encouraged, and the strings used even for
> the built-in levels should be configurable.

I'm with you 100%. The criteria on the wiki page say as much, but I
should probably clarify that django-notify does not actually provide
any default levels/classes/tags yet. Thanks for the feedback.

David Cramer

unread,
Oct 12, 2009, 11:52:01 AM10/12/09
to django-d...@googlegroups.com
I would definitely say it needs to be configurable then at the least.
As using cookies to send messges is fine if you guarantee the entity
is removed onthe next hit. Otherwise it becomes a performance issue
storing them for longer periods of time.

While I don't think that is common it should definitely be a factor in
the decision.

Sent from my iPhone

On Oct 12, 2009, at 9:39 AM, Tobias McNulty <tob...@caktusgroup.com>
wrote:

>

Paul McLanahan

unread,
Oct 12, 2009, 11:53:13 AM10/12/09
to django-d...@googlegroups.com
For the potential API...

http://code.djangoproject.com/wiki/SessionMessages#PotentialAPI

It would be really nice if one could optionally iterate over the
message lists separately. For example:

{% if request.messages.error %}
<ul class="messages error">
{% for message in request.messages.error %}
<li>{{ message }}</li>
{% endfor %}
</ul>
{% endif %}

And then get the rest normally...

{% if request.messages %}
<ul class="messages">
{% for message in request.messages %}
<li{% if message.tags %} class="{{ message.tags }}"{% endif
%}>{{ message }}</li>
{% endfor %}
</ul>
{% endif %}

This would allow for much more flexibility from a design perspective.

Paul

Tobias McNulty

unread,
Oct 12, 2009, 12:07:37 PM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 11:52 AM, David Cramer <dcr...@gmail.com> wrote:
> I would definitely say it needs to be configurable then at the least.
> As using cookies to send messges is fine if you guarantee the entity
> is removed onthe next hit. Otherwise it becomes a performance issue
> storing them for longer periods of time.
>
> While I don't think that is common it should definitely be a factor in
> the decision.

django-notify lets you plug in a pure session storage if that's what you want.

And if you do use cookies, they'll get removed 90% of the time on the
next request, and in most other cases removed on the subsequent
request (e.g., in the case of 2+ chained redirects).

The point is just to keep them around until they're actually
displayed, not indefinitely.

Tobias McNulty

unread,
Oct 12, 2009, 12:11:22 PM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 11:53 AM, Paul McLanahan <pmcla...@gmail.com> wrote:
> For the potential API...
>
> http://code.djangoproject.com/wiki/SessionMessages#PotentialAPI
>
> It would be really nice if one could optionally iterate over the
> message lists separately. For example:

I have no particular issue with allowing messages to be iterated in
full and/or by type. We could also just sort the messages and let you
use {% regroup messages by tag %}, but I suppose you want the ability
to pull out a particular type of error and that could get tedious with
{% regroup %}.

Paul McLanahan

unread,
Oct 12, 2009, 12:19:25 PM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 12:11 PM, Tobias McNulty <tob...@caktusgroup.com> wrote:
> I have no particular issue with allowing messages to be iterated in
> full and/or by type.  We could also just sort the messages and let you
> use {% regroup messages by tag %}, but I suppose you want the ability
> to pull out a particular type of error and that could get tedious with
> {% regroup %}.

I think regroup would be fine. My thought was that with iteration by
type you'd be able to easily highlight just one type if you so choose,
and even have any of the types displayed in a completely different
area of the markup if you needed. Having both abilities would be the
best-of-all-possible-worlds in my opinion, but this is starting to
smell like scope-creep. I'll just say that the backends and API are
the most important things, and having individually iterable lists is
just a "nice to have." I'd definitely be happy to help with the
implementation though if the group thinks it a worthy feature.

Paul

David Cramer

unread,
Oct 12, 2009, 2:02:50 PM10/12/09
to django-d...@googlegroups.com
I'm -1 on adding .errors or whatever. The main reasoning being that "levels" are simply numeric values (even though django-notices does provide default labels for its built-ins). Regroup is very easy and I don't think this is something that needs built-in, as there's no better way to optimize it than regroup. However, with doing this it'd be very important that it doesn't clear the messages unless you're pulling it out. django-notices handles this by popping out elements in the iter (I believe), so that if you don't pop the message out, it's still present.

----
David Cramer

Tobias McNulty

unread,
Oct 12, 2009, 2:32:43 PM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 2:02 PM, David Cramer <dcr...@gmail.com> wrote:
> I'm -1 on adding .errors or whatever. The main reasoning being that "levels"
> are simply numeric values (even though django-notices does provide default
> labels for its built-ins). Regroup is very easy and I don't think this is
> something that needs built-in, as there's no better way to optimize it than
> regroup. However, with doing this it'd be very important that it doesn't
> clear the messages unless you're pulling it out. django-notices handles this
> by popping out elements in the iter (I believe), so that if you don't pop
> the message out, it's still present.

In Python logging the levels are integers, but you still get
convenience methods like logger.error('msg'), etc.

I don't see why a similar method to filter the messages by type would
be a bad thing.

For the record I have no personal need for this feature (and hence
strong feelings about it), but nor do I see why adding it would be a
bad thing IF the need is actually there. Right now it sounds like
regroup will work fine for its original proponent, so, in the interest
of keeping things simple, let's assume it will not be included unless
someone brings up a compelling case.

David Cramer

unread,
Oct 12, 2009, 2:38:55 PM10/12/09
to django-d...@googlegroups.com
Sorry for the misunderstanding, I was talking specifically on retrieval of messages. I definitely see no reason not to keep it in line with the logging module in terms of errors/warnings/info messages. However, using things to "retrieve all error messages" should be left up to the user. It's no quicker doing it internally than it is using itertools.groupby or regroup in a template.

----
David Cramer

Paul McLanahan

unread,
Oct 12, 2009, 3:06:10 PM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 2:32 PM, Tobias McNulty <tob...@caktusgroup.com> wrote:
> For the record I have no personal need for this feature (and hence
> strong feelings about it), but nor do I see why adding it would be a
> bad thing IF the need is actually there.  Right now it sounds like
> regroup will work fine for its original proponent, so, in the interest
> of keeping things simple, let's assume it will not be included unless
> someone brings up a compelling case.

That's fair enough. Simple is good, and if someone really needs
different message retrieval methods, I'm sure a custom middleware
could provide anything they'd need.

Tobias: Are you planning on coding up a proposal app? It doesn't sound
like there's far to go from a branch or fork of django-notify. I
should have some time to contribute to the code or testing if you need
it.

Thanks for your work on this (and on django-notify),

Paul

Tobias McNulty

unread,
Oct 12, 2009, 3:34:14 PM10/12/09
to django-d...@googlegroups.com
On Mon, Oct 12, 2009 at 3:06 PM, Paul McLanahan <pmcla...@gmail.com> wrote:
> Tobias: Are you planning on coding up a proposal app? It doesn't sound
> like there's far to go from a branch or fork of django-notify. I
> should have some time to contribute to the code or testing if you need
> it.

Thanks for offering up your time. Once there seems to be a general
consensus about where we're going I plan to do that, yes. We're
getting closer but I'm not convinced we're there, yet.

Before we get too far, I'd appreciate hearing feedback from one or
more of the core devs (and from some of the folks who were involved in
this discussion back when it was happening on the ticket) on the wiki
page[1] and the general direction this project is going.

Tobias

[1] http://code.djangoproject.com/wiki/SessionMessages

Chris Beaven

unread,
Oct 13, 2009, 12:13:13 AM10/13/09
to Django developers
On Oct 13, 8:34 am, Tobias McNulty <tob...@caktusgroup.com> wrote:
> Before we get too far, I'd appreciate hearing feedback from one or
> more of the core devs (and from some of the folks who were involved in
> this discussion back when it was happening on the ticket) on the wiki
> page[1] and the general direction this project is going.

Thanks for championing this issue Tobias. I'm not great at
evangelizing :P

I've just updated django-notify to use notification levels (similar to
django-notices) - not having standard levels was the only thing I felt
was missing from django-notify (unfortunately, this is an incompatible
change for people who were using it to add custom tags, but I felt it
was worth it for the sake of moving forwards)

One thought: It would be backwards incompatible to change
{{ messages }}, it's already being used for User messages. To avoid
confusion, I have consistently steered away from using the term
"messages" by itself in documentation and code. I see that the wiki
page doesn't make the same distinction.

Tobias McNulty

unread,
Oct 13, 2009, 12:31:42 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 12:13 AM, Chris Beaven <smile...@gmail.com> wrote:
> One thought: It would be backwards incompatible to change
> {{ messages }}, it's already being used for User messages. To avoid
> confusion, I have consistently steered away from using the term
> "messages" by itself in documentation and code. I see that the wiki
> page doesn't make the same distinction.

Yes, that's a choice we'll have to make. Most folks seem to default to
calling them messages (or maybe that's just me).

That said, I see your point. What about {{ request.messages }}? Is
that still going to be confusing? Whatever the name becomes, I don't
really see the need for the extra context processor. You'll likely be
writing the name far more often in your views than in your templates.

This also raises the issue of how this app will live side by side with
User messages. I threw out a few ideas on the wiki (give User
messages a much narrower purpose than they have now and/or phase them
out altogether), but I'm sure there are other (probably better) ones.

My only real issue with "notifications" is that it's more letters to
type and fewer letters in the actual message (before the line hits 80
chars).

Tobias

Russell Keith-Magee

unread,
Oct 13, 2009, 3:27:19 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 3:34 AM, Tobias McNulty <tob...@caktusgroup.com> wrote:
>
> Before we get too far, I'd appreciate hearing feedback from one or
> more of the core devs (and from some of the folks who were involved in
> this discussion back when it was happening on the ticket) on the wiki
> page[1] and the general direction this project is going.

On the whole, I'm comfortable with what I'm seeing. I haven't looked
into the codebase for django-notify, but the feature set certainly
makes it look like a reasonable starting point.

Notwithstanding a teardown of the actual code, I have three comments
on progress thus far:

Firstly, given the similarity between the APIs, it seems like we
should be trying to maintain parity with logging. That means:

request.messages.add(messages.INFO, 'message')

with:

request.messages.info('message')

as an alias. Note that this is a reversal of argument order from the
proposed API. I'm not absolutely tied to this suggestion - I'm just
noting that adding Django support for Python logging is also on the
cards for v1.2, and it seems weird that we would introduce two APIs
with similar external APIs, but not try to maintain parity.

Secondly, I share Chris' concern about overloading 'messages'. This
overlaps with the bikeshed issue of what to call the new contrib
package - whatever name is picked for the contrib package should
probably be reflected in the context variable, etc.

Lastly, one issue that seems unaddressed is the transition plan for
replacing user.message_set. The preamble to the wiki page makes a
compelling argument for ditching message_set, but remains silent on a
proposal for how Django (as a project) should be treating this new
alternative.

The solution might well be "use it if you want, ignore it if you
don't", but given the problems with user.message_set, it might be
worth giving some thought to a deprecation/transition plan.

Yours,
Russ Magee %-)

Hanne Moa

unread,
Oct 13, 2009, 3:53:43 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 09:27, Russell Keith-Magee
<freakb...@gmail.com> wrote:
> I'm just
> noting that adding Django support for Python logging is also on the
> cards for v1.2, and it seems weird that we would introduce two APIs
> with similar external APIs, but not try to maintain parity.

In fact, it would be very useful to both log and add a message at the
same time, iff there is an error. So that shouldn't deliberately be
made hard to do but maybe suitable for a lazy man's shortcut: one call
that does both.


HM

David Cramer

unread,
Oct 13, 2009, 8:17:22 AM10/13/09
to django-d...@googlegroups.com
I don't agree with one calling doing logging and notices. There's really no reason to mix in logging with the notices framework as they serve completely different purposes. I believe Russel is just suggesting the APIs match so that there is no additional learning curve, which makes complete sense.

----
David Cramer

Tobias McNulty

unread,
Oct 13, 2009, 8:22:08 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 3:53 AM, Hanne Moa <hann...@gmail.com> wrote:
> In fact, it would be very useful to both log and add a message at the
> same time, iff there is an error. So that shouldn't deliberately be
> made hard to do but maybe suitable for a lazy man's shortcut: one call
> that does both.

That is binding together two technologies that are still fairly
conceptual in terms of their integration with Django at this point.

I'm a little worried that we wouldn't get it right the first time
around. The extreme case would involve implementing messaging as a
python logging handler. Frankly I'm not sure it's worth it.

Furthermore, "error" message to the user may not always be, in fact
often probably isn't, an error for the web app, so it'd be important
to differentiate between the two. E.g., you might want to log an
ERROR message under the INFO python logging level.

Short of providing this do-all call in the implementation itself, I
see a few options:

1) write your own wrapper if you do a lot of that

2) add some logging calls to the messaging code so that you can see
messages that were shown to the user if you want to, under a static
logging level (e.g., INFO)

I like #2, provided that the default logging handler in Django is a
NullHandler (I think it is/will be).

Tobias

Jacob Kaplan-Moss

unread,
Oct 13, 2009, 9:48:34 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 2:27 AM, Russell Keith-Magee
<freakb...@gmail.com> wrote:
> Lastly, one issue that seems unaddressed is the transition plan for
> replacing user.message_set. The preamble to the wiki page makes a
> compelling argument for ditching message_set, but remains silent on a
> proposal for how Django (as a project) should be treating this new
> alternative.

This is my main concern, and the thing keeping me from being
enthusiastic about the current state of the proposal. Many apps --
including, notably, django.contrib.admin -- rely on user.message_set.
We can't simply remove it; we need to follow a deprecation schedule.
But we also shouldn't just keep two parallel "messaging"
implementations; that's silly.

The best solution, I think, would be to implement user messages
(user.message_set, get_and_delete_messages, etc) in terms of the new
framework, and then gradually deprecate and remove the old APIs.

Jacob

Hanne Moa

unread,
Oct 13, 2009, 9:51:29 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 14:22, Tobias McNulty <tob...@caktusgroup.com> wrote:
> On Tue, Oct 13, 2009 at 3:53 AM, Hanne Moa <hann...@gmail.com> wrote:
>> In fact, it would be very useful to both log and add a message at the
>> same time, iff there is an error. So that shouldn't deliberately be
>> made hard to do but maybe suitable for a lazy man's shortcut: one call
>> that does both.
>
> That is binding together two technologies that are still fairly
> conceptual in terms of their integration with Django at this point.
>
> I'm a little worried that we wouldn't get it right the first time
> around.  The extreme case would involve implementing messaging as a
> python logging handler.  Frankly I'm not sure it's worth it.

I think y'all misunderstood what I tried to say. I think it'll be
common to "bundle" them like this (it's not as if we can prevent
anyone from doing it!) and that there might one day be a desire for an
official shortcut to do it.

> Short of providing this do-all call in the implementation itself, I
> see a few options:
>
> 1) write your own wrapper if you do a lot of that

Already have, for django-notify. Since I already need to test that a
request has a "notifications"-attribute I have a handy place to attach
other "messaging" in the wide sense too, like logging, debug-stuff,
assert False when manually checking things...

> 2) add some logging calls to the messaging code so that you can see
> messages that were shown to the user if you want to, under a static
> logging level (e.g., INFO)

... or under its own prefix so that it can be nulled.
getLogger('msg.'+__name__) or however it winds up looking.


HM

Paul McLanahan

unread,
Oct 13, 2009, 9:54:00 AM10/13/09
to django-d...@googlegroups.com
On Tue, Oct 13, 2009 at 3:27 AM, Russell Keith-Magee
<freakb...@gmail.com> wrote:
> Secondly, I share Chris' concern about overloading 'messages'. This
> overlaps with the bikeshed issue of what to call the new contrib
> package - whatever name is picked for the contrib package should
> probably be reflected in the context variable, etc.
>
> Lastly, one issue that seems unaddressed is the transition plan for
> replacing user.message_set. The preamble to the wiki page makes a
> compelling argument for ditching message_set, but remains silent on a
> proposal for how Django (as a project) should be treating this new
> alternative.
>
> The solution might well be "use it if you want, ignore it if you
> don't", but given the problems with user.message_set, it might be
> worth giving some thought to a deprecation/transition plan.

This may be adding undue complexity, but I wanted to throw it out and
see what you thought anyway. I think a possible solution to Russ' last
two points may indeed be overloading the "messages" template variable.
What I'm thinking is that we keep the message_set (of course) and add
the new "request.messages.*" API, but completely remove the messages
part of the auth context_processor and have the default backend for
the new system pull both for the first release (purely for backward
comparability). Messages from the old message_set system could get a
default level in the new system. This shouldn't be an issue since the
people using it will likely be existing projects where they won't be
using any level information in their templates. We couple this
combined backend with settings that will allow you to easily disable
the DB part of the backend (or make it clear how to and strongly
suggest switching backends), and provide a setting for the default
message level that defaults to INFO.

I think this could ease transition as the two systems could live
together for the first release. The subsequent releases could switch
the default backend to the existing fallback one, and then eventually
remove the message_set part of the auth app all together.

I do recognize that this may create more confusion than is necessary.
It may also be an unacceptable performance drain on the new system to
support the old at all. I just think as someone who has projects that
still use the message_set, and especially considering the 3rd party
apps out there which may use it as well, it might be a nice balance to
be able to mix in the new and better while having as little impact on
the templates as possible.

Technically, this may be more of a challenge than it's worth however.
The current auth context_processor is in core, and it just really
feels wrong to me to couple this new app to either core or
contrib.auth. We could just remove the "messages" part of the auth
context_processor and add a new processor from the new app to the list
of defaults that will provide all of the "messages". And actually,
we'll need to at least override that part of the auth
context_processor anyway, or else we'll still get a db hit for every
request with a logged in user due to the call to
user.get_and_delete_messages.

Sorry about throwing my brainstorm at you. Perhaps it'll spark someone
else's solution.

Thanks,

Paul

Tobias McNulty

unread,
Oct 13, 2009, 7:45:31 PM10/13/09