Why does transaction management only commit on is_dirty?
Christophe Pettus
I realize that the answer to this question is, "Why commit if there are no changes to the database?", but it's a bit more complicated than that.
Let's assume a reasonably common case:
1. psycopg2 + PostgreSQL.
2. Transaction middleware enabled, or @commit_on_success decorator.
3. Read-only view function.
In this case, psycopg2 will begin a transaction on the first database read, but no COMMIT will ever be sent to the database. Until the connection actually closes, this means that the connection will be in <idle in transaction> state on the PostgreSQL server, an expensive state to be in. (A pile-up of <idle in transaction> connections is a pretty common occurrence in Django applications, in my experience.)
It seems that this problem is trivially fixed by always committing in the places that a commit depends on is_dirty(). The commit should be near-free on any back-end, since there are no changes, and it eliminates the IIT issue on PostgreSQL (and perhaps other problems on MySQL that I'm not as familiar with).
What am I missing?
--
-- Christophe Pettus
x...@thebuild.com
Jacob Kaplan-Moss
> Why does transaction management only commit on is_dirty?
It's a bug: http://code.djangoproject.com/ticket/9964.
Looks like the patch there is OK, but still needs some work (there's a
couple of TODOs still).
Jacob
Alex Gaynor
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>
>
Those patches are all well out of date, so if this is something you're
interested in, fixing that up would be a good start.
Alex
--
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me
Christophe Pettus
On Oct 22, 2010, at 4:01 PM, Jacob Kaplan-Moss wrote:
> It's a bug: http://code.djangoproject.com/ticket/9964.
>
> Looks like the patch there is OK, but still needs some work (there's a
> couple of TODOs still).
On it! :)
Christophe Pettus
On Oct 22, 2010, at 4:01 PM, Jacob Kaplan-Moss wrote:
>
> Looks like the patch there is OK, but still needs some work (there's a
> couple of TODOs still).
Looking at the history of the ticket, it looks like there is some concern about keeping the "current behavior" to maintain backwards compatibility.
Which raises the question: Just what is the current behavior that we'd like to preserve? The current situation seems to be quite indeterminate; the transaction just stays open until it is closed... somehow, by some means (probably a rollback when the connection closes). Is this really explicit-enough behavior that maintaining it is important? Are applications really relying on it?
Shai Berger
On Saturday 23 October 2010, Christophe Pettus wrote:
> On Oct 22, 2010, at 4:01 PM, Jacob Kaplan-Moss wrote:
> > It's a bug: http://code.djangoproject.com/ticket/9964.
> >
> > Looks like the patch there is OK, but still needs some work (there's a
> > couple of TODOs still).
>
I was quite surprised to see that comment, in light of
http://code.djangoproject.com/ticket/9964#comment:15 where Russell, Malcolm
and Jacob decided that the patch's approach was unacceptable.
> Looking at the history of the ticket, it looks like there is some concern
> about keeping the "current behavior" to maintain backwards compatibility.
>
> Which raises the question: Just what is the current behavior that we'd like
> to preserve?
The current behavior is to keep "clean" transactions open. As benchmarks I ran
then indicated ( http://code.djangoproject.com/ticket/9964#comment:6 ), this
may under some scenarios actually improve performance.
> The current situation seems to be quite indeterminate; the
> transaction just stays open until it is closed... somehow, by some means
> (probably a rollback when the connection closes).
(more probably until the relevant thread handles a request which updates the
database through the models, or otherwise dirties a transaction)
> Is this really
> explicit-enough behavior that maintaining it is important? Are
> applications really relying on it?
>
I would be extremely surprised if there is any application out there which
conciously relies on it. I would be almost as surprised if the behavior is
changed to "always close transactions" and no website starts to break
misteriously or experience 20% slow-downs. The change is deep. This is why I
tried to make it so that all new projects start with correct behavior, but
upgrades under old projects will not change the behavior.
If Jacob has indeed changed his mind about the viability of this approach --
or if your "just do the right thing" approach is accepted -- I will be
delighted to cooperate on updating the patches. Just please CC me with
responses, because I tend to neglect monitoring this list.
Thanks,
Shai.