Feature request: post/pre commit/rollback signals

[Jesús Espino]

Hi folks!

The propose is add post and pre signals for commit and rollback actions (as sqlalchemy orm events). This allows attach some code when a transaction is committed or rolled back.

I have some problem, such as send email only when a transaction is committed successfully, and it's can be done with simple monkey patching the db backend. But it not appear to be an isolated problem, and would be awesome if these signals are included in core.

If a purpose is accepted, I can take care of making the issue + pull-request.

Greetings.

Jesus.

[Andreas Pelme]

Hi,

This is a tricky issue that has been discussed earlier, you will probably want to check out ticket #14051:

https://code.djangoproject.com/ticket/14051

—
Andreas Pelme

[Andrey Antukh]

Hi!

In my opinion, "the default case are solved with atomic block" seems to be a workaround instead of a solution. I understand that signals in general are evil and register global callbacks is not a very good solution, but I think, that a orm should give some generic facility to attach code to execute when a transaction is successfully committed or rolled back.

For example, one possible use case can be this:

import threading

from django.dispatch import receiver

from django.core.signals import request_started

from .signals import post_commit

_local = threading.local()

_local.post_commit_tasks = []

@receiver(post_commit, dispatch_uid="_run_defered_tasks")

def _run_defered_tasks(sender, **kwargs):

    while _local.post_commit_tasks:

        task = _local.post_commit_tasks.pop(0)

        task[0](*task[1], **task[2])

@receiver(request_started, dispatch_uid="_clean_tasks")

def _clean_tasks(sender, **kwargs):

    _local.post_commit_tasks = []

def delay_until_transaction(func, *args, **kwargs):

    _local.post_commit_tasks.append((func, args, kwargs))

This is one of much other examples that try to solve this (problem/use case) out of django, and I think that django/django-orm should provide some facility to defer execution some code after a transaction is commited or rolled back.

In the last aaugustin comment, he suggest one approach without using global signals, and it seems  very interesting approach. Can these  approach to be generalized and included in django? 

As Jesus said, it not seems a isolated problem, and would be awesome if django provides some generic and good way to do this. 

Greetings.

Andrey.

2014/1/10 Andreas Pelme <and...@pelme.se>

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/948D5F1E-F86D-44A0-9EEA-F1E9A6A4A382%40pelme.se.

For more options, visit https://groups.google.com/groups/opt_out.

--

Andrey Antukh - Андрей Антух - <andrei....@kaleidos.net> / <ni...@niwi.be>

http://www.niwi.be

https://github.com/niwibe

[Aymeric Augustin]

On 11 janv. 2014, at 11:05, Andrey Antukh <ni...@niwi.be> wrote:

> In my opinion, "the default case are solved with atomic block" seems to be a workaround instead of a solution.

As a short reminder, we’re talking about inconsistencies that arise when:

1) you do something in the database;
2) you do something related in another system — send an email, enqueue a task, cache a value;
3) the database transaction is rolled back, canceling the effects of 1) but not those of 2).

I may have been unclear when I closed the ticket. I’m not saying that atomic blocks solve the problem. I’m saying that the problem doesn’t happen in autocommit mode, which is the default in Django >= 1.6.

If you’re using transactions (i.e. ATOMIC_REQUESTS or atomic blocks), you may encounter the problem. But then I’m positive that transaction signals aren’t the solution.

> I understand that signals in general are evil and register global callbacks is not a very good solution, but I think, that a orm should give some generic facility to attach code to execute when a transaction is successfully committed or rolled back.

I’m not having this argument once more, so, there you go:
https://github.com/aaugustin/django-transaction-signals
https://pypi.python.org/pypi/django-transaction-signals-do-not-use

You may also be interested in:
https://github.com/davehughes/django-transaction-signals
https://pypi.python.org/pypi/django_transaction_signals

(The name conflict is unfortunate; I didn’t check before writing my package; at least that forced me to include a warning in the name.)

Jokes aside, transaction signals are just a (wrong) means to an (legitimate) end. I’ve documented the solutions I’m aware of:
https://github.com/aaugustin/django-transaction-signals#alternatives

Additional ideas welcome!

--
Aymeric.

[Andrey Antukh]

Thanks for your package and clear explanation ;)

Greetings.

Andrey.

2014/1/11 Aymeric Augustin <aymeric....@polytechnique.org>

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/989FB6EA-12A2-4D54-9821-FB8473F3AF29%40polytechnique.org.

For more options, visit https://groups.google.com/groups/opt_out.

[Carl Meyer]

Hi,

Those interested in this feature may find django-transaction-hooks [1] useful. It is my attempt to implement the commonly-useful case (run some code after the current transaction successfully commits, providing the code wasn't registered during a savepoint which was later rolled back.) I think the API provided by django-transaction-hooks is actually better for that use case than transaction signals are, since it does the work of handling some situations correctly (savepoints, closed connections), which in the case of signals are left up to the signal receiver to handle (or more likely, not bother to handle.)

Testing and feedback welcome. If this becomes widely used and seems to fill a need, I think there is a chance it could become part of Django core. (See #21803.)

Carl

  [1] https://github.com/carljm/django-transaction-hooks

[Aymeric Augustin]

On 18 janv. 2014, at 19:50, Carl Meyer <ca...@oddbird.net> wrote:

> Those interested in this feature may find django-transaction-hooks [1] useful. It is my attempt to implement the commonly-useful case (run some code after the current transaction successfully commits, providing the code wasn't registered during a savepoint which was later rolled back.) I think the API provided by django-transaction-hooks is actually better for that use case than transaction signals are, since it does the work of handling some situations correctly (savepoints, closed connections), which in the case of signals are left up to the signal receiver to handle (or more likely, not bother to handle.)
>
> Testing and feedback welcome. If this becomes widely used and seems to fill a need, I think there is a chance it could become part of Django core. (See #21803.)

I think this is the right approach. I would use that package. I support adding it to Django.

Since it doesn’t do anything until after the transaction has successfully committed, it cannot interfere with the transaction management code. That’s my most important requirement and the reason why I’m so strongly opposed to pre-commit/rollback signals.

If you look at the first email in this thread, the use case is “do something when a transaction is committed successfully”. In the later email that gives an example, it only uses post_commit. Generic transaction signals look like a collective hallucination to me, or at least a pretty bad case of YAGNI.

--
Aymeric.

[Andrey Antukh]

Hi!

2014/1/18 Aymeric Augustin <aymeric....@polytechnique.org>

On 18 janv. 2014, at 19:50, Carl Meyer <ca...@oddbird.net> wrote:

> Those interested in this feature may find django-transaction-hooks [1] useful. It is my attempt to implement the commonly-useful case (run some code after the current transaction successfully commits, providing the code wasn't registered during a savepoint which was later rolled back.) I think the API provided by django-transaction-hooks is actually better for that use case than transaction signals are, since it does the work of handling some situations correctly (savepoints, closed connections), which in the case of signals are left up to the signal receiver to handle (or more likely, not bother to handle.)
>
> Testing and feedback welcome. If this becomes widely used and seems to fill a need, I think there is a chance it could become part of Django core. (See #21803.)

I think this is the right approach. I would use that package. I support adding it to Django.

 

You are absolutely right! This is much best approach than using signals (I also don't like signals, my approach was only for show a use case).

And I think that this is much less intrusive (and without signals), and would be awesome if it be added to django. This use case is very common in web development.

Since it doesn’t do anything until after the transaction has successfully committed, it cannot interfere with the transaction management code. That’s my most important requirement and the reason why I’m so strongly opposed to pre-commit/rollback signals.

If you look at the first email in this thread, the use case is “do something when a transaction is committed successfully”. In the later email that gives an example, it only uses post_commit. Generic transaction signals look like a collective hallucination to me, or at least a pretty bad case of YAGNI.

--
Aymeric.

Thanks!

Greetings.

Andrey 

[Anssi Kääriäinen]

On Saturday, January 18, 2014 9:02:41 PM UTC+2, Aymeric Augustin wrote:

On 18 janv. 2014, at 19:50, Carl Meyer <ca...@oddbird.net> wrote:

> Those interested in this feature may find django-transaction-hooks [1] useful. It is my attempt to implement the commonly-useful case (run some code after the current transaction successfully commits, providing the code wasn't registered during a savepoint which was later rolled back.) I think the API provided by django-transaction-hooks is actually better for that use case than transaction signals are, since it does the work of handling some situations correctly (savepoints, closed connections), which in the case of signals are left up to the signal receiver to handle (or more likely, not bother to handle.)
>
> Testing and feedback welcome. If this becomes widely used and seems to fill a need, I think there is a chance it could become part of Django core. (See #21803.)

I think this is the right approach. I would use that package. I support adding it to Django.

If documentation warns clearly that there is no guarantee whatsoever that the hook is always ran after commit +1 for addition in Django. The most obvious case of abuse for this sort of hook is doing something business critical in the post-commit hook. It must be clear that this is not a substitute for two phase commit or other ways of avoiding crash-just-after-commit problem.

 - Anssi

[Carl Meyer]

On 01/18/2014 01:19 PM, Anssi Kääriäinen wrote:
> On Saturday, January 18, 2014 9:02:41 PM UTC+2, Aymeric Augustin wrote:
>
> On 18 janv. 2014, at 19:50, Carl Meyer <ca...@oddbird.net

Right. I have this warning in the current docs for
django-transaction-hooks, though it should probably be more prominent.
I'll make sure it gets into the eventual Django patch, too.

Carl