Custom ManyRelatedManager?

[Andreas Hasenkopf]

Hi everyone,

on StackOverflow the question on how to overload the ManyRelatedManager
for ManyToMany relations has been asked multiple times, but none of the
answers seemed to be satisfying. So I would like to talk about it with you.

## My problem

The backend of my Django app gets triggered by events of a RabbitMQ
server. After pre-filtering the backend dispatches a Celery task to
process data from a remote system to which the event belongs.
Unfortunately events for the same object can arrive in bursts leading to
multiple Celery tasks processing the same objects almost simultaneously.

## My solution so far

In my custom ModelManager I explicitly use Postgres Advisory locks, e.g.
in `get_or_create`, which is working really well.

## Missing link

As my data model contains quite a few ManyToMany relations I would
prefer to define the usage of locks only once in the methods of a class
derived from ManyRelatedManager, e.g. in the `set` method.
So far I was unable to figure out how to do that. Any suggestions?

## Example

To give you a better understanding let's look at some code excerpts:

```
class ExtendedQuerySet(QuerySet):
def get_or_create(self, defaults=None, _deadlocks=5, **kwargs):
try:
with advisory_lock(self.model.__name__):
return super().get_or_create(defaults, **kwargs)
except DeadlockDetected as error:
# retry in a second...
...

class ExtendedManager(Manager.from_queryset(ExtendedQuerySet)):
pass

class Reference(models.Model):
# removed field definitions

objects = ExtendedManager()

class Request(models.Model):
# removed some field definitions
references = models.ManyToManyField(Reference)

objects = ExtendedManager()
```

If two Celery tasks try to call e.g.
`Reference.objects.get_o_create(...)` at the same time, the locks work
as expected. But e.g. calling `request.references.set(...)` can fail
(unique constraint violation) in the second Celery task due to a race
condition, that I would like to avoid using locks.


Thanks for your help.


Andreas

--
Andreas Hasenkopf
Phone: +49 151 11728439
Homepage: https://hasenkopf.xyz
GPG Pub Key: http://goo.gl/4mOsM