Serializing forms and formsets to pass to Celery task

[Bernd Wechner]

I'm using Celery quite effectively with Django as many are. It's just awesome for running time consuming background tasks, and providing progress bar updates to a web page. Love it.

Now I would like a celery task to do some form processing. That is in a Django view, where I have access to the form (or formset) that's just been submitted, with all its data (in form.request.POST) and methods (like form.save() which saves it to the database as a Django object or objects (in the case of a formset).

Of course anything we pass to a Celery task must be serialized (as it's passed through the broker) and there is plenty of help around on serializing Django data objects but I can't find diddly squat on serialzing forms and every effort I've tried, JSON, pickle, dill, all fails.

I can serialize the POST data of course, and that great but I can't see how the Celery task can then re-instantiate a form object from that.

I posted this on stackoverflow as well with some richer detail: https://stackoverflow.com/questions/60433090/processing-a-django-form-in-a-celery-task-how-can-the-celery-task-see-the-form

I wonder if anyone has any experience to share here. Essentially I see two possible solutions:

1. Serialize the form or formsets somehow. Anything better than dill out there that might?
2. Serialize some part of the form or formsets (eg the POST data) and re-instantiate a form or formset at the other end using it.
   

Either approach would work, 1. would seem less hassle if it has a solution, and 2. seems more likely to provide one to me right now.

Kind regards,

Bernd.

[Naveen Arora]

Hi Bernd,

You can't serialize a form data object this way.

All the solution i can see is to save data temporarily and use it.

Cheers,

[Jason]

As said before, you really can't do that too well with celery.  Its a good practice to keep the message body between django -> broker -> worker to be as small as possible because we don't know what your configuration for the broker is.  In addition, you should never, ever, put sensitive information in a message, just the identifier of a thing and let celery/django pull from a data store when the task starts.

what I would do is save the form data to some intermediate data store, and pass in the identifier to the celery task for retrieval.  That ensures your data integrity and security is high.

[Naveen Arora]

Yes that would work well. You can always find a completely different way of achieving what you are trying to achieve.

[Integr@te System]

Hi Bernd,

You could take a look at serializers section on Note part, for more information, and may be your own answer.

http://docs.celeryproject.org/en/master/userguide/calling.html#calling-serializers

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/3f1b4374-ca68-4fcd-a33a-555bc72cbae8%40googlegroups.com.