Some years ago I used celery for a project.
Today I face roughly the same use case again.
The last time I used it, celery felt to heavy weighted, too complicated.
I found this alternative:
http://python-rq.org/There are two things which speak agains python-rq:
- redis is a cache. I need a persistent queue. If a power-failure happens, no job must get lost.
- AFAIK python-rq uses python pickle to serialize data. I want a language neutral data exchange format.
At the moment my favourite protocol is gRPC (protocol-buffers). I never used it up to now, but this would be a nice use case.
Maybe I am missing the right terms. What does celery implement? According to wikipedia "asynchronous task queue". But
my favorite search engine could not reveal a gRPC based server implemented in Python ....
What do you think?