Persistent job queue

[Brian Adkins]

I haven't been able to find a persistent job queue for Racket, so I thought I'd ask if one exists before writing a simple one. I did see Jay's job queue package, but it doesn't appear to persist to disk, and that is a requirement.

I typically use Postgres for my apps, and I would prefer to also use Postgres for the persistence of the job queue to avoid introducing something new (e.g. Redis) to my server setup. Amazon RDS provides great multi-site availability, so I want to continue to leverage that.

Thanks,

Brian Adkins

[Bogdan Popa]

> I haven't been able to find a persistent job queue for Racket, so I thought
> I'd ask if one exists before writing a simple one. I did see Jay's job
> queue package, but it doesn't appear to persist to disk, and that is a
> requirement.

FWIW, I am working on a new Redis client package[1] that should be
complete sometime next week and I have a little task queue example
here[2] based on Redis streams.

I plan on writing a task queueing library in the future based on this
client (I have some experience with the problem domain[3]), but it may
be a while before I get to it.

> I typically use Postgres for my apps, and I would prefer to also use
> Postgres for the persistence of the job queue to avoid introducing
> something new (e.g. Redis) to my server setup. Amazon RDS provides great
> multi-site availability, so I want to continue to leverage that.

Recent versions of Postgres make this pretty easy to do so you might not
need a library. Here's a little snippet from one of my older Racket
applications that uses a Postgres table as a persistent task queue, in
case it helps:

https://gist.github.com/Bogdanp/c796b15a0fa8d577a5f46ff25230baa6



[1]: https://github.com/Bogdanp/racket-redis
[2]: https://github.com/Bogdanp/racket-redis/tree/master/examples/task-queue
[3]: https://dramatiq.io