re:dash queue gets stuck every now and then

[Elik Rozenboim]

So we've been running into a weird problem lately than I'm not sure how I should approach.

I haven't been able to understand the circumstances in which it happens but over the last month our re:dash instance has frozen 3 times in such a way that the webserver runs normally but queries are infinitely stuck in the queue for some reason.

Restarting the celery workers solves the problem but I'd like to stop it from happening again. 

We're running the latest stable version on an EC2 t2.micro instance with a single redshift cluster as the main datasource. 

[Eran Sandler]

Queries are run using a set of celery workers. Which run through the supervisor (you can find the init script  in /etc/init.d/)

If things are stuck like this it might be because the celery processes are down or are stuck.

Since t2.micro is a very small instance you might have some queries that require more RAM to run and that might be killing the processes via the OS Out of Memory process killer.

I suggest checking if it's running (if you have celery processes up)  and also watch live if you see them die of replace the process ID (if they get restarted again).

If that is the case I would consider moving to  a bigger instance.

Eran

--
You received this message because you are subscribed to the Google Groups "re:dash users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redash-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Elik Rozenboim]

Thanks, will try that!