Utilizing Redis in OpenCPU

[kevin...@gmail.com]

Hello,

First of all, thanks for creating and maintaining OpenCPU.

We are looking at different approaches for refactoring file writing/reading in OpenCPU code with something else, e.g. Redis. We are exploring wrapping the saveRDS and readRDS in appropriate "session save/load" helper functions that can then potentially re-route to something else.  This helps isolate the code for persisting the data to something else other than the file system.  How do you feel about that approach?

I have seen that, back in 2014 (https://github.com/opencpu/opencpu/issues/103), replacing writing to files on disk with storing temporary sessions in Redis has been considered before. What was the outcome of this consideration?

The cloud application which my team is working on allows the user to interactively manipulate around 1 MB of data set (maybe a larger size support later). OpenCPU server instances are hosted in multiple containers which are all mapped to the shared file system. Each time the user performs an operation, the result R data frame is send back to the UI to progressively show the output. For cloud resilience reasons (shared file systems aren’t ideal), we would like remove this dependency and use something else like Redis.

Thanks and regards,
Kevin

[Jeroen Ooms]

Yes I think it makes sense to use a database client in R (such as the redux package), to save/load data from R. Ideally you would store your data in a standard data-interchange format (e.g. json) so that you can access the same data from other languages outside of R.

I never implemented this by default because I think this is more application logic, which may be different for other use cases. Other folks may prefer storing their data on disk, or in postgres, or mongo, etc.

--
You received this message because you are subscribed to the Google Groups "opencpu" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencpu+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencpu/8329523a-211e-4f44-b7e6-48836b1fc19bn%40googlegroups.com.