mclapply

[Jonathan]

Howdy,

We have R shiny server pro running on a machine with 20 CPUs. We have code that uses mclapply. When we run the code in R outside of R shiny, it runs fine on linux. But when we put the code in a reactive block, the reactive block executes, but behaves as if it skips the mclapply call. If we replace mclapply with lapply in the R shiny app, it runs correctly. When we use mclapply with mc.cores=1, it runs fine.

Has anyone gotten mclapply to work with R shiny?

Jonathan

[Joe Cheng]

There may be an incompatibility with httpuv, I'd try that first. Something like:

app <- list(call = function(req) {

  # perform mclapply call here

  list(

    status = 200L,

    headers = list("Content-Type" = "text/html"),

    body = "OK"

  )

})

httpuv::runServer("0.0.0.0", 8001, app)

You can replace "OK" with some evidence that the mclapply actually did something.

--
You received this message because you are subscribed to the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shiny-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robin Cura]

Hi,

I have a working app using mclapply (parallel package), see it running there, and the code here (in global.R, function run_simulation).

This runs on shiny-server v1.1.0.10000

HTH,

Robin

Robin Cura

Doctorant à l'UMR Géographie-cités

LabEx DynamiTe & Université Paris 1 Panthéon-Sorbonne

[Jonathan]

Thank you for your replies. Joe, I did not try exactly what you show, but I have tested that mclapply runs and does what it's supposed to do on the server without any reactivity. Robin, my setup is similar to your code. However, when I clicked on the link, your code was not working.

The issue is embedded within a relatively complex Shiny app. When I tried to make a simpler example, I could not replicate the problem. Here's a little more specific description of the problem. 

 

The problem occurs with this statement:

haz_test <- reactive({ analysis_test( ..., input$nmbr, session)  }) 

where

input$nmbr is a numericInput defined in ui.R

analysis_test() is a function that calls mclapply() with mc.cores=10

... are other arguments

When the app starts, the above works fine on the 1st call. mclapply() is called within analysis_test() and executes properly. But then while the app is running, when I change the value of input$nmbr, analysis_test() does not get called again in response to the change. It is as if the reactivity is broken. It doesn't recognize that input$nmbr changed, and analysis_test() does not get called. If I change to mc.cores=1, then the reactivity works fine. Also if I insert "input$nmbr" explicitly like the following, then it works fine even with mc.cores = 10!! It is a very strange problem. I can not replicate it with a small example (I tried), And I can't share the code that it happens in. The same issue happens with both Chrome and Firefox, and when I run it in Rstudio viewer window.

haz_test <- reactive({

   input$nmbr

   analysis_test( ..., input$nmbr, session)

}) 

[Winston Chang]

I think this may be an inherent limitation of mclapply. From the docs:

"mclapply is a parallelized version of lapply, but there is an important difference: mclapply does not affect the calling environment in any way, the only side-effect is the delivery of the result (with the exception of a fall-back to lapply when there is only one core)."

A little background: A function without side effects is one that takes an input and returns an output without affecting anything else. A function with a side effect alters the external environment in some way -- for example, if a function modifies a global variable, that's a side effect.

In the case of Shiny, the way that a reactive expression or observer takes a dependency on another reactive object is via side effects -- the evaluation of input$nmbr alters the state of the input$nmbr so that it knows to invalidate the reactive expression when its value changes.

My understanding is that mclapply doesn't have side effects because it spawns a new process for each run, and the new processes can't affect the state of the original process.

I think you should be able to work around this by simply evaluating input$nmbr outside of the mclapply, like this:

haz_test <- reactive({

  input$nmbr # Take a dependency on this

  analysis_test( ..., input$nmbr, session)

})

-Winston