doParallel in Windows

[Yaming Su]

Hi,

I attempted some parallel simulation using mrgsolve in Windows.  Kyle's example in one of his training materials (shown below) worked fine. I assume the below example simply replicated the simulation twice.

cl <- makeCluster(4)
registerDoParallel(cl)

clusterCall(cl, function() {
  library(mrgsolve)
  library(dplyr)
}
)

mod. <- mrgsolve:::house()


out. <- foreach(i=1:2) %dopar% {
  mod. %>%
    ev(amt=1000, ii=8, addl=3) %>%
    mrgsim %>% mutate (irep=i)
} %>% bind_rows

stopCluster(cl)


I then tried another example where I compiled a model,  and did the simulation using the same method as the above, but got the error message as below:

Error in { : 
  task 1 failed - "Model may not have been successfully compiled.
   Source: C:/Users/YSU/mrgsolve_projects/doParallel/simplePK.cpp
   Shared object (DLL): C:/Users/YSU/AppData/Local/Temp/RtmpYlKCuf/simplePK1cec4daa42bf.dll"

The above message is from the below R code:
code <- '
$PARAM TVCL = 1, TVVC = 35, KA = 1.2, WT = 70, WTCL = 0.75

$CMT GUT CENT

$MAIN
double CL = TVCL*pow(WT/70, WTCL);
double V = TVVC*pow(WT/70, 1);

$ODE
dxdt_GUT = -KA*GUT;
dxdt_CENT = KA*GUT -(CL/V)*CENT;

$TABLE
table(CP) = CENT/V;

$CAPTURE CL

'
mod3 <- mread("simplePK", code=code)

cl <- makeCluster(4)
registerDoParallel(cl)

clusterCall(cl, function() {
  library(mrgsolve)
  library(dplyr)
}
)

out3 <- foreach(i=1:100) %dopar% {
  mod3 %>%
  ev(amt=1000, cmt=1, WT=70) %>%
  mrgsim %>% mutate (irep=i)
} %>% bind_rows

I wonder if someone can help me to figure out why it failed.  

Finally, a follow up question is if someone can show me the appropriate way to set up parallel  simulation (i.e., set seeds etc.) when OMEGA and SIGMA are sampled in mrgsolve simulation under Windows OS.  I saw some well documented examples in Kyle's training, but seems they were done under Linux OS.

Regards,
Yaming

[Kyle Baron]

Yaming - 

Can you try a setup like this?

https://github.com/mrgsolve/examples/blob/master/PrTS/pts.R#L273-L287

I have way more experience with mclapply than with doParallel.  But to me it seems like your setup is generally ok  ... but try to load the model shared object first thing on the worker (inside the %dopar% piece) 

It's not unusual to find that the worker nodes do not know about your model (that has been loaded as a dll).  We created a function called loadso that will just load the shared object into the current R session.  That is most likely where your error message is coming from.  Whether you need to export objects like mod3 into the cluster ... I'm not totally sure, but I usually export a bunch of stuff to be safe. 

Try the loadso(mod3) piece and let me know what happens.

Kyle

--
MetrumRG Website: http://www.metrumrg.com/opensourcetools.html
---
You received this message because you are subscribed to the Google Groups "mrgsolve" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mrgsolve+unsubscribe@metrumrg.com.
To post to this group, send email to mrgs...@metrumrg.com.
Visit this group at https://groups.google.com/a/metrumrg.com/group/mrgsolve/.

--

Kyle Baron

Metrum Research Group

860-735-7043, Ext. 202

ky...@metrumrg.com

[Kyle Baron]

As far as reproducibility goes, it looks like you also need a package called doRNG and use the %dorng% operator rather than %dopar%.

I worked out the example here:

https://github.com/mrgsolve/examples/blob/master/misc_demo/reproducible_doParallel.R

That house model is special since it is compiled into mrgsolve.  For testing the parallelization, I would recommend building your own model (in a distinct shared object) and using that to test.  I put an example into the link above that shows how you would use the house model ... but I would stay away from it if I were you.  It's just a funny setup ... there is a lot of work to get that all set up and in the right place and it's just too different than the usual / expected setup to use it in this situation.

Please let me know if this works / doesn't work.

Kyle

[Kyle Baron]

Info re: doRNG package https://cran.r-project.org/web/packages/doRNG/vignettes/doRNG.pdf

[Kyle Baron]

You might see a warning like this:

2: In file.remove(so) :
  cannot remove file 'C:/Users/MetrumRG/AppData/Local/Temp/2/RtmpI3GTql/mrgsolve-so-0.7.4.9000-x86_64-w64-mingw32/test/test-so-dac334711ec.dll', reason 'Permission denied'

That means that the dll is loaded on a worker that hasn't been properly shut down.  So it's important to call stopCluster before trying to build the model again.