Parallel chains error with nimble function

[Chris Bodily]

Hi All,

I am simply trying to use nimble to speed up a custom function.  I created a nimble function that compiles successfully and is much faster that just using R code.  When I try use the compiled function with the parallel package, I receive an error:  Error in checkForRemoteErrors(val) :   4 nodes produced errors; first error: NULL value passed as symbol address

I created a simpler example with a test function that generates the same error and I hope someone can help me see what I am missing here.  I'm running R 4.4.1 on Windows 10.

Thanks

Chris

library(nimble)
library(parallel)

myPower <- nimbleFunction(
  run = function(x = double(0, default = 1), power=double(0, default=2)) {
    returnType(double(0))
    return(x^power)
  })
myPowerc <- compileNimble(myPower)

cl <- makeCluster(getOption("cl.cores", 4))
res <- parLapply(cl,split(1:100,1:100),myPowerc,power=2)
stopCluster(cl)

[Daniel Turek]

Chris, great question.  Unfortunately, with the present system of nimble's compilation, you'll need each process to have its own version of the compiled nimbleFunction.  So the approach can still be used, but has to be modified something akin to:

library(parallel)

my_parallel_function <- function(x, power) {
    library(nimble)
    myPower <- nimbleFunction(
        run = function(x=double(0, default=1), power=double(0, default=2)) {

            returnType(double(0))
            return(x^power)
        })
    myPowerc <- compileNimble(myPower)

    myPowerc(x, power)
}

cl <- makeCluster(4)
res <- parLapply(cl, 1:100, my_parallel_function, power = 2)
stopCluster(cl)

Please let me know if this helps you get started.

Cheers,

Daniel

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/678a8f7d-d758-4199-89ea-b2f767cda03an%40googlegroups.com.