Performance of ENMevaluate function with parallel option in R across different OS

[HysLee]

Dear All,

I am testing the ENMevaluate function in R with "parallel" option on mac, windows, and linux (cloud). I have observed that every time it takes more time with the parallel option enabled than when it was disabled. For example, it took around 14 minutes to run on my mac, but with either doSNOW or doParallel (4 cores), it took about 15 minutes. What could be possible reasons for this performance decrease?

Here is the code:

model_test = ENMevaluate(occ, predictors, method = "testing", occs.testing=occ_test,

algorithm = "maxent.jar", n.bg = 10000, tune.args=list(fc='LQHPT', rm=RM),

overlap = FALSE, bin.output = FALSE, clamp = TRUE, parallelType = "doSNOW",

parallel = TRUE, numCores = 4, progbar = TRUE, updateProgress = FALSE)

Additionally, I have noticed a significant speed difference between R on Mac and R on Windows and Linux. I ran the same code above on 2019 Intel Mac (2.9GHz dual-core i5) and a Dell (12th gen i7-12850hx 2.1GHz). Mac was almost  4 times faster than Dell...  Could this much of a speed difference be possible due to just the OS? And the same performance decrease issue with Dell... 

Thanks,

HLee

[Bede-Fazekas Ákos]

Dear HLee,
parallelization is not always faster than sequential processing. If you have a small amount of data, and the task takes only some minutes with sequential processing, then I don't recommend parallelization. During the additional steps (sharing objects with child processes, gathering results etc.) you might loose more time than the time you save with the parallel processing in the core step.
It is not about ENMevaluate or the OS, its about the way parallelization works. However, there are several parallel methods, some of them have less overhead. E.g. in Unix-like systems, such as Linux, forking is enabled. This method does not make copies from the objects/variables but can share the same object with the child processes.
HTH,
Ákos
________
Ákos Bede-Fazekas
Centre for Ecological Research, Hungary

--
You received this message because you are subscribed to the Google Groups "Maxent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxent+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/maxent/e988625f-42ed-4792-822e-4a6559f2082an%40googlegroups.com.