good use of multicore PC and some

[Masato]

Hello,
I am seeking a quick way to perform the permutation test of scantwo
and refineqtl, etc on a multiple processor PC
I'm wondering if there is a sample code for running r/qtl on multicore
processor.
For my understanding, `snow` is for the cluster rather than multicore
PC. I tried `pnmath` but it seemed not to using all the available CPUs
in my PC (8 core MacPro)
'multicore' package of R (and 'snow') seems to require some knowledge
for cluster, which I have little idea.
If there is an example code for multicore, snow, fork or whatever, I
would do like to know.

I have tried to divide a permutation test into 8 x n.perm=125, and run
on 8 terminals of 64bit version of R in X11.
The problem was CPUs couldn't perform 100% since the shortness of RAM.
I am adding more RAMs. I feel, however, I eventually need to figure
out the better way to use all CPUs at 100%.

Another question, I particularly would like to ask Karl, is,
will scantwo be able to perform the extended h-k regression soon?
I always use imputation method because of the robustness but my fish
genome is relatively large, so performing a permutation test of
scantwo with imp is almost impossible. hk regression would be fair but
ehk sounds the best for me.

Thank you.
Best regards,
Masato

p.s. I'm interested in the genetics of behavior and have found that I
cannot ignore epistatic interactions. r/qtl is the best to detect
them! I enjoy watching how r/qtl evolve. Thanks!

[Karl Broman]

I've only tried snow, and its use has now been implemented, for
permutation tests in scanone and scantwo, in the latest version of R/
qtl, through the argument n.cluster. On my 8-core Mac Pro, I can do
10,000 permutation replicates for scanone with Haley-Knott regression
in just 30 seconds. You can't get around the RAM issue, however.

I'll try to remember to send some example code later tonight, but if
you look within source code for the scanone function, you can see what
I've done with it.

I probably won't have the extended H-K method implemented in scantwo
this year. I've got many other things that I view as higher
priority. If you have reasonably complete genotype data and no
selective genotyping, H-K is the thing to use.

regards,
karl

[Karl Broman]

I'd said that I would send some example code. Here's code to do a
permutation test on the fake.bc data with a cluster of 8 nodes. We
first use scanone with the n.cluster argument, and then I spell out
the gory detail, which you might use to parallelize other things. (We
will be incorporating this sort of thing in upcoming versions of R/
qtl.)

data(fake.bc)
fake.bc <- calc.genoprob(fake.bc, step=1)

# requires the package rlecuyer
operm <- scanone(fake.bc, method="hk", n.perm=1000, n.cluster=8)

##############################
# doing it "by hand"
##############################
library(snow)

# set up 8 node cluster
cl <- makeCluster(8)

# set up random number generator
clusterSetupRNG(cl)

# each cluster needs to load the R/qtl package
clusterEvalQ(cl, require(qtl, quietly=TRUE))

# do the permutations
operm <- clusterCall(cl, scanone, fake.bc, method="hk", n.perm=125,
n.cluster=1)

# stop the clusters
stopCluster(cl)

# combine the results
for(i in 2:length(operm))
operm[[1]] <- c(operm[[1]], operm[[i]])

# keep just the combined results
operm <- operm[[1]]

karl

[Masato]

Hello Karl,
Thank you very much for all your comments and suggestions.
n.cluster option of scantwo (and scanone) worked perfectly.
However, since I plan to use a cluster system on my Univ, your codes
are very helpful to be ready for it.
Again, thank you for kind explanations.
Best,
Masato