R parallel instructions?

[Bob Carpenter]

Could someone write some instructions for
(1) running R in parallel and (2) extracting
output?

If you write the skeleton of R code, I'm happy
to fill in the words around it. I could probably
do (2) on my own, but I don't know how to do (1).

- Bob

[Jiqiang Guo]

I would be happy to do that.  But could I ask for what is this instruction for in particular so that I would have a better idea. 

For running in parallel in R, we only provide sflist2stanfit to combine results from run stan() in parallel.  In particular, see the notes and examples in the doc of sflist2stanfit.  So we do not care how the parallelism is done using whatever package, on clusters, or just one box.  In the example of sflis2stanfit, only an example using package parallel is given. 

> ?sflist2stanfit

For extract, the doc of extract and as.array.stanfit give a lot of details and examples.  

> ?extract

> ?as.array.stanfit 

--

Jiqiang

[Bob Carpenter]

On 11/7/12 12:38 PM, Jiqiang Guo wrote:
> I would be happy to do that. But could I ask for what is this instruction for in particular so that I would have a
> better idea.

I may be the exception, but I find it much easier
to follow a tutorial than to root through R's doc
system. One advantage is that it lets me know what
functions are there and how they're supposed to be used
and in what sequence.

> For running in parallel in R, we only provide sflist2stanfit to combine results from run stan() in parallel. In
> particular, see the notes and examples in the doc of sflist2stanfit.

I hadn't even realized you'd written a function
in RStan to do parallel sampling. I'll try to figure it
out from the doc and will let you know how it goes.

This stems in large part from my own immense frustration
with just about every aspect of R. I find its conventions
are counter to just about everything I know and believe
about programming. Today I've been struggling unsuccessfully
with ggplot2, which I find even more confusing than the
basic plot stuff, despite having read the book twice and
having it and the online doc right in front of me.

- Bob

[Jiqiang Guo]

To be clear, it is not a function to do parallel sampling.  It just combines results 

from runs of stan() regardless of they are executed in parallel or not. 

In addition, I had added a couple of lines using extract in the getting started wiki page 

and the doc of stan(). 

I think one issue for doc or tutorial is that people do not know where to begin with. 

For us, I suppose it is just to write it everywhere.  I guess I can more about parallel 

stuff to the getting started wiki page and the vignettes. 

I am not a big fan about R as I started programming from C.  My way to deal with it

is to learn the least of R that is needed for me.  For ggplot, I just mimic examples from internet

to accomplish my goal. 

--

Jiqiang

[Ben Goodrich]

There are code snippets by me and Sebastian Webber on this thread

https://groups.google.com/d/topic/stan-users/3goteHAsJGs/discussion

The main obstacle is that there are no universal instructions for doing things in parallel. For !Windows and local, using mclapply() in the parallel package is the best.

But no matter what anyone is doing, basically you just call stan() in parallel and use sflist2stanfit() to merge the result into one stanfit object.

Ben

On Wednesday, November 7, 2012 11:45:26 AM UTC-5, Bob Carpenter wrote:

[Bob Carpenter]

[I took this off group.]

On 11/7/12 1:17 PM, Jiqiang Guo wrote:
> To be clear, it is not a function to do parallel sampling. It just combines results
> from runs of stan() regardless of they are executed in parallel or not.

> In addition, I had added a couple of lines using extract in the getting started wiki page
> and the doc of stan().

Thanks -- I think that's all I needed in the way of
help.

> I think one issue for doc or tutorial is that people do not know where to begin with.
> For us, I suppose it is just to write it everywhere. I guess I can more about parallel
> stuff to the getting started wiki page and the vignettes.

I don't think we need to be that redundant in
writing how to do things. But I do think it might
help to have something more tutorial about the whole
process of fitting models than just the R doc (which I
think is at the right level of detail as is).

> I am not a big fan about R as I started programming from C. My way to deal with it
> is to learn the least of R that is needed for me.

Say it isn't so --- we need your R expertise!

> For ggplot, I just mimic examples from internet
> to accomplish my goal.

That works for me, too, until I want to produce a print-ready
.png file with legible labels (the trick is to specify sizes
and resolution -- I find height=1200 and res=300 works well when
shrunk down to height=2in in LaTeX includes).

- Bob