stan and distributed computing

[Sachinthaka Abeywardana]

Apologies if I'm asking a far too general question but I was wondering if there is a guide/ tutorial to using rstan on AWS, or atleast a linux cluster. The important outcome I'm hoping for is to use MORE than one machine. (I'm assuming that rstan already uses all available cores).

So far this is the best I could find but doesn't really go through examples:

http://www.r-bloggers.com/set-up-rstan-on-amazon-ec2/

[Bob Carpenter]

Not that we distribute. Each cluster works differently, so
it's hard to write something general.

Let us know if you find something written up that works.

- Bob

On Nov 30, 2015, at 11:27 PM, Sachinthaka Abeywardana <sachin.ab...@gmail.com> wrote:
>
> Apologies if I'm asking a far too general question but I was wondering if there is a guide/ tutorial to using rstan on AWS, or atleast a linux cluster.
>

> So far this is the best I could find but doesn't really go through examples:
> http://www.r-bloggers.com/set-up-rstan-on-amazon-ec2/
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[David Manheim]

The r-bloggers tutorial seems like a good way to set up a single remote EC2 machine with RStan. Following that, you can use any of the regular Stan models via command line RStan on that remote machine - it's not distributing the analysis among multiple machines, though - that would be harder.

You can then run one chain per EC2 computer, and combine them afterwards; how to do this is detailed here; https://groups.google.com/d/msg/stan-users/c8wb87yQkPc/Cb7mWSDgeWkJ (The task of sampling each chain doesn't parallelize, at least in the current architecture,, so it's not really useful to have a larger network except to sample more chains.)

Cheers,

David

[Bob Carpenter]

> On Dec 1, 2015, at 4:01 PM, David Manheim <davidm...@gmail.com> wrote:
>
> The r-bloggers tutorial seems like a good way to set up a single remote EC2 machine with RStan. Following that, you can use any of the regular Stan models via command line RStan on that remote machine - it's not distributing the analysis among multiple machines, though - that would be harder.
>
> You can then run one chain per EC2 computer, and combine them afterwards; how to do this is detailed here; https://groups.google.com/d/msg/stan-users/c8wb87yQkPc/Cb7mWSDgeWkJ (The task of sampling each chain doesn't parallelize, at least in the current architecture,, so it's not really useful to have a larger network except to sample more chains.)

Or more than one model.

Or to do work on a more powerful machine, depending
on what kind of EC2 instance you rent and what you have
locally.

- Bob

[David Manheim]

Ok, yes and yes.