Refactoring the Stan repository

[Daniel Lee]

Currently, the stan-dev/stan repository has multiple conceptual pieces inside it. Bob and I discussed this for a while yesterday and decided it would be good to split out the independent pieces.

This is how I'd like to split it apart:

• stan-dev/stan: the core Stan library
  
• C++ API
• Stan reference manual
• Most of the Stan compiler (stanc). The parts that live in src/stan/gm/ should stay.
  
• stan-dev/cmdstan: the command line interface to Stan
  
• CmdStan will be the command line / shell interface to Stan to mirror RStan and PyStan.
• bin/print
• bin/stanc
• CmdStan will have its own getting started guide.
  
• stan-dev/webpages: the webpages for mc-stan.org
• stan-dev/example-models: Stan models and data
  
• BUGS examples
• ARM models
• other models
• possibly some descriptive text, maybe we can figure out how to put down stable values so each of the interfaces {R, Py, Cmd}Stan can test against the models

Some reasons for doing this:

• CmdStan should be decoupled from the main Stan library.
• {R, Py, Cmd}Stan can be released independently of upgrades to the Stan library.
  
• Testing times can be reduced since there would be clear separation from unit tests and higher level tests.
• Webpages can be uploaded independently without having to freeze and update Stan versions.
• Makefiles can be simplified (I think).

If anyone has any thoughts, please send them out.

Daniel

[Michael Betancourt]

How will CmdStan be refactored since it's currently implemented directly in the model executable?

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Bob Carpenter]

On 10/18/13 12:22 PM, Michael Betancourt wrote:
> How will CmdStan be refactored since it's currently implemented directly in the model executable?

I've been thinking for a while that it'd be nice to have
bin/stanc just compile the model and not the command.

I'd like to have the model compile w/o a main() in it.
It's not really part of the model concept, but rather
something that uses it.

If we make the model dynamically linkable, we can then compile
the command once and reuse it.

Otherwise, we can statically generate a new command with
the name of the model class in it and compile that and link
it to a .o for the model.

- Bob

> On Fri, Oct 18, 2013 at 5:11 PM, Daniel Lee <bea...@alum.mit.edu <mailto:bea...@alum.mit.edu>> wrote:
>
> Currently, the stan-dev/stan repository has multiple conceptual pieces inside it. Bob and I discussed this for a
> while yesterday and decided it would be good to split out the independent pieces.
>
> This is how I'd like to split it apart:
>

> * stan-dev/stan: the core Stan library
> o C++ API
> o Stan reference manual
> o Most of the Stan compiler (stanc). The parts that live in src/stan/gm/ should stay.
> * stan-dev/cmdstan: the command line interface to Stan
> o CmdStan will be the command line / shell interface to Stan to mirror RStan and PyStan.
> o bin/print
> o bin/stanc
> o CmdStan will have its own getting started guide.
> * stan-dev/webpages: the webpages for mc-stan.org <http://mc-stan.org>
> * stan-dev/example-models: Stan models and data
> o BUGS examples
> o ARM models
> o other models
> o possibly some descriptive text, maybe we can figure out how to put down stable values so each of the

> interfaces {R, Py, Cmd}Stan can test against the models
>
> Some reasons for doing this:
>

> * CmdStan should be decoupled from the main Stan library.
> * {R, Py, Cmd}Stan can be released independently of upgrades to the Stan library.
> * Testing times can be reduced since there would be clear separation from unit tests and higher level tests.
> * Webpages can be uploaded independently without having to freeze and update Stan versions.
> * Makefiles can be simplified (I think).

>
> If anyone has any thoughts, please send them out.
>
>
> Daniel
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+u...@googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

[Michael Betancourt]

If we make the model dynamically linkable, we can then compile
the command once and reuse it.

Will that work with all of the templating?

 

Otherwise, we can statically generate a new command with
the name of the model class in it and compile that and link
it to a .o for the model.

In either case it'll still be a single call to make and a single executable, so the user-experience doesn't change? 

[Daniel Lee]

Yes. Absolutely.

[Michael Betancourt]

Cool, sounds good to me.

[Bob Carpenter]

On 10/18/13 12:32 PM, Michael Betancourt wrote:
> If we make the model dynamically linkable, we can then compile
> the command once and reuse it.
>
>
> Will that work with all of the templating?

We'd have to provide the instantiations explicitly. But there
are only a few we ever use.

This is being done right now by the main(). All the template instantiations
called by the main get built.

> Otherwise, we can statically generate a new command with
> the name of the model class in it and compile that and link
> it to a .o for the model.
>
>
> In either case it'll still be a single call to make and a single executable, so the user-experience doesn't change?

Absolutely.

- Bob

[Ben Goodrich]

On Friday, October 18, 2013 12:11:52 PM UTC-4, Daniel Lee wrote:

stan-dev/cmdstan: the command line interface to Stan

• CmdStan will be the command line / shell interface to Stan to mirror RStan and PyStan.
• bin/print
• bin/stanc
• CmdStan will have its own getting started guide.
  

The same thing could be said for splitting RStan into two packages: one that does parsing and compilation of models and one that does everything else (i.e. has the stanfit class definition, etc.) It would then be possible to distribute binary versions of the everything else package and rstanarm for people who are not at the stage where they can write their own Stan models but could do precompiled models or open the results of a model someone else does.

Ben

[Bob Carpenter]

On 10/18/13 2:02 PM, Ben Goodrich wrote:
> On Friday, October 18, 2013 12:11:52 PM UTC-4, Daniel Lee wrote:
>
> stan-dev/cmdstan: the command line interface to Stan
>

> o CmdStan will be the command line / shell interface to Stan to mirror RStan and PyStan.
> o bin/print
> o bin/stanc

> o CmdStan will have its own getting started guide.

>
> The same thing could be said for splitting RStan into two packages: one that does parsing and compilation of models and
> one that does everything else (i.e. has the stanfit class definition, etc.) It would then be possible to distribute
> binary versions of the everything else package and rstanarm for people who are not at the stage where they can write
> their own Stan models but could do precompiled models or open the results of a model someone else does.

The actual parsing code's going to be in the stan-dev/stan repository.
So RStan's already split this way.

One way I'd like to see RStan factored is in terms of a non-Stan-dependent
output analaysis package like Coda and everything else.

- Bob