sharing code in Py/R/CmdStan: effective_sample_size

[Allen Riddell]

Hi,

There are many opportunities for eliminating extra implementations of identical
functionality in PyStan/RStan/CmdStan. For example, there are three
implementations of effective_sample_size (and split_potential_scale_reduction).

I'd like to get that number down to two by eliminating the version in PyStan
and just using the functions in stan/mcmc/chains.hpp from Python.

Python's Rcpp, called Cython, handles C++ vectors without any trouble. So the
main challenge is just what to do about moving between a C++ vector and an Eigen
VectorXd. As this seems like it might be a general concern, I thought I'd raise
the question of an appropriate strategy on the list.

One thought I had is that in the areas where a simple conversion is required,
Stan might expose a function that accepts a vector in addition to a version
that accepts a VectorXd.

Thanks,

-a

[Ido Rosen]

Hi,

On Tue, Jan 28, 2014 at 3:20 PM, Allen Riddell <a...@ariddell.org> wrote:

Python's Rcpp, called Cython, handles C++ vectors without any trouble. So the
main challenge is just what to do about moving between a C++ vector and an Eigen
VectorXd. As this seems like it might be a general concern, I thought I'd raise
the question of an appropriate strategy on the list.

One thought I had is that in the areas where a simple conversion is required,
Stan might expose a function that accepts a vector in addition to a version
that accepts a VectorXd.

An alternative to converting to std::vector<> would be to use minieigen[1], pyeigen[2], or similar Eigen bindings for Python, which provide similar functionality to RcppEigen[3], which provides R bindings for Eigen.  (These are broader than just VectorXd, and may not require a copy to an STL vector, which could save time for large vectors.)

Last I checked, the two Python bindings I mentioned were still under development, but they may have matured by now.  Even if they haven't fully matured, the VectorXd bindings/parts are probably stable.  I've used RcppEigen and it's pretty stable.

Sorry in advance if you've already considered this and I'm just stating the obvious. :-)

Ido

[1] https://pypi.python.org/pypi/minieigen

[2] https://launchpad.net/pyeigen

[3] http://cran.r-project.org/web/packages/RcppEigen/index.html

[Michael Betancourt]

Allen, it’s straightforward to provide vector wrappers to these functions without
any overhead using Eigen maps to avoid any copying. Can you list the functions
which you’d like to have vector signatures? All Git markup style if possible.

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Allen Riddell]

On 01/28, Michael Betancourt wrote:
> Allen, it's straightforward to provide vector wrappers to these functions without
> any overhead using Eigen maps to avoid any copying. Can you list the functions
> which you'd like to have vector signatures? All Git markup style if possible.
>

Thanks Michael.

My sense is that this is indeed the easiest route. I imagine working with
vectors would be easier in Rcpp as well.

Let me get the code working before I pin down the specific function signatures
that are needed. I just wanted to float this idea before working on it...

Best, -a

[Michael Betancourt]

Let me get the code working before I pin down the specific function signatures
that are needed. I just wanted to float this idea before working on it...

No worries — just trying to leverage GitHub for all its worth.  Also circumvent

our notoriously poor memories.  :-)

[Bob Carpenter]

This was, by the way, one of the reasons I thought
std::vector interfaces would be easier and more portable.

What Michael's suggesting is probably the path of least
resistance and shouldn't be inefficient (other than to
compile).

A more idiomatic approach within C++ would be to define the
interfaces in terms of iterators. A more generic way to do
it with references to container arguments would be to template
out the container type and just use accessors (for example, []
works for arrays, standard vectors, C++11 arrays, Eigen vectors, and
many of the Boost sequence containers). The cost is that the code's a
bit more complex, which is why we've (or at least I've) tended not
to do it that way. But I think we're all getting better at C++,
so maybe this shouldn't be such a hurdle any more.

- Bob

[Allen Riddell]

Thanks for the references Ido. I've got to try out boost::python one of these
days.

There's also this new project Ceygen which looks interesting:
https://github.com/strohel/Ceygen .

Best wishes,

-a