Re: Higher-Order Autodiff Signatures

[Michael Betancourt]

Bob,

I'm getting a weird template error, my struct is defined as

template <typename M>
struct _softabs_fun {

const M& _model;
std::ostream* _o;

_softabs_fun(M& m, std::ostream* out): _model(m), _o(out) {};

template <typename T>
T operator()(const Eigen::Matrix<T,Eigen::Dynamic,1>& x) const {
Eigen::Map<Eigen::Matrix<T,Eigen::Dynamic,1> > eigen_x(&(x[0]), x.size());
std::vector<int> dummy_int;
return _model.log_prob<true,true,T>(eigen_x, dummy_int, _o);
}
};

and when I try to compile with a toy model I get the two errors:

src/stan/mcmc/hmc/hamiltonians/softabs_metric.hpp:28:41: error: expected ';' after return statement
return _model.log_prob<true,true,T>(eigen_x, dummy_int, _o);
^
;
src/stan/mcmc/hmc/hamiltonians/softabs_metric.hpp:26:56: error: no matching constructor for initialization of
'Eigen::Map<Eigen::Matrix<fvar<var>, Eigen::Dynamic, 1> >'
Eigen::Map<Eigen::Matrix<T,Eigen::Dynamic,1> > eigen_x(&(x[0]), x.size());
^ ~~~~~~~~~~~~~~~~~
src/stan/agrad/autodiff.hpp:172:29: note: in instantiation of function template specialization
'stan::mcmc::_softabs_fun<rhmc_test_model_namespace::rhmc_test_model>::operator()<stan::agrad::fvar<stan::agrad::var> >' requested
here
fvar<var> fx_fvar = f(x_fvar);


Not sure why the Eigen::Map is failing, and I'm even more clueless as to why the compiler thinks that log_prob only has to template arguments.

On Oct 13, 2013, at 8:28 PM, Bob Carpenter <ca...@alias-i.com> wrote:

> Can we put this back on stan-dev? I added Daniel and
> Peter to this thread as they're involved in the
> implementation.
>
> You don't want to set T__. You should be able to get
> away with something like this:
>
> template <typename M>
> struct my_fun {
> const M& model_;
> std::ostream& o_;
>
> ...obvious ctor...
>
> template <typename T>
> T operator()(const Eigen::Matrix<T,Eigen::Dynamic,1>& x) const {
> ...copy x to std::vector<T> v;...
> ...crate dummy empty int vector... (or you could put it in the class)
> return model_.log_prob<true,true,T>(v,dummy_int,o_);
> }
> }
>
> Then you pass in my_fun<ModelType>(model,output_stream) as the functional.
> (The output stream can be 0).
>
> If we change the underlying type of the parameter vector to use
> Eigen, we wouldn't need to copy. I wrote this anticipating that was
> our plan. But that's going to be the least of our worries speed-wise.
> That would also allow this all to be simplified to a bind (lambda)
> operation.
>
> - Bob
>
>
> On 10/12/13 6:00 PM, Michael Betancourt wrote:
>> So to compute the functionals would I just have to wrap
>>
>> template <bool propto__, bool jacobian__, typename T__>
>> T__ log_prob(vector<T__>& params_r__,
>> vector<int>& params_i__,
>> std::ostream* pstream__ = 0) const {
>>
>> in a struct, setting T__ to fvar<var>?
>>
>> On Oct 12, 2013, at 8:42 PM, Bob Carpenter <ca...@alias-i.com> wrote:
>>
>>> You can build up everything with just forward and
>>> reverse mode. The signatures your requested for RMHMC
>>> are all implemented as functionals in:
>>>
>>> src/stan/agrad/autodiff.hpp
>>>
>>> Examples of usage are in:
>>>
>>> src/test/agrad/autodiff_test.cpp
>>

[Bob Carpenter]

The model log prob's defined for std::vector only. The
refactoring never made it all the way through to the model.

The signature looks like this:

template <bool propto__, bool jacobian__, typename T__>
T__ log_prob(vector<T__>& params_r__,
vector<int>& params_i__,

std::ostream* pstream__ = 0) const;

And you shouldn't need the T in the call to log_prob in
the definition of operator() --- it should be inferrable from
the argument, which should be std::vector<T> --- but it
doesn't hurt to include it.

- Bob

[Michael Betancourt]

Gah, I mapped in the wrong direction! Thanks.

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Bob Carpenter]

I should've just implemented an example.

We can continue the refactor to Eigen vectors when
2.0 comes out. Then it'll be even easier.

- Bob

[Michael Betancourt]

I do not understand C++ one bit.

The class-functor:

    template <typename M>

    class _softabs_fun {

      

    public:

      

      _softabs_fun(M& m, std::ostream* out): _model(m), _o(out) {};

      

      template <typename T>

      T operator()(const Eigen::Matrix<T,Eigen::Dynamic,1>& x) const {

        

        std::vector<T> v(x.size());

        Eigen::Map<Eigen::Matrix<T,Eigen::Dynamic,1> >(&(v[0]), v.size()) = x;

        std::vector<int> dummy_int;

        

        return _model.log_prob<true,true,T>(v, dummy_int, _o);

      }

      

    private:

      

      const M& _model;

      std::ostream* _o;

      

    };

results in the errors:

src/stan/mcmc/hmc/hamiltonians/softabs_metric.hpp:30:41: error: expected ';' after return statement

        return _model.log_prob<true,true,T>(v, dummy_int, _o);

                                        ^

                                        ;

src/stan/mcmc/hmc/hamiltonians/softabs_metric.hpp:30:16: error: reference to non-static member function must be called

        return _model.log_prob<true,true,T>(v, dummy_int, _o);

I get that T in the log_prob instantiation can be inferred but I don't get why it errs out if I specify it explicitly.

Secondly, I have no idea why the compiler is can't reference the explicit model object.  Other model method work fine, it's just log_prob that errs.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

[Bob Carpenter]

- Bob

On 10/14/13 10:32 AM, Michael Betancourt wrote:
> I do not understand C++ one bit.

It's pretty complicated! And rather ad hoc around the edges.

>
> The class-functor:
>
> template <typename M>
>
> class _softabs_fun {
>
> public:
>
> _softabs_fun(M& m, std::ostream* out): _model(m), _o(out) {};
>
> template <typename T>
>
> T operator()(const Eigen::Matrix<T,Eigen::Dynamic,1>& x) const {
>
> std::vector<T> v(x.size());
>
> Eigen::Map<Eigen::Matrix<T,Eigen::Dynamic,1> >(&(v[0]), v.size()) = x;
>
> std::vector<int> dummy_int;
>
> return _model.log_prob<true,true,T>(v, dummy_int, _o);
>
>
> }
>
> private:
>
> const M& _model;
>
> std::ostream* _o;
>
> };
>
>
> results in the errors:
>
>
> src/stan/mcmc/hmc/hamiltonians/softabs_metric.hpp:30:41: error: expected ';' after return statement
> return _model.log_prob<true,true,T>(v, dummy_int, _o);
> ^
> ;
> src/stan/mcmc/hmc/hamiltonians/softabs_metric.hpp:30:16: error: reference to non-static member function must be called
> return _model.log_prob<true,true,T>(v, dummy_int, _o);
>
> I get that T in the log_prob instantiation can be inferred but I don't get why it errs out if I specify it explicitly.
>
> Secondly, I have no idea why the compiler is can't reference the explicit model object. Other model method work fine,
> it's just log_prob that errs.

No idea what the problem is yet. I'll try to get something working
ASAP and send you the code.

I want my RMHMC!

- Bob

[Ben Goodrich]

Is this in some branch? I can try to compile it under Linux to see if there is a better error message. -- Ben

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Bob Carpenter]

I can take care of this issue.

The autodiff functions are in develop. Don't know where
Michael's working.

- Bob

> On Oct 14, 2013, at 9:12 AM, Bob Carpenter <ca...@alias-i.com <javascript:>> wrote:
>
> The model log prob's defined for std::vector only. The
> refactoring never made it all the way through to the model.
>
> The signature looks like this:
>
> template <bool propto__, bool jacobian__, typename T__>
> T__ log_prob(vector<T__>& params_r__,
> vector<int>& params_i__,
> std::ostream* pstream__ = 0) const;
>
> And you shouldn't need the T in the call to log_prob in
> the definition of operator() --- it should be inferrable from
> the argument, which should be std::vector<T> --- but it
> doesn't hurt to include it.
>
> - Bob
>
> On 10/13/13 5:37 PM, Michael Betancourt wrote:
>
> Bob,
>
> I'm getting a weird template error, my struct is defined as
>
> template <typename M>
> struct _softabs_fun {
>
> const M& _model;
> std::ostream* _o;
>
> _softabs_fun(M& m, std::ostream* out): _model(m), _o(out) {};
>
> template <typename T>

> T operator()(const Eigen::Matrix<T,Eigen::__Dynamic,1>& x) const {
> Eigen::Map<Eigen::Matrix<T,__Eigen::Dynamic,1> > eigen_x(&(x[0]), x.size());
> std::vector<int> dummy_int;
> return _model.log_prob<true,true,T>(__eigen_x, dummy_int, _o);

> }
> };
>
> and when I try to compile with a toy model I get the two errors:
>

> src/stan/mcmc/hmc/__hamiltonians/softabs_metric.__hpp:28:41: error: expected ';' after return statement
> return _model.log_prob<true,true,T>(__eigen_x, dummy_int, _o);
> ^
> ;
> src/stan/mcmc/hmc/__hamiltonians/softabs_metric.__hpp:26:56: error: no matching constructor for
> initialization of
> 'Eigen::Map<Eigen::Matrix<__fvar<var>, Eigen::Dynamic, 1> >'
> Eigen::Map<Eigen::Matrix<T,__Eigen::Dynamic,1> > eigen_x(&(x[0]), x.size());
> ^ ~~~~~~~~~~~~~~~~~
> src/stan/agrad/autodiff.hpp:__172:29: note: in instantiation of function template specialization
>
> 'stan::mcmc::_softabs_fun<__rhmc_test_model_namespace::__rhmc_test_model>::operator()<__stan::agrad::fvar<stan::agrad:__:var> >' requested

> here
> fvar<var> fx_fvar = f(x_fvar);
>
>
> Not sure why the Eigen::Map is failing, and I'm even more clueless as to why the compiler thinks
> that log_prob only has to template arguments.
>

> On Oct 13, 2013, at 8:28 PM, Bob Carpenter <ca...@alias-i.com <javascript:>> wrote:
>
> Can we put this back on stan-dev? I added Daniel and
> Peter to this thread as they're involved in the
> implementation.
>
> You don't want to set T__. You should be able to get
> away with something like this:
>
> template <typename M>
> struct my_fun {
> const M& model_;
> std::ostream& o_;
>
> ...obvious ctor...
>
> template <typename T>

> T operator()(const Eigen::Matrix<T,Eigen::__Dynamic,1>& x) const {

> ...copy x to std::vector<T> v;...
> ...crate dummy empty int vector... (or you could put it in the class)

> return model_.log_prob<true,true,T>(__v,dummy_int,o_);
> }
> }
>
> Then you pass in my_fun<ModelType>(model,__output_stream) as the functional.

> (The output stream can be 0).
>
> If we change the underlying type of the parameter vector to use
> Eigen, we wouldn't need to copy. I wrote this anticipating that was
> our plan. But that's going to be the least of our worries speed-wise.
> That would also allow this all to be simplified to a bind (lambda)
> operation.
>
> - Bob
>
>
> On 10/12/13 6:00 PM, Michael Betancourt wrote:
>
> So to compute the functionals would I just have to wrap
>
> template <bool propto__, bool jacobian__, typename T__>
> T__ log_prob(vector<T__>& params_r__,
> vector<int>& params_i__,
> std::ostream* pstream__ = 0) const {
>
> in a struct, setting T__ to fvar<var>?
>

> On Oct 12, 2013, at 8:42 PM, Bob Carpenter <ca...@alias-i.com <javascript:>> wrote:
>
> You can build up everything with just forward and
> reverse mode. The signatures your requested for RMHMC
> are all implemented as functionals in:
>
> src/stan/agrad/autodiff.hpp
>
> Examples of usage are in:
>

> src/test/agrad/autodiff_test.__cpp

>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing
> list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+u...@__googlegroups.com <javascript:>.
> For more options, visit https://groups.google.com/__groups/opt_out
> <https://groups.google.com/groups/opt_out>.

>
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+u...@__googlegroups.com <javascript:>.
> For more options, visit https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>.

[Michael Betancourt]

feature/rhmc

[Bob Carpenter]

OK, as soon as I sat down to write this, I realized what the
problem was. You need this:

template <typename M>
struct log_prob_functional {
const M& m_;
std::ostream& o_;
log_prob_functional(const M& m, std::ostream& o)
: m_(m), o_(o) { }
template <typename T>
T operator()(const std::vector<T>& theta) {
std::vector<int> ns;
return m_.template log_prob<true,true,T>(theta,ns,o_);
}
};

Note the absolutely absurd syntax "m_.template log_prob<true,true,T>".
C++ syntax is full of the need for these odd qualifications of what
seems obvious from looking at the expression, here that the function's
a template function!

- Bob

[Peter Li]

I'm not sure what functions you're using, but some of the stuff in the develop branch won't work for fvar<fvar<var>> or fvar<var>--I don't remember which ones, but everything should be fixed in the feature/clean_up_fvar branch at least on macs.

[Bob Carpenter]

I believe the reason Peter's patches haven't been merged is
that the tests don't run under Windows --- it blows out stack
trying to compile the higher-order derivative code.

Of course, this means that we won't be able to run RMHMC
on Windows, either, but I'm OK with that.

I'd like to figure out how to merge in Peter's fixes,
though. Can we conditionally not run some tests on Windows?

- Bob

[Daniel Lee]

On Mon, Oct 14, 2013 at 4:12 PM, Bob Carpenter <ca...@alias-i.com> wrote:

I believe the reason Peter's patches haven't been merged is
that the tests don't run under Windows --- it blows out stack
trying to compile the higher-order derivative code.

Of course, this means that we won't be able to run RMHMC
on Windows, either, but I'm OK with that.

I'd like to figure out how to merge in Peter's fixes,
though.   Can we conditionally not run some tests on Windows?

Yes, can we take care of this first thing after v2.0.0 is out?

 

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+unsubscribe@googlegroups.com.

[Daniel Lee]

I want to restructure our tests and this will be a good time to separate out what should and shouldn't be tested on Windows.

[Bob Carpenter]

Yup. I wasn't suggesting doing it before 2.0!

Michael will have to merge in another branch before then to
get the up-to-date fvar implementations.

- Bob

On 10/14/13 4:15 PM, Daniel Lee wrote:

> On Mon, Oct 14, 2013 at 4:12 PM, Bob Carpenter <ca...@alias-i.com <mailto:ca...@alias-i.com>> wrote:
>
> I believe the reason Peter's patches haven't been merged is
> that the tests don't run under Windows --- it blows out stack
> trying to compile the higher-order derivative code.
>
> Of course, this means that we won't be able to run RMHMC
> on Windows, either, but I'm OK with that.
>
> I'd like to figure out how to merge in Peter's fixes,
> though. Can we conditionally not run some tests on Windows?
>
>
>
> Yes, can we take care of this first thing after v2.0.0 is out?
>
>
> - Bob
>
> On 10/14/13 2:57 PM, Peter Li wrote:
>
> I'm not sure what functions you're using, but some of the stuff in the develop branch won't work for
> fvar<fvar<var>> or fvar<var>--I don't remember which ones, but everything should be fixed in the
> feature/clean_up_fvar branch at least on macs.
>
> On Oct 14, 2013, at 2:54 PM, Michael Betancourt wrote:
>
> feature/rhmc
>

> On Oct 14, 2013, at 7:27 PM, Bob Carpenter <ca...@alias-i.com <mailto:ca...@alias-i.com>> wrote:
>
> I can take care of this issue.
>
> The autodiff functions are in develop. Don't know where
> Michael's working.
>
> - Bob
>
> On 10/14/13 2:03 PM, Ben Goodrich wrote:
>
> Is this in some branch? I can try to compile it under Linux to see if there is a better error
> message. -- Ben
>
> On Monday, October 14, 2013 10:32:54 AM UTC-4, Michael Betancourt wrote:
>
> I do not understand C++ one bit.
>
> The class-functor:
>
> template <typename M>
>
> class _softabs_fun {
>
> public:
>
> _softabs_fun(M& m, std::ostream* out): _model(m), _o(out) {};
>
> template <typename T>
>

> T operator()(const Eigen::Matrix<T,Eigen::__Dynamic,1>& x) const {
>
> std::vector<T> v(x.size());
>
> Eigen::Map<Eigen::Matrix<T,__Eigen::Dynamic,1> >(&(v[0]), v.size()) = x;
>
> std::vector<int> dummy_int;
>
> return _model.log_prob<true,true,T>(__v, dummy_int, _o);

>
>
> }
>
> private:
>
> const M& _model;
>
> std::ostream* _o;
>
> };
>
>
> results in the errors:
>
>

> src/stan/mcmc/hmc/__hamiltonians/softabs_metric.__hpp:30:41: error: expected ';' after return
> statement
> return _model.log_prob<true,true,T>(__v, dummy_int, _o);
> ^
> ;
> src/stan/mcmc/hmc/__hamiltonians/softabs_metric.__hpp:30:16: error: reference to non-static

> member function must be called

> return _model.log_prob<true,true,T>(__v, dummy_int, _o);

>
> I get that T in the log_prob instantiation can be inferred but I don't get why it errs out if I
> specify it explicitly.
>
> Secondly, I have no idea why the compiler is can't reference the explicit model object. Other
> model method work
> fine, it's just log_prob that errs.
>
>
>
>

> On Mon, Oct 14, 2013 at 10:03 AM, Bob Carpenter <ca...@alias-i.com <mailto:ca...@alias-i.com>

> <javascript:>> wrote:
>
> I should've just implemented an example.
>
> We can continue the refactor to Eigen vectors when
> 2.0 comes out. Then it'll be even easier.
>
> - Bob
>
>
> On 10/14/13 4:24 AM, Michael Betancourt wrote:
>
> Gah, I mapped in the wrong direction! Thanks.
>

> On Oct 14, 2013, at 9:12 AM, Bob Carpenter <ca...@alias-i.com <mailto:ca...@alias-i.com>

> <javascript:>> wrote:
>
> The model log prob's defined for std::vector only. The
> refactoring never made it all the way through to the model.
>
> The signature looks like this:
>
> template <bool propto__, bool jacobian__, typename T__>
> T__ log_prob(vector<T__>& params_r__,
> vector<int>& params_i__,
> std::ostream* pstream__ = 0) const;
>
> And you shouldn't need the T in the call to log_prob in
> the definition of operator() --- it should be inferrable from
> the argument, which should be std::vector<T> --- but it
> doesn't hurt to include it.
>
> - Bob
>
> On 10/13/13 5:37 PM, Michael Betancourt wrote:
>
> Bob,
>
> I'm getting a weird template error, my struct is defined as
>
> template <typename M>
> struct _softabs_fun {
>
> const M& _model;
> std::ostream* _o;
>
> _softabs_fun(M& m, std::ostream* out): _model(m), _o(out) {};
>
> template <typename T>

> T operator()(const Eigen::Matrix<T,Eigen::____Dynamic,1>& x) const {
> Eigen::Map<Eigen::Matrix<T,____Eigen::Dynamic,1> > eigen_x(&(x[0]),

> x.size());
> std::vector<int> dummy_int;

> return _model.log_prob<true,true,T>(____eigen_x, dummy_int, _o);

> }
> };
>
> and when I try to compile with a toy model I get the two errors:
>

> src/stan/mcmc/hmc/____hamiltonians/softabs_metric.____hpp:28:41: error: expected
> ';' after return statement
> return _model.log_prob<true,true,T>(____eigen_x, dummy_int, _o);
> ^
> ;
> src/stan/mcmc/hmc/____hamiltonians/softabs_metric.____hpp:26:56: error: no

> matching constructor for
> initialization of

> 'Eigen::Map<Eigen::Matrix<____fvar<var>, Eigen::Dynamic, 1> >'
> Eigen::Map<Eigen::Matrix<T,____Eigen::Dynamic,1> > eigen_x(&(x[0]),
> x.size());
> ^ ~~~~~~~~~~~~~~~~~
> src/stan/agrad/autodiff.hpp:____172:29: note: in instantiation of function
> template specialization
>
>
> 'stan::mcmc::_softabs_fun<____rhmc_test_model_namespace::____rhmc_test_model>::operator()<____stan::agrad::fvar<stan::__agrad:__:var> >' requested

> here
> fvar<var> fx_fvar = f(x_fvar);
>
>
> Not sure why the Eigen::Map is failing, and I'm even more clueless as to why the
> compiler thinks
> that log_prob only has to template arguments.
>
> On Oct 13, 2013, at 8:28 PM, Bob Carpenter <ca...@alias-i.com

> <mailto:ca...@alias-i.com> <javascript:>> wrote:
>
> Can we put this back on stan-dev? I added Daniel and
> Peter to this thread as they're involved in the
> implementation.
>
> You don't want to set T__. You should be able to get
> away with something like this:
>
> template <typename M>
> struct my_fun {
> const M& model_;
> std::ostream& o_;
>
> ...obvious ctor...
>
> template <typename T>

> T operator()(const Eigen::Matrix<T,Eigen::____Dynamic,1>& x) const {

> ...copy x to std::vector<T> v;...
> ...crate dummy empty int vector... (or you could put it in the class)

> return model_.log_prob<true,true,T>(____v,dummy_int,o_);
> }
> }
>
> Then you pass in my_fun<ModelType>(model,____output_stream) as the functional.

> (The output stream can be 0).
>
> If we change the underlying type of the parameter vector to use
> Eigen, we wouldn't need to copy. I wrote this anticipating that was
> our plan. But that's going to be the least of our worries speed-wise.
> That would also allow this all to be simplified to a bind (lambda)
> operation.
>
> - Bob
>
>
> On 10/12/13 6:00 PM, Michael Betancourt wrote:
>
> So to compute the functionals would I just have to wrap
>
> template <bool propto__, bool jacobian__, typename T__>
> T__ log_prob(vector<T__>& params_r__,
> vector<int>& params_i__,
> std::ostream* pstream__ = 0) const {
>
> in a struct, setting T__ to fvar<var>?
>
> On Oct 12, 2013, at 8:42 PM, Bob Carpenter <ca...@alias-i.com

> <mailto:ca...@alias-i.com> <javascript:>> wrote:
>
> You can build up everything with just forward and
> reverse mode. The signatures your requested for RMHMC
> are all implemented as functionals in:
>
> src/stan/agrad/autodiff.hpp
>
> Examples of usage are in:
>

> src/test/agrad/autodiff_test.____cpp

>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan
> development mailing
> list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+u...@__googlegroups.__com <http://googlegroups.com> <javascript:>.
> For more options, visit https://groups.google.com/____groups/opt_out
> <https://groups.google.com/__groups/opt_out>

> <https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>>.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development
> mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+u...@__googlegroups.__com <http://googlegroups.com> <javascript:>.
> For more options, visit https://groups.google.com/____groups/opt_out
> <https://groups.google.com/__groups/opt_out> <https://groups.google.com/__groups/opt_out

> <https://groups.google.com/groups/opt_out>>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing
> list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+unsubscribe@__googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/__groups/opt_out
> <https://groups.google.com/groups/opt_out>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing
> list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+unsubscribe@__googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/__groups/opt_out
> <https://groups.google.com/groups/opt_out>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+unsubscribe@__googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to

> stan-dev+unsubscribe@__googlegroups.com <mailto:stan-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/__groups/opt_out <https://groups.google.com/groups/opt_out>.
>
>
> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.

[Michael Betancourt]

> Yup. I wasn't suggesting doing it before 2.0!

Yeah, but like _seconds_ after 2.0, right? :-D

For motivation, the first ever run of RHMC in Stan. The samples are consistent and everything.

21:53:35[~/Documents/Research/Code/Inference/stan-dev/stan (feature/rhmc)]-betan: ./rhmc_test sample algorithm=hmc engine=static metric=softabs
method = sample (Default)
sample
num_samples = 1000 (Default)
num_warmup = 1000 (Default)
save_warmup = 0 (Default)
thin = 1 (Default)
adapt
engaged = 1 (Default)
gamma = 0.050000000000000003 (Default)
delta = 0.65000000000000002 (Default)
kappa = 0.75 (Default)
t0 = 10 (Default)
algorithm = hmc (Default)
hmc
engine = static
static
int_time = 6.2831853071795898 (Default)
metric = softabs
softabs
alpha = 1 (Default)
max_fp = 1 (Default)
fp_threshold = 1e-08 (Default)
stepsize = 1 (Default)
stepsize_jitter = 0 (Default)
id = 0 (Default)
data
file = (Default)
init = 2 (Default)
random
seed = 3219696823
output
file = samples.csv (Default)
append_sample = 0 (Default)
diagnostic_file = (Default)
append_diagnostic = 0 (Default)
refresh = 100 (Default)

Iteration: 1 / 2000 [ 0%] (Warmup)
Iteration: 100 / 2000 [ 5%] (Warmup)
Iteration: 200 / 2000 [ 10%] (Warmup)
Iteration: 300 / 2000 [ 15%] (Warmup)
Iteration: 400 / 2000 [ 20%] (Warmup)
Iteration: 500 / 2000 [ 25%] (Warmup)
Iteration: 600 / 2000 [ 30%] (Warmup)
Iteration: 700 / 2000 [ 35%] (Warmup)
Iteration: 800 / 2000 [ 40%] (Warmup)
Iteration: 900 / 2000 [ 45%] (Warmup)
Iteration: 1000 / 2000 [ 50%] (Warmup)
Iteration: 1001 / 2000 [ 50%] (Sampling)
Iteration: 1100 / 2000 [ 55%] (Sampling)
Iteration: 1200 / 2000 [ 60%] (Sampling)
Iteration: 1300 / 2000 [ 65%] (Sampling)
Iteration: 1400 / 2000 [ 70%] (Sampling)
Iteration: 1500 / 2000 [ 75%] (Sampling)
Iteration: 1600 / 2000 [ 80%] (Sampling)
Iteration: 1700 / 2000 [ 85%] (Sampling)
Iteration: 1800 / 2000 [ 90%] (Sampling)
Iteration: 1900 / 2000 [ 95%] (Sampling)
Iteration: 2000 / 2000 [100%] (Sampling)

Elapsed Time: 0.067093 seconds (Warm-up)
0.044717 seconds (Sampling)
0.11181 seconds (Total)

[Michael Betancourt]

And with NUTS -- the samples again look good so aside from gross inefficiency I think we might have something...

23:17:02[~/Documents/Research/Code/Inference/stan-dev/stan (feature/rhmc)]-betan: ./rhmc_test sample algorithm=hmc engine=nuts metric=softabs

method = sample (Default)
sample
num_samples = 1000 (Default)
num_warmup = 1000 (Default)
save_warmup = 0 (Default)
thin = 1 (Default)
adapt
engaged = 1 (Default)
gamma = 0.050000000000000003 (Default)
delta = 0.65000000000000002 (Default)
kappa = 0.75 (Default)
t0 = 10 (Default)
algorithm = hmc (Default)
hmc

engine = nuts (Default)
nuts
max_depth = 10 (Default)

metric = softabs
softabs
alpha = 1 (Default)
max_fp = 1 (Default)
fp_threshold = 1e-08 (Default)
stepsize = 1 (Default)
stepsize_jitter = 0 (Default)
id = 0 (Default)
data
file = (Default)
init = 2 (Default)
random

seed = 3224693894

Elapsed Time: 0.107014 seconds (Warm-up)
0.076146 seconds (Sampling)
0.18316 seconds (Total)

[Bob Carpenter]

And that's with geometric NUTS, too?

This is so awesome, I'm at a loss for an adjective.

- Bob

[Bob Carpenter]

No way to inspire us like getting a working version of RMHMC.

(By the way, I re-read the section of Baez and he did say addition
and multiplication on C^inf(M) was defined pointwise; I'm just way
too used to working in the composition algebra.)

- Bob

[Michael Betancourt]

> And that's with geometric NUTS, too?

Yup -- although just a 1D gaussian so far. I'll try the funnel today.