user defined functions library

[Jonah Gabry]

I think we should host a repository of useful user-defined functions, perhaps separated into distributions, rngs, and operations. Even if it takes a while to build up a sizable library this would eventually be a very helpful resource for users. If I set this up should it go under the BSD or CC-BY license?

Jonah

[Dustin Tran]

I like this idea. And if they become used enough that they should be ported into stan math, then we can rigorize their inputs/outputs and add unit tests.

Dustin

> On Mar 12, 2016, at 3:05 PM, Jonah Gabry <jga...@gmail.com> wrote:
>
> I think we should host a repository of useful user-defined functions, perhaps separated into distributions, rngs, and operations. Even if it takes a while to build up a sizable library this would eventually be a very helpful resource for users. If I set this up should it go under the BSD or CC-BY license?
>
> Jonah
>

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Bob Carpenter]

I'd prefer BSD for code and CC-BY for doc. See
the "contributing case studies" here:

http://mc-stan.org/documentation/case-studies.html

We've actually rolled the license agreement into the
pull requests going forward.

- Bob

[Jonah Sol Gabry]

On Saturday, March 12, 2016, Bob Carpenter <ca...@alias-i.com> wrote:

I'd prefer BSD for code and  CC-BY for doc.  See
the "contributing case studies" here:

http://mc-stan.org/documentation/case-studies.html

We've actually rolled the license agreement into the
pull requests going forward.

- Bob

Ok thanks. I like BSD for code and CC-BY for doc.

[Sebastian Weber]

Great idea!

How about adding specific topic domains (PK/PD modeling)?

Where would you place this? In the stan github? Are you planing to let users write unit-tests for it? Eventually via the expose_stan_functions of rstan coupled with testthat or RUnit?

Sebastian

[Daniel Lee]

Hi Jonah,

Thanks for bringing this up. I've been thinking having a "cookbook" would be useful. Some thoughts:

- I'd suggest a separate repo. Something that's strictly controlled with consistent formatting, a high level of certainty before it gets in.

- Stan: I think we should revisit how to include files.

- Math: I think we should expose all our check functions.

- Stan: I think we should have a unit testing framework for Stan functions. This last one will be to demonstrate that the functions being submitted are actually sound and can be trusted.

Unit testing for Stan functions. I'm not certain about all the details, but perhaps we can have:

// foo.stan

functions {

   real bar(real baz, vector qux) {
       ...

       return value;

   }

}

// foo.test.stan?

#include foo.stan

transformed data {

  real baz1;

  real baz2;

  vector qux1;

  vector qux2;

  baz1 = 1;

  baz2 = 2;
  ...

}

transformed parameters {

  real baz3;

  vector qux3;
  ...

}

test {

   EXPECT_EQ(123, bar(baz1, qux1));
   EXPECT_EQ(456, bar(baz2, qux2));

}

test {

   EXPECT_REJECT(bar(infinity(), qux1));

}

test {

   real norf;

   norf = bar(baz3, qux3);

   EXPECT_EQ(789, norf);

   EXPECT_EQ(0, gradient(norf, baz3));
   EXPECT_EQ({1, 2, 3}, gradient(norf, qux3));

}

//

I'm not really fond of the all caps, so please look at the function and not the form. I think using a unit testing framework that outlines what we expect is going to be the right way to go.

I can see the tests being generated as C++ that could either be run from the command line after built or run within the interfaces. If those are strict requirements, we can design for that.

Daniel

[Bob Carpenter]

For now, we're just talking about functions defined within
Stan programs, right? (Though I agree that includes for
Stan C++ would be good and exposing all the validation functions
would go a long way to allowing users to write robust argument
checking.)

For now, I think we can start the way we're starting with
case studies --- take them from contributors and see how it
goes. So they could go in example-models or a new repo.
This all needs to get a bit more organized, but I think that's
hard to anticipate until it gets a bit bigger.

If we have to wait for a unit testing framework in Stan, that'll
probably be a while.

- Bob

[Daniel Lee]

On Monday, March 14, 2016, Bob Carpenter <ca...@alias-i.com> wrote:

For now, we're just talking about functions defined within
Stan programs, right?  (Though I agree that includes for
Stan C++ would be good and exposing all the validation functions
would go a long way to allowing users to write robust argument
checking.)

Yes. 

 

For now, I think we can start the way we're starting with
case studies --- take them from contributors and see how it
goes.  So they could go in example-models or a new repo.
This all needs to get a bit more organized, but I think that's
hard to anticipate until it gets a bit bigger.

 I'd prefer a new repo. Or calling example-models something else. 

If we have to wait for a unit testing framework in Stan, that'll
probably be a while.

I don't think we should wait, but I think the functions should be tested. I'm assuming we are thinking this is a validated set of functions that were collecting. If I've misunderstood the scope, please let me know.

Perhaps we can write c++ tests for now as we spec out a test framework?

[Jonah Sol Gabry]

On Sun, Mar 13, 2016 at 9:49 PM, Daniel Lee <bea...@alum.mit.edu> wrote:

 I'd prefer a new repo. Or calling example-models something else. 

I agree that it should be separate from example-models. Maybe stan-function-library?

 

I don't think we should wait, but I think the functions should be tested. I'm assuming we are thinking this is a validated set of functions that were collecting. If I've misunderstood the scope, please let me know.

Yeah, that's what I have in mind. I'm not sure how much effort it would take to do C++ testing, and I would understand if people want to avoid R, but it would be pretty easy to do testing using RStan's expose_stan_functions. There could be an R script that brings the function(s) into R using expose_stan_functions and then runs some tests in R.