PyStan output

[Krzysztof Sakrejda]

From the users mailing list it looks like PyStan's output .csv file is not compatible with stansummary:

On Friday, May 27, 2016 at 12:42:27 AM UTC-4, Jeff Alstott wrote:
> bin/stansummary output.csv
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::bad_lexical_cast> >'
what(): bad lexical cast: source type value could not be interpreted as target
Aborted

Is PyStan _trying_ to write a compatible file or using its own format?

Krzysztof

[Allen B. Riddell]

I think PyStan uses/used stan/services/io/write_iteration_csv.hpp or
something related from stan. Frankly, I'm not interested in supporting
csv output at all in PyStan. If someone is just dumping Stan output to a
csv file they shouldn't be using pystan, they should be using cmdstan.

AR

> --
> You received this message because you are subscribed to the Google Groups "stan development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Krzysztof Sakrejda]

On Friday, May 27, 2016 at 11:10:55 AM UTC-4, Allen B. Riddell wrote:
> I think PyStan uses/used stan/services/io/write_iteration_csv.hpp or
> something related from stan.

OK, thanks for the pointer.

> Frankly, I'm not interested in supporting
> csv output at all in PyStan. If someone is just dumping Stan output to a
> csv file they shouldn't be using pystan, they should be using cmdstan.

We regularly do workshops where each participant chooses their interface
and the presenter can't know all the interfaces. Inter-operability based
on common code in stan-dev/stan looks like the only way to mitigate this
issue.

Krzysztof

[Allen B. Riddell]

On 05/27, Krzysztof Sakrejda wrote:
> On Friday, May 27, 2016 at 11:10:55 AM UTC-4, Allen B. Riddell wrote:
> > I think PyStan uses/used stan/services/io/write_iteration_csv.hpp or
> > something related from stan.
>
> OK, thanks for the pointer.
>
> > Frankly, I'm not interested in supporting
> > csv output at all in PyStan. If someone is just dumping Stan output to a
> > csv file they shouldn't be using pystan, they should be using cmdstan.
>
> We regularly do workshops where each participant chooses their interface
> and the presenter can't know all the interfaces. Inter-operability based
> on common code in stan-dev/stan looks like the only way to mitigate this
> issue.

I agree with this. If CSV and rdump are the interchange formats and
there are functions which support conversion in stan-dev/stan then all's
well.

Best,

AR

[Krzysztof Sakrejda]

On Friday, May 27, 2016 at 11:31:56 AM UTC-4, Allen B. Riddell wrote:
> I agree with this. If CSV and rdump are the interchange formats and
> there are functions which support conversion in stan-dev/stan then all's
> well.

+1
> Best,
>
> AR

[Bob Carpenter]

Another reason to support a serialized output is to cut
down on memory contention between the draws and the sampler.
If you run the sampler, dump to file, then read back in,
then the peak memory requirements are much lower.

I think it makes sense to split PyStan into two parts.
One that just wraps Stan and produces output and one
that reads in MCMC output and analyzes it. ShinyStan and
other packages like Loo are doing some of that output
analysis for us on the R side. So there's a lot to coordinate!

- Bob