Problems with (maybe) different versions of PyStan

[Jingjing Chen]

Hi Stan Users,

I was testing my script, which used the PyStan package, on a cluster and the script ran completely fine. Then I simply copied the script to another cluster, because its more capable of handling my final dataset, and it failed almost every time. The error is always as following. 

std::bad_alloc [origin: bad_alloc]

If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

/cm/local/apps/slurm/var/spool/job60020/slurm_script: line 16:  8421 Segmentation fault

I am very puzzled. I checked the PyStan version on both cluster, and found that the first one has version 2.11.0.0, and the second cluster has version 2.14.0.0. I am not sure if that could be the problem. If not, can someone suggest a reason that the same code perform so differently on two machines?

Thanks,

Jingjing

[Bob Carpenter]

You should run 2.14 in both places as there was a bug in 2.11 that
led to slightly biased analyses.

That could either mean that the other cluster doesn't have enough
memory or that you're mixing binaries. You need to compile the Stan
program on the same platform on which you try to run it.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Jingjing Chen]

Thanks for your reply. 

I am not quite sure what "mixing binaries" means. Can you clarify?

Also what do you mean by "compile the Stan program"? I am using the PyStan package, and I simply import it to my python script and run the python script, and it works. Do you mean that that is not the correct way using it?

Thanks,

Jingjing

在 2017年3月22日星期三 UTC-4下午6:18:21，Bob Carpenter写道：

[Bob Carpenter]

You need Allen Riddell's help here.

PyStan translates a Stan program to C++, compiles the C++
to binary form (using a given compiler), then it runs the
binary. The C++ compiler you use needs to generate binary
code that's compatible with running through Python.

- Bob

[ridd...@fastmail.com]

What do you mean by "import"? Are you pickling (serializing) your model
to disk and then unpickling it? That will not work for reasons that Bob
mentioned.

[Jingjing Chen]

I am using PyStan. 

http://pystan.readthedocs.io/en/latest/getting_started.html

It is the Python interface for Stan. In Python, when you want to use a module, you do import module. That is what I mean by import. You can see the example on the website above under 'Using PyStan'. 

I have been using PyStan for almost a year now, and I haven't run into anything that says I need to generate some binary code. It worked fine before. The problem just rose when I started to use this new cluster 3 months ago. I copy the same script, which worked on the old cluster that I have been using, to this new cluster. And then I started to almost constantly see this 'Segmentation Fault'.

I am not quite sure what is the 'pickling(serializing)' thing you mentioned.

Thanks,

Jingjing

在 2017年3月23日星期四 UTC-4下午3:54:14，ridd...@fastmail.com写道：