model running too slow and does not converge with database-like organizing for ragged data structure

[Lei Zhang]

Dear Stan experts,

I followed the relevant manual chapter (p149 on Stan Reference 2.11) to reorganize my data to a database-like structure to handle missing responses. 

The model works pretty well with the data when ignoring those missing trials (non-responses were automatically replaced by our experimental program, so those generated data were treated as if they were recorded in this case). This model ran pretty smoothly and it was able to reach convergence (r_hat in the worst case was <=1.05). 

However, when I was taking those missing trials seriously, and reconstructed my data into the database-like structure, the model runs nearly 10 times slower and the chains do not converge, with neff <= 10, out of 1000 samples after warm-up per chain (4000 total valid samples).

The .stan models are nearly the same except that one is used for the whole dataset (when ignoring the missing data), and the other is used for the ragged dataset. There is another rescaling in the ragged dataset but this is mostly identical to the whole dataset. 

My QUESTION is: since the only difference between this two models is the data organization/structure, should I conclude that the database-like reconstruction causes this problem? Or did I make some mistake? Note that for the whole dataset, every subject has 100 trials, while in the ragged data structure, the trial number varies between 71 and 100, mean = 92.2, SD = 6.9. Is the variance in the trial numbers a potential problem for Stan sampling? 

I attached the two models for the whole dataset and the ragged dataset, respectively. I am estimating these models on a Linux server with RStan 2.8.2. and R3.2.0.

Any suggestion is appreciated!

Thanks a lot,

Lei

[Bob Carpenter]

I'm afraid the programs were too complicated for me to follow.
Our advice is always to start simple and build up so you
can see where things go wrong with minimal examples, but it's
not always easy to do this.

You should get the same answers and same convergence if you
have defined the same density in two different ways. There
may be big speed differences, though. But it sounds like you
don't have the same models in both cases, so all bets are off.

What you have to be careful for is variables that are not
bounded above and below and don't have priors---that will create
an improper posterior that will destroy all the sampling.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <RevLearn.stan><RevLearn_ms_lg.stan>

[Lei Zhang]

Hi Bob, 

thanks for your advice. These two models are indeed identical except that they are dealing with different data organizations. The model at its core is a simple reinforcement learning model. We have tested the simplest version and gradually built up to the current version, which we are satisfied with. 

I checked the model again, but I didn't find any unbounded parameters/variables without declaring priors. In fact, the only place I changed is the data section and the indexing approach. With the central part of the model being untouched, I am really confused with the dis-convergence. So, any other thoughts?

I attached the model for the database-like structure, with modified indexing so that it's a bit easier to read. 

Thanks again.

Lei

[Bob Carpenter]

They can't be the same likelihood if they get
different answers. You want to go through the
loops by hand to write down the likelihoods in each
case to make sure you're coding the same model.

- Bob

> <RevLearn_ms_lg.stan>