myglmer2stan

[Nik Ford]

I have a question:

In JAGS, burn-in shows on the screen and happens relatively quick (depending on data size& how long one sets burnin) however, mine appears to be 'silent' and, while set at 1000, taking forever, if it is running at all.

Is this common?

It has been sitting at Chain 1, Iteration: 1/10000 [ 0%] (Warmup)
For ~40 minutes. I've reset it a few times for lower burnin and interations to test this.

Thanks.

--
~Nik

[Ben Goodrich]

On Thursday, November 3, 2016 at 12:32:41 PM UTC-4, Nik Ford wrote:

In JAGS, burn-in shows on the screen and happens relatively quick (depending on data size& how long one sets burnin) however, mine appears to be 'silent' and, while set at 1000, taking forever, if it is running at all.

It is running. You can set stan()'s refresh argument to be some number smaller than 10 to see updates at intervals more frequent than 10%. But it is quite likely that your model will take quite a long time and might still not produce good results without a reparameterization. You shouldn't need 10000 iterations anyway. I'm not sure what myglmer2stan is but you should probably be using stan_glmer in the rstanarm package.

Ben

[Nicole Ford]

Thanks, I will try that.  I was expecting it to update more often like JAGS does.

Much larger iterations are expected given my data and my discipline.  I will add that bit of code next time!

Thanks! Very helpful!

(It is now showing 30%)

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/1kyA_d-BChI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

~Nicole Ford

Ph.D. Candidate/ Instructor

Department of Government and International Affairs

University of South Florida

office: SOC 012M

e: nmh...@mail.usf.edu

http://gia.usf.edu/student/nford/

https://usf.academia.edu/NicoleFord

[Matt Espe]

Did you start with fewer iterations first? One common difference between JAGS and Stan is that Stan often has much higher effective samples per iteration compared to JAGS. When I made the transition, I found I could often get more effective samples from Stan with 1/10 the iterations.

Second, I believe glmer2stan is depreciated and no longer maintained (last update was 2015). You should really look into the rstanarm package. 

Matt

[Nicole Ford]

Thanks.  Good to know.  I stumbled on it and figured I would try it since it was suppose to be good for interactions.

I actually started with a much larger iteration (100k) and went down to see if I would get some action.

I will look into the other packages you both mentioned. 

[Bob Carpenter]

Iterations are meaningless in and of themselves. What
you want to do is set a threshold for the effective
sample size (an estimate is reported as n_eff by Stan;
don't know about RStanArm). There can be orders of
magnitude or more differences in n_eff per iteration based
on algorithm. In some cases, Stan can generate an n_eff
in 500 iterations larger than JAGS can generate in 200,000
iterations, and in other cases, the n_eff rate per iterations
will be close to each other. It depends primarily on the
shape of the posterior.

Of course, I understand that by "my discipline" you may mean
reviewers won't accept a paper that doesn't have some
ridiculous number of draws in in it. Usually it's not much
use to get n_eff >> 100 or so, because at that point, the
mean estimates have 1/10 the error of the posterior standard
deviation.

- Bob

> On Nov 3, 2016, at 1:51 PM, Nicole Ford <nmh...@mail.usf.edu> wrote:
>
> Thanks, I will try that. I was expecting it to update more often like JAGS does.
>
> Much larger iterations are expected given my data and my discipline. I will add that bit of code next time!
>
> Thanks! Very helpful!
>
> (It is now showing 30%)
>
> On Thursday, November 3, 2016, Ben Goodrich <goodri...@gmail.com> wrote:
> On Thursday, November 3, 2016 at 12:32:41 PM UTC-4, Nik Ford wrote:
> In JAGS, burn-in shows on the screen and happens relatively quick (depending on data size& how long one sets burnin) however, mine appears to be 'silent' and, while set at 1000, taking forever, if it is running at all.
>
> It is running. You can set stan()'s refresh argument to be some number smaller than 10 to see updates at intervals more frequent than 10%. But it is quite likely that your model will take quite a long time and might still not produce good results without a reparameterization. You shouldn't need 10000 iterations anyway. I'm not sure what myglmer2stan is but you should probably be using stan_glmer in the rstanarm package.
>
> Ben
>
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/1kyA_d-BChI/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> ~Nicole Ford
> Ph.D. Candidate/ Instructor
> Department of Government and International Affairs
> University of South Florida
> office: SOC 012M
> e: nmh...@mail.usf.edu
> http://gia.usf.edu/student/nford/
> https://usf.academia.edu/NicoleFord
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Nicole Ford]

Yes, that's exactly it: I've been told 100k is the minimum expected for larger n data.  

[Ben Goodrich]

On Thursday, November 3, 2016 at 4:05:15 PM UTC-4, Nik Ford wrote:

Yes, that's exactly it: I've been told 100k is the minimum expected for larger n data.  

You have been told wrong (when that is applied to MCMC algorithms in general as opposed to Stan specifically).

[Nicole Ford]

I figured as much.  It's pretty obvious theoretically, let alone looking at the traceplots, etc.  I'm guessing it is because the discipline is fairly new to Bayes and a bit resistant in general. The standard argument regarding priors, etc. is what I was told when I first took this method up four years ago.  I'm the only one in my department that does this. 

On Thursday, November 3, 2016, Ben Goodrich <goodri...@gmail.com> wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/1kyA_d-BChI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jonah Gabry]

Yeah 100k is totally arbitrary and just nonsense without considering the particular MCMC algorithm. I wouldn't start with more than 2000 (the default) if you're using stan_glmer in rstanarm. If you need more you can increase but that's usually plenty for convergence (especially if you set the QR argument to TRUE).

Jonah

[Jonah Gabry]

Also for larger N data you often (not always) need _fewer_ iterations not more.

[Jonah Gabry]

Forgot to say that instead of reparameterizing yourself I would first try stan_glmer. If you're still having trouble after that definitely follow up and we can help you write your own Stan program for your model.

Jonah

[Nicole Ford]

Jonah, thank you.

The reason I stumbled on this bit of code was because I was having difficulty pulling a few plots from r-- I was able to pull traceplots, density, etc. But I was having difficulty pulling HDI and effects. I was starting to wonder if maybe my JAGS code was wrong or maybe I was just that incompetent at writing more detailed plot coding. (I am also the only one who uses R)

The myglmer2stan seemed to cover both bases- I feel confident my frequentist HLM code is correct and it seemed to have fairly straightforward plotting code.

The output looks a bit different in Stan from JAGS, however (looking at it now). 

It required I make my IV an integer of 1-4 (it's a factor with four named labels and wanted the labels to be integers). But of course that flattened that variable which wasn't what I was hoping.  I think it did I'm really not sure what I'm looking at.

I will take a look at the packages mentioned here and see if I can navigate those.  I'm on a bit of a time crunch now. 

Thanks, everyone, for your assistance.  

On Friday, November 4, 2016, Jonah Gabry <jga...@gmail.com> wrote:

Forgot to say that instead of reparameterizing yourself I would first try stan_glmer. If you're still having trouble after that definitely follow up and we can help you write your own Stan program for your model.

Jonah

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/1kyA_d-BChI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.