random effects model in Stan

[Linas Mockus]

Hi,

Can somebody advice on the size of nonlinear mixed effects model that could be solved in Stan in reasonable time? For example items that are manufactured in batches exhibit intra-batch variability (items within same batch are different) while average for each batch is also different. This could be handled by two level hierarchical model and thus the number of parameters to estimate grow with increasing  # of batches. I wonder how many batches one may reasonably consider and are there any tricks to overcome dimensionality explosion problem. For instance, nonlinear mixed effects model might involve solution of ODEs.

Thank you,
Linas

[Krzysztof Sakrejda]

On Friday, May 6, 2016 at 4:05:10 PM UTC-4, Linas Mockus wrote:

Hi,

Can somebody advice on the size of nonlinear mixed effects model that could be solved in Stan in reasonable time? For example items that are manufactured in batches exhibit intra-batch variability (items within same batch are different) while average for each batch is also different. This could be handled by two level hierarchical model

Well, I'm currently working with a model that runs in about 8 (?) hours that includes five levels (roughly 77, 292, 2299, 30, and 2300 units per level) for each of three key parameters.  The last two levels are a batch effect and a batch x group interaction.  

 

and thus the number of parameters to estimate grow with increasing  # of batches. I wonder how many batches one may reasonably consider and are there any tricks to overcome dimensionality explosion problem.

After long enough you know the batch-to-batch variance and can start dropping early batches.

 

For instance, nonlinear mixed effects model might involve solution of ODEs.

This I don't know.

Krzysztof

 

Thank you,
Linas

[Linas Mockus]

Krzysztof,

Impressive. Is your model nonlinear? What do you mean by dropping batches? Simply ignoring data for those batches?

Thank you,

Linas

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/-p_fHp2BU0A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sebastian Weber]

Stan is so impressive when it comes to many random effects, but the real bummer are ODEs! Just read the autodiff paper on arXiv on the scaling of the ODE system size and you quickly see how expensive ODEs are. On a different note, I just recently simplified an ODE Stan model from using ODEs to using an approximation to the ODE. I had 1400 subjects in the data set and the run time went down from 3 days to only ~4h (!) a factor of more than ~15x speedup (the model was still non-linear, just not using ODEs).

Long story short: Think twice if you can avoid the ODE... although we are working on improving these things, but for structural reasons, the price to pay for ODEs will be high regardless.

Sebastian

[Krzysztof Sakrejda]

On Friday, May 6, 2016 at 4:57:40 PM UTC-4, Linas Mockus wrote:

Krzysztof,

Impressive. Is your model nonlinear?

Bayesian chest-thumping? :P  The model is a Weibull-uniform mixture (to inflate the tail, so that's the three parameters) with truncation (so the data

are {delay, max_delay}.  

 

What do you mean by dropping batches? Simply ignoring data for those batches?

Yeah, or running the model in windows.  At some point extra batches just don't provide any more info 

on hyper-parameters.  Run the model on two windows and compare estimates of the parameters

you care about.  If they're not stable over time (batches) you'd like to know that anyway.  I guess you

could summarize prior batches as priors into a new run but if you have enough data to be taxing

Stan that probably doesn't get you much extra in terms of estimates.

 

 

Thank you,

Linas

On Fri, May 6, 2016 at 4:46 PM, Krzysztof Sakrejda <krzysztof...@gmail.com> wrote:

On Friday, May 6, 2016 at 4:05:10 PM UTC-4, Linas Mockus wrote:

Hi,

Can somebody advice on the size of nonlinear mixed effects model that could be solved in Stan in reasonable time? For example items that are manufactured in batches exhibit intra-batch variability (items within same batch are different) while average for each batch is also different. This could be handled by two level hierarchical model

Well, I'm currently working with a model that runs in about 8 (?) hours that includes five levels (roughly 77, 292, 2299, 30, and 2300 units per level) for each of three key parameters.  The last two levels are a batch effect and a batch x group interaction.  

 

and thus the number of parameters to estimate grow with increasing  # of batches. I wonder how many batches one may reasonably consider and are there any tricks to overcome dimensionality explosion problem.

After long enough you know the batch-to-batch variance and can start dropping early batches.

 

For instance, nonlinear mixed effects model might involve solution of ODEs.

This I don't know.

Krzysztof

 

Thank you,
Linas

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/-p_fHp2BU0A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Krzysztof Sakrejda]

On Friday, May 6, 2016 at 5:10:17 PM UTC-4, Sebastian Weber wrote:

Stan is so impressive when it comes to many random effects, but the real bummer are ODEs! Just read the autodiff paper on arXiv on the scaling of the ODE system size and you quickly see how expensive ODEs are. On a different note, I just recently simplified an ODE Stan model from using ODEs to using an approximation to the ODE.

Which approximation?  Is this the one that uses splines as an intermediate step?

[Sebastian Weber]

I used a home-brewn approximation which is specific to the problem, but now that you raise the spline thing I should look that up; I remember this was useful, but I never used it.

[Krzysztof Sakrejda]

On Friday, May 6, 2016 at 5:21:03 PM UTC-4, Sebastian Weber wrote:

I used a home-brewn approximation which is specific to the problem, but now that you raise the spline thing I should look that up; I remember this was useful, but I never used it.

Ok, just curious.  I think this was the paper I read on it: http://www.stat.tamu.edu/~bmallick/UQ/JASA.pdf

[Linas Mockus]

Sebastian,

I have no choice to avoid DOEs and will appreciate any advice / try out new code. The system of ODEs describes the physiologically based pharmacokinetic model which involves a bunch (I would say several dozen) DOEs. I can share the code if interested.

Thank you,

Linas

[Linas Mockus]

There is a well known orthogonal collocation method to approximate ODEs. https://en.wikipedia.org/wiki/Orthogonal_collocation

[Krzysztof Sakrejda]

On Friday, May 6, 2016 at 5:39:36 PM UTC-4, Linas Mockus wrote:

There is a well known orthogonal collocation method to approximate ODEs. https://en.wikipedia.org/wiki/Orthogonal_collocation

Thanks for the pointer!  

[lmockus]

Perhaps a better reference (as pertinent to ODE) could be http://www.tc1.ch.tum.de/fileadmin/w00bhu/www/Vorlesungen/Reaktordesign__Praktikum_/Votrag_Kollokation.pdf

Looks like it is a quite powerful technique once collocation points are defined. It would be awesome to expand Stan with this technique so ODEs are not constraint anymore.

--

[Krzysztof Sakrejda]

That's much clearer! And it sounds a lot like the paper I mentioned (but I'm a lumper not a splitter so take that with a grain of salt).  It would be neat to do this (that long-term to do list...)

Krzysztof

On Friday, May 6, 2016 at 8:03:49 PM UTC-4, Linas Mockus wrote:

Perhaps a better reference (as pertinent to ODE) could be http://www.tc1.ch.tum.de/fileadmin/w00bhu/www/Vorlesungen/Reaktordesign__Praktikum_/Votrag_Kollokation.pdf

Looks like it is a quite powerful technique once collocation points are defined. It would be awesome to expand Stan with this technique so ODEs are not constraint anymore.

 

------ Original Message ------

From: "Krzysztof Sakrejda" <krzysztof...@gmail.com>

To: "Stan users mailing list" <stan-...@googlegroups.com>

Sent: 5/6/2016 5:45:32 PM

Subject: Re: [stan-users] Re: random effects model in Stan

 

On Friday, May 6, 2016 at 5:39:36 PM UTC-4, Linas Mockus wrote:

There is a well known orthogonal collocation method to approximate ODEs. https://en.wikipedia.org/wiki/Orthogonal_collocation

Thanks for the pointer!  

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/-p_fHp2BU0A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.