Caching for Stan binaries?

[Titus von der Malsburg]

Would it make sense to have a caching option for the function brm, such that it recompiles the sampler only when the Stan code has changed?  I think this could save users quite a bit of time.  Sorry, if this feature already exists but a search in the documentation didn't turn up anything and the default behavior is not to cache on my system.

Loosely related question: The other day I fit a large number of models and at some point received an error message saying that R had reached the limit of shared objects that can be loaded. Can this be avoided by unloading the shared objects for a sampler after the sampling process has finished?

Thanks, and again, great package!

  Titus

[Paul Buerkner]

(1) You can use the 'update' method to refit your model without recompliation if possible.

(2) I haven't seen this error message yet. This is probably more an rstan related question, I think.

--
You received this message because you are subscribed to the Google Groups "brms-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+unsubscribe@googlegroups.com.
To post to this group, send email to brms-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brms-users/30960efe-43a5-49ed-a7fb-abb19520a414%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Titus von der Malsburg]

On Wednesday, August 30, 2017 at 12:56:19 PM UTC+2, Paul Buerkner wrote:

(1) You can use the 'update' method to refit your model without recompliation if possible.

This helps only in some situations but not all. One usecase is where you recompile an Knitr document with different data. Update works only if I change the code before recompiling (from brm to update).  Another case is when you rerun your script with a fresh R session where you don't have the old fit object.  Both cases would be solved if brm could transparently cache samplers on disk.

 

(2) I haven't seen this error message yet. This is probably more an rstan related question, I think.

Okay, I will ask there when it happens again.

  Titus

 

2017-08-30 12:42 GMT+02:00 Titus von der Malsburg <mals...@gmail.com>:

Would it make sense to have a caching option for the function brm, such that it recompiles the sampler only when the Stan code has changed?  I think this could save users quite a bit of time.  Sorry, if this feature already exists but a search in the documentation didn't turn up anything and the default behavior is not to cache on my system.

Loosely related question: The other day I fit a large number of models and at some point received an error message saying that R had reached the limit of shared objects that can be loaded. Can this be avoided by unloading the shared objects for a sampler after the sampling process has finished?

Thanks, and again, great package!

  Titus

--
You received this message because you are subscribed to the Google Groups "brms-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+...@googlegroups.com.

[Paul Buerkner]

I see. Do you have an idea, on how such caching could look like?

To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+unsubscribe@googlegroups.com.

To post to this group, send email to brms-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/brms-users/ccc3fe15-494b-4b79-b8b0-afd367c5bd05%40googlegroups.com.

[Titus von der Malsburg]

On Wednesday, August 30, 2017 at 1:25:14 PM UTC+2, Paul Buerkner wrote:

I see. Do you have an idea, on how such caching could look like?

One way to do this would be to calculate a checksum for the stan code (sha1 or similar)  and then use that checksum as the name of the shared object containing the sampler.  Next time, you again compute this checksum, and if there is already a shared object with that name you skip the compilation step and just use the existing sampler.

I don't understand the architecture of brms and rstan well enough to be able to tell whether this feature should be implemented in brms or in rstan.

Not a terribly important feature but nice to have at some point.

  Titus

[Paul Buerkner]

I think rstan already has something like that in place, but it has been causing problems in the past such as segfaults and other hard to fix problems. That is why I explicitely disabled usage of those shared objects.

Can you try it just using rstan and some simple stan model and tell me whether rstan alone behaves in the desired way?

--

You received this message because you are subscribed to the Google Groups "brms-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+unsubscribe@googlegroups.com.
To post to this group, send email to brms-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/brms-users/6e029440-9fa5-409a-8d08-25221500311c%40googlegroups.com.

[Titus von der Malsburg]

On Wednesday, August 30, 2017 at 12:56:19 PM UTC+2, Paul Buerkner wrote:

(2) I haven't seen this error message yet. This is probably more an rstan related question, I think.

Just for the record, this is the error message:

  Compiling the C++ model

  Quitting from lines 1632-1643 (/home/malsburg/test.Rnw) 

  Error in dyn.load(libLFile) : 

    unable to load shared object '/tmp/RtmpJ7KuDT/file3745e4a2a37.so':

    `maximal number of DLLs reached...

Happened after fitting roughly 100 models in one session.  I will also report to rstan.

  Titus

[Titus von der Malsburg]

On Wednesday, August 30, 2017 at 1:37:56 PM UTC+2, Paul Buerkner wrote:

I think rstan already has something like that in place, but it has been causing problems in the past such as segfaults and other hard to fix problems. That is why I explicitely disabled usage of those shared objects.

Can you try it just using rstan and some simple stan model and tell me whether rstan alone behaves in the desired way?

Yes, I can do this, but not before the weekend.

  Titus 

[Paul Buerkner]

Many thanks for the updates!

--

You received this message because you are subscribed to the Google Groups "brms-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+unsubscribe@googlegroups.com.
To post to this group, send email to brms-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/brms-users/6274c76d-cbd3-4e6a-a642-60174d34b4de%40googlegroups.com.

[Titus von der Malsburg]

GitHub issue at rstan:

https://github.com/stan-dev/rstan/issues/448

On Thursday, August 31, 2017 at 2:39:27 PM UTC+2, Paul Buerkner wrote:

Many thanks for the updates!

Am 31.08.2017 2:34 nachm. schrieb "Titus von der Malsburg" <mals...@gmail.com>:

On Wednesday, August 30, 2017 at 1:37:56 PM UTC+2, Paul Buerkner wrote:

I think rstan already has something like that in place, but it has been causing problems in the past such as segfaults and other hard to fix problems. That is why I explicitely disabled usage of those shared objects.

Can you try it just using rstan and some simple stan model and tell me whether rstan alone behaves in the desired way?

Yes, I can do this, but not before the weekend.

  Titus 

--
You received this message because you are subscribed to the Google Groups "brms-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+...@googlegroups.com.