Reducing the size of a stanfit object

[Cameron Bracken]

Hi,

I have a stanfit object that is quite large (8,500 parameters, 10,000 samples). For postprocessing purposes I don’t really need all those samples. Instead of running the model again, I’d like to subsample from what I have, something like

stanfit2 = sub_sample(stanfit,1000)

Is there some R code out there that does this?

Thanks,
Cameron

​

[Daniel Lee]

Check out the extract function in rstan.

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Cameron Bracken]

I’d like to keep the data in the stanfit object if possible.

You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/2IsD6y-crgA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

[Jonah]

The draws are tucked away in the sim slot of a stanfit object (along with some other stuff). So you could make a copy of your stanfit and then change the sim slot in the copy. All of the metadata and whatnot would still describe the original stanfit with all the draws, but the relevance of that depends on what you need for your post-processing. That can be changed to if need be. A different approach would be to create an empty stanfit object and fill it in with what you need.

[Cameron Bracken]

I can manage that, thanks for the responses. 

On Wed, Aug 12, 2015 at 6:00 PM Jonah <jga...@gmail.com> wrote:

The draws are tucked away in the sim slot of a stanfit object (along with some other stuff). So you could make a copy of your stanfit and then change the sim slot in the copy. All of the metadata and whatnot would still describe the original stanfit with all the draws, but the relevance of that depends on what you need for your post-processing. That can be changed to if need be. A different approach would be to create an empty stanfit object and fill it in with what you need.

[Jonah]

I haven't tried it, but maybe something like this:

sub_sample <- function(stanfit, n, keep_warmup = TRUE) {
  sim <- stanfit@sim
  samp <- sim$samples
  W <- sim$warmup
  I <- sim$iter
  sel <- c(if (keep_warmup) 1:W, sample((W + 1):I, size = n))
  subsamp <- lapply(samp, function(chain_samp) {
    lapply(chain_samp, function(x) x[sel])
  })
  stanfit@sim$samples <- subsamp
  stanfit
}

This doesn't change any of the metadata and probably needs to be adjusted if you did any thinning, but it's along the right lines.  

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Cameron Bracken]

Excellent, that works perfectly for my purposes, thanks!

There was just one small typo in the code, samps should be samp in the lapply call. Thanks again! 

Cameron

On Thu, Aug 13, 2015 at 10:14 AM Jonah <jga...@gmail.com> wrote:

I haven't tried it, but maybe something like this:

sub_sample <- function(stanfit, n, keep_warmup = TRUE) {
  sim <- stanfit@sim
  samp <- sim$samples
  W <- sim$warmup
  I <- sim$iter
  sel <- c(if (keep_warmup) 1:W, sample((W + 1):I, size = n))

  subsamp <- lapply(samps, function(chain_samps) {
    lapply(chain_samps, function(x) x[sel])

  })
  stanfit@sim$samples <- subsamp
  stanfit
}

This doesn't change any of the metadata and probably needs to be adjusted if you did any thinning, but it's along the right lines.  

On Thursday, August 13, 2015 at 11:53:01 AM UTC-4, Cameron Bracken wrote:

I can manage that, thanks for the responses. 

On Wed, Aug 12, 2015 at 6:00 PM Jonah <jga...@gmail.com> wrote:

The draws are tucked away in the sim slot of a stanfit object (along with some other stuff). So you could make a copy of your stanfit and then change the sim slot in the copy. All of the metadata and whatnot would still describe the original stanfit with all the draws, but the relevance of that depends on what you need for your post-processing. That can be changed to if need be. A different approach would be to create an empty stanfit object and fill it in with what you need.

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/2IsD6y-crgA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/2IsD6y-crgA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

​

[Jonah]

Yeah that's a typo, thanks. I just edited my previous post to fix it.