Unable to print fit in PyStan
Christopher Grainger
<ipython-input-5-60318b13f8b2> in <module>()
----> 1 print fit
/tmp/tmpcAOlhh/pystan/stanfit4anon_model_9422209fd4fc97d65fed706c893188fa_1bfee611767a7a299666ddae3ddff840.so in stanfit4anon_model_9422209fd4fc97d65fed706c893188fa_1bfee611767a7a299666ddae3ddff840.StanFit4anon_model_9422209fd4fc97d65fed706c893188fa.__str__ (/tmp/tmpcAOlhh/pystan/stanfit4anon_model_9422209fd4fc97d65fed706c893188fa_1bfee611767a7a299666ddae3ddff840.cpp:7226)()
/usr/local/lib/python2.7/dist-packages/pystan/misc.pyc in _print_stanfit(fit, pars, probs, digits_summary)
79 date = fit.date.strftime('%c') # %c is locale's representation
80 footer = footer.format(sampler, date)
---> 81 s = _summary(fit, pars, probs)
82 body = _array_to_table(s['summary'], s['summary_rownames'],
83 s['summary_colnames'], digits_summary)
/usr/local/lib/python2.7/dist-packages/pystan/misc.pyc in _summary(fit, pars, probs, **kwargs)
200 if probs is None:
201 probs = (0.025, 0.25, 0.5, 0.75, 0.975)
--> 202 ss = _summary_sim(fit.sim, pars, probs)
203 # TODO: include sem, ess and rhat: ss['ess'], ss['rhat']
204 s1 = np.column_stack([ss['msd'][:, 0], ss['sem'], ss['msd'][:, 1], ss['quan'], ss['ess'], ss['rhat']])
/usr/local/lib/python2.7/dist-packages/pystan/misc.pyc in _summary_sim(sim, pars, probs)
288 tidx_rowm = list(itertools.chain(*tidx_rowm))
289 tidx_len = len(tidx_colm)
--> 290 lmsdq = [_get_par_summary(sim, i, probs) for i in tidx_colm]
291 msd = np.row_stack([x['msd'] for x in lmsdq])
292 quan = np.row_stack([x['quan'] for x in lmsdq])
/usr/local/lib/python2.7/dist-packages/pystan/misc.pyc in _get_par_summary(sim, n, probs)
337 """
338 # _get_samples gets chains for nth parameter
--> 339 ss = _get_samples(n, sim, inc_warmup=False)
340 msdfun = lambda chain: (np.mean(chain), np.std(chain, ddof=1))
341 qfun = lambda chain: mquantiles(chain, probs)
/usr/local/lib/python2.7/dist-packages/pystan/misc.pyc in _get_samples(n, sim, inc_warmup)
865 ss = []
866 for s, nw in zip(sim['samples'], sim['warmup2']):
--> 867 nth_key = list(s['chains'].keys())[n]
868 r = s['chains'][nth_key] if inc_warmup else s['chains'][nth_key][nw:]
869 ss.append(np.asarray(r))
/usr/lib/python2.7/collections.pyc in keys(self)
100 def keys(self):
101 'od.keys() -> list of keys in od'
--> 102 return list(self)
103
104 def values(self):
/usr/lib/python2.7/collections.pyc in __iter__(self)
77 curr = root[1] # start at the first node
78 while curr is not root:
---> 79 yield curr[2] # yield the curr[KEY]
80 curr = curr[1] # move to next node
Allen Riddell
PyStan's extract() is much slower than RStan's and it has problems with large
numbers of parameters. In short, PyStan makes copies of all the parameters when
it doesn't strictly need to. I'm working on a fix as we speak.
For now, I'd recommend using extract() by itself (which is slow but it shouldn't
stall) or using CmdStan.
Best wishes,
-a
On 08/28, Christopher Grainger wrote:
> I'm using iPython 2.2 and Python 2.7 and PyStan 2.4.0 on an AWS machine. My
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Jesse Bloom
Thanks,
Jesse
Allen Riddell
Extract and summary might be somewhat slow for large (1,000+ parameter) models,
but they should not be extremely slow (i.e., no more than a few seconds or tens
of seconds).
RStan does the rhat and ess calculations used in summary in C++ so it's faster
than PyStan. There's an issue tracking this: https://github.com/stan-dev/pystan/issues/21
Thanks,
Allen