term

[cdip...@g.hmc.edu]

I have a (mixture) model with a few latent Gaussian processes, and as you might expect, instantiation is a pain. I generated a toy dataset, and if I the initial parameters to the ground truth of what they should be, sampling still works (though it's pretty darn slow) but I'm not confident (even after running for 5K warmup iterations) I've converged to the stationary distribution. I'd like to check this using a MAP estimate or variational Bayes with ADVI, and neither work.

Here is the output when I run the optimization method (in PyStan):

(env) ➜  src git:(master) python3 dhmm.py --likelihood gaussian --data ../data/toy_x.csv \

 --input ../data/toy_t.csv --test-input ../data/toy_test_t.csv --init ../data/toy_init.pkl \

 --output-file out/toy_opt.pkl --opt --num-components 2 --iter 10000 --lengthscale 1.0

Using cached StanModel

STAN OPTIMIZATION COMMAND (LBFGS)

init = user

save_iterations = 1

init_alpha = 0.001

tol_obj = 1e-12

tol_grad = 1e-08

tol_param = 1e-08

tol_rel_obj = 10000

tol_rel_grad = 1e+07

history_size = 5

seed = 0

initial log joint probability = -2662.82

   Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes

Error evaluating model log probability: Non-finite gradient.

Error evaluating model log probability: Non-finite gradient.

Error evaluating model log probability: Non-finite gradient.

Error evaluating model log probability: Non-finite gradient.

Error evaluating model log probability: Non-finite gradient.
...

Error evaluating model log probability: Non-finite gradient.

Error evaluating model log probability: Non-finite gradient.

Error evaluating model log probability: Non-finite gradient.

     25      -362.584   4.79461e-08   1.52197e+11       1e-12       0.001      300  LS failed, Hessian reset

Optimization terminated with error:

 Line search failed to achieve a sufficient decrease, no more progress can be made

I think this is weird since presumably the NUTS sampler uses gradient information as well, so I should have (but didn't) seen this when I sampled instead of optmized.

Similarly, with ADVI, I have this different issue (and it says it _can_ evaluate the gradient):

(env) ➜  src git:(master) python3 dhmm.py --likelihood gaussian --data ../data/toy_x.csv \

 --input ../data/toy_t.csv --test-input ../data/toy_test_t.csv --init ../data/toy_init.pkl \

 --output-file out/toy_opt.pkl --vb --num-components 2 --iter 10000 --lengthscale 1.0

Using cached StanModel

This is Automatic Differentiation Variational Inference.

(EXPERIMENTAL ALGORITHM: expect frequent updates to the procedure.)

Gradient evaluation took 0.048721 seconds
1000 iterations under these settings should take 48.721 seconds.
Adjust your expectations accordingly!

# Begin eta adaptation.

Traceback (most recent call last):
  File "dhmm.py", line 112, in <module>

   output_samples=args.samples,

 File "dhmm.py", line 39, in dhmm

   data=data, verbose=True, seed=seed, **kwargs

 File ".../src/cache.py", line 75, in stan_cache

   return sm.vb(**kwargs)

 File "/usr/local/lib/python3.5/site-packages/pystan/model.py", line 878, in vb

   ret, sample = fit._call_sampler(stan_args)

 File "stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.pyx", line 804, in stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.StanFit4Model._call_sampler (/var/folders/bw/n0kn7tm92171m6s4_dy83f_m0000gn/T/tmp741ifeld/stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.cpp:15590)

 File "stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.pyx", line 398, in stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224._call_sampler (/var/folders/bw/n0kn7tm92171m6s4_dy83f_m0000gn/T/tmp741ifeld/stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.cpp:8454)

ValueError: stan::variational::advi::adapt_eta: Cannot compute ELBO using the initial variational distribution. Your model may be either severely ill-conditioned or misspecified.

The model code is attached, but my main question is likely independent of the actual model. Why can't the optimization methods compute a finite gradient although the sampling (and ADVI) methods can? Why is ADVI breaking down and is there way to resolve this?

Thanks a ton!

[Bob Carpenter]

MCMC is more robust than ADVI or MAP and also computes
the reference result. ADVI is an approximation, but the
code's not very robust in Stan to fit. MAP computes something
different, and a MAP solution might not exist in many cases
(like hierarchical models) where the posterior mode is well
defined.

What do you mean by "instantiation" when you say it's a pain?
Initialization? Did you try default inits?

All of HMC, ADVI, and L-BFGS use gradient information, but
the algorithms are totally different. Also, ADVI isn't very
robust. And L-BFGS won't find a mode where one doesn't
exist.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <dhmm_gaussian.stan>

[Conner DiPaolo]

Thanks, that insight is very helpful. And yes, I mean initialization. No method works with the default inits (even sampling, see output below). This might result from some data living outside a (-5,5) box in 2d, and if the mean of the Gaussian is sampled initially from a (-2,2) box then these points would have basically zero likelihood under a Gaussian (very roughly speaking). I'll try rescaling the data and see if that fixes things later today, but other than that I can't think of anything.

(env) ➜  src git:(master) python3 dhmm.py --likelihood gaussian --data ../data/toy_x.csv --input ../data/toy_t.csv --test-input ../data/toy_test_t.csv --output-file out/toy.pkl --sample --num-components 2 --iter 10000 --lengthscale 1.0

Using cached StanModel

multiprocessing.pool.RemoteTraceback: 

"""

Traceback (most recent call last):

  File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 119, in worker

    result = (True, func(*args, **kwds))

  File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar

    return list(map(*args))

  File "stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.pyx", line 365, in stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224._call_sampler_star (/var/folders/bw/n0kn7tm92171m6s4_dy83f_m0000gn/T/tmp741ifeld/stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.cpp:7820)

  File "stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.pyx", line 398, in stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224._call_sampler (/var/folders/bw/n0kn7tm92171m6s4_dy83f_m0000gn/T/tmp741ifeld/stanfit4anon_model_8e8a301e317be9d54c577c5c41e69425_3362315270486115224.cpp:8454)

RuntimeError: Rejecting initial value:

  Log probability evaluates to log(0), i.e. negative infinity.

  Stan can't start sampling from this initial value.

Rejecting initial value:

  Log probability evaluates to log(0), i.e. negative infinity.

  Stan can't start sampling from this initial value.

... (continues for a while with the same error) ...

Initialization between (-2, 2) failed after 100 attempts. 

 Try specifying initial values, reducing ranges of constrained values, or reparameterizing the model.

... (then same thing for other chains) ...

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <dhmm_gaussian.stan>

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Bob Carpenter]

What happens under random inits? Do you get -infinity log
density? Or does it fail with an exception somewhere? The
latter usually indicates a bug in the coding, usually missing
constraints on parameters. All parameter values satisfying
the declared constraints should have support (finite log density)
in the model.

The other common problem in GPs is lack of jitter---trying to make
the model too deterministic.

- Bob

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> > <dhmm_gaussian.stan>
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[cdip...@g.hmc.edu]

Attached is the output using random inits for {sampling, vb, opt}. All of them have negative infinity log probability (throwing a
RuntimeError: Rejecting initial value), which given what you said would imply that my model is ill-defined with respect to the constraints?

[Bob Carpenter]

I don't see any error message there. Are you using the latest
PyStan? This may just mean that some of the arithmetic underflows
to zero before having log assigned.

You can print out target() as you go:

print("1: target = ", target());

...

print("2: target = ", target());

and so on to try to figure out where the negative infinity is getting
introduced. It may be that some of the arithmetic can be made
more robust by working directly on the log scale.

- Bob

P.S. There should be a comma after "i.e.".

> On Jan 3, 2017, at 4:39 PM, cdip...@g.hmc.edu wrote:
>
> Attached is the output using random inits for {sampling, vb, opt}. All of them have negative infinity log probability (throwing a
> RuntimeError: Rejecting initial value), which given what you said would imply that my model is ill-defined with respect to the constraints?
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> <rand_init_opt.out><rand_init_sample.out><rand_init_vb.out>

[Conner DiPaolo]

Turns out you were correct in saying that working directly on the log-scale can make the arithmetic more robust. Specifically, the Cholesky parametrized normal likelihood was the issue, and using the following representation directly (dropping the 2pi constant) lets the model run (even with opt, haven't tried out vb yet though will report back when I do). I wonder if this was a problem with using the Sigma = LL' factorization when L didn't have positive diagonal entries, and therefore wasn't the _Cholesky_ factorization, even though the math could be the same. If Stan assumes that the diagonal elements are positive, the log determinant could be computed more efficiently (see below: the square in L_ii^2 could be brought out of the sum as a scaling factor). In that case, taking the log of L_ii could be undefined (in the real plane) and return infinity or something. Does Stan do this under the hood?

On Thu, Jan 5, 2017 at 6:03 AM, Bob Carpenter <ca...@alias-i.com> wrote:

I don't see any error message there.  Are you using the latest
PyStan?  This may just mean that some of the arithmetic underflows
to zero before having log assigned.

You can print out target() as you go:

  print("1: target = ", target());

  ...

  print("2: target = ", target());

and so on to try to figure out where the negative infinity is getting
introduced.  It may be that some of the arithmetic can be made
more robust by working directly on the log scale.

- Bob

P.S.  There should be a comma after "i.e.".

> On Jan 3, 2017, at 4:39 PM, cdip...@g.hmc.edu wrote:
>
> Attached is the output using random inits for {sampling, vb, opt}. All of them have negative infinity log probability (throwing a
> RuntimeError: Rejecting initial value), which given what you said would imply that my model is ill-defined with respect to the constraints?
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <rand_init_opt.out><rand_init_sample.out><rand_init_vb.out>

--
You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Bob Carpenter]

Yes, the determinant calculation for the Cholesky-parameterized
multi-normal is done efficiently. Ben keeps a close watch on
our linear algebra for critical functions like this.

The relevant code for the determinant is in

stan/math/prim/mat/prob/multi_normal_cholesky_lpdf.hpp

and looks just like you'd expect for Cholesky factor L and
log density accumulator lp (recall the determinant is in
the denominator of the density).

lp -= L.diagonal().array().log().sum() * size_vec;

size_vec is the number of variates, so that we only compute
this determinant once if it's called with an array of
variates. That is, if we call y ~ multi_normal_cholesky(mu, L) and y
is an array of vectors. Then rather than computing it and adding
it for each vector element of y, we just multiply.

Also, we use template traits programming to wrap this in a static
conditional that only includes the decrement if the matrix L
isn't data. If it's data, the result is a constant term and is
dropped.

We also use specialized arithmetic to compute the quadratic form
using matrix division that's specialized for a triangular matrix,
where this gets called in a loop for each y in the sequence
of variates (may just be one if the argument isn't an array of
vectors):

sum_lp_vec += dot_self(mdivide_left_tri_low(L, y_minus_mu));

and then the whole result gets multiplied at once and subtracted
from the accumulator:

lp -= 0.5 * sum_lp_vec;

Eigen's great!

- Bob

> On Jan 13, 2017, at 4:44 AM, Conner DiPaolo <cdip...@g.hmc.edu> wrote:
>
> Turns out you were correct in saying that working directly on the log-scale can make the arithmetic more robust. Specifically, the Cholesky parametrized normal likelihood was the issue, and using the following representation directly (dropping the 2pi constant) lets the model run (even with opt, haven't tried out vb yet though will report back when I do). I wonder if this was a problem with using the Sigma = LL' factorization when L didn't have positive diagonal entries, and therefore wasn't the _Cholesky_ factorization, even though the math could be the same. If Stan assumes that the diagonal elements are positive, the log determinant could be computed more efficiently (see below: the square in L_ii^2 could be brought out of the sum as a scaling factor). In that case, taking the log of L_ii could be undefined (in the real plane) and return infinity or something. Does Stan do this under the hood?
>

> <image.png>

>
> On Thu, Jan 5, 2017 at 6:03 AM, Bob Carpenter <ca...@alias-i.com> wrote:
> I don't see any error message there. Are you using the latest
> PyStan? This may just mean that some of the arithmetic underflows
> to zero before having log assigned.
>
> You can print out target() as you go:
>
> print("1: target = ", target());
>
> ...
>
> print("2: target = ", target());
>
> and so on to try to figure out where the negative infinity is getting
> introduced. It may be that some of the arithmetic can be made
> more robust by working directly on the log scale.
>
> - Bob
>
> P.S. There should be a comma after "i.e.".
>
> > On Jan 3, 2017, at 4:39 PM, cdip...@g.hmc.edu wrote:
> >
> > Attached is the output using random inits for {sampling, vb, opt}. All of them have negative infinity log probability (throwing a
> > RuntimeError: Rejecting initial value), which given what you said would imply that my model is ill-defined with respect to the constraints?
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> > <rand_init_opt.out><rand_init_sample.out><rand_init_vb.out>
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[Conner DiPaolo]

Since this assumes that L_{ii} > 0 for all i, I'd suggest revising the relevant section 57.3 for the multi_normal_cholesky to make this explicit because it currently says that the parameterization is valid if L is lower triangular and Sigma = L L^T, which includes L_{ii} potentially non-positive. I can submit a pull request tomorrow as this is probably really simple to change unless there's some other protocol to changing documentation. This documentation issue might also be present in other Cholesky parameterized densities, though I haven't checked.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> > <rand_init_opt.out><rand_init_sample.out><rand_init_vb.out>
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to stan-users+unsubscribe@googlegroups.com.

[Bob Carpenter]

Yes, diagonals all have to be positive. And for correlation
matrices, the rows additionally have to be unit length.
I added a comment to the running next manual issue and will
fix it for the next release:

https://github.com/stan-dev/stan/issues/2122#issuecomment-272713519

- Bob

> > > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> > > To post to this group, send email to stan-...@googlegroups.com.
> > > For more options, visit https://groups.google.com/d/optout.
> > > <rand_init_opt.out><rand_init_sample.out><rand_init_vb.out>
> >
> > --
> > You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> > To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.

> > To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

> > To post to this group, send email to stan-...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Stan users mailing list" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/stan-users/h4Xmp-qq3KA/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to stan-users+...@googlegroups.com.

> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.