Inference by exhaustive enumeration / methods for exact inference?

[David Wadden]

Another question - in the poisson ball example (poisson-ball.blog), the posterior is given in the comments after the model code:

   * Answers to some queries on this model, computed by exhaustive

   * enumeration up to 170 balls:
.
.
.
   *    Result from leili (computed using BigFraction, enumerated to 1000 balls)

The docs don't seem to explain how to do inference by exhaustive enumeration in BLOG. Is there a command-line flag to make this possible, or did this require some extra magic beyond what's available to end users?

Rejection sampling would be a nice alternative for drawing exact samples, but (I mentioned this on GitHub) rejection sampling barfs for this model:

(dw)$ bin/blog example/poisson-ball.blog -s blog.sample.RejectionSampler

Using fixed random seed for repeatability.

............................................

Constructing inference engine of class blog.engine.SamplingEngine

Constructing sampler of class blog.sample.RejectionSampler

Evidence: [ObsColor(Draw[0]) = Blue, ObsColor(Draw[1]) = Green, ObsColor(Draw[2]) = Blue, ObsColor(Draw[3]) = Green, ObsColor(Draw[4]) = Blue, ObsColor(Draw[5]) = Green, ObsColor(Draw[6]) = Blue, ObsColor(Draw[7]) = Green, ObsColor(Draw[8]) = Blue, ObsColor(Draw[9]) = Green]

Query: [/*DerivedVar*/ size({b for Ball b : true})]

Running for 10000 samples...

Query Reporting interval is 10000

Exception in thread "main" java.lang.NullPointerException

at blog.sample.RejectionSampler.nextSample(RejectionSampler.java:183)

at blog.engine.SamplingEngine.answerQueries(SamplingEngine.java:168)

at blog.Main.run(Main.java:190)

at blog.Main.main(Main.java:153)

Any idea what's going on?

Thanks again.

[Yi Wu]

Hi David,

Can you just try LWSampler (which is the default engine)? I am not sure what's the problem with RejectionSampler. I will investigate it. 

Well, usually you can just use LWSampler. It is the likelihood weighing algorithm (or saying Importance Sampling). RejectSampler is rarely used.

For the comment about the likelihood, it is just for reference. We compute this using some irrelevant Java code via dynamic programming. The main purpose is to compute the exact posterior (Exact result, not from the sampling process), so that we could have a correct answer to measure whether our inference engine is producing the correct output.

Hopefully my answer will help.

Best

--
You received this message because you are subscribed to the Google Groups "blog-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blog-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blog-user/3e8e2219-49b9-4e65-bbd5-6e72a8da6aa3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Wadden]

OK, this all make sense. Thanks for the quick response.

[Lei Li]

One more comment, the enumeration result is calculated separately for purpose of verifying the correctness.

Cheers,

Lei

To view this discussion on the web visit https://groups.google.com/d/msgid/blog-user/53aec16c-f157-458e-ae2a-45fcbfbe4747%40googlegroups.com.

[David Wadden]

That makes sense, thanks for clarifying. Final question: 

Short version: is there a hack that would allow me for me to compute the posterior by exhaustive enumeration for a varying number of observations? Even if there's just some Java code you could send me to do the trick, that would be great.

Long version: I'm doing some runtime profiling of inference quality in Venture (roughly, MIT's attempt at probabilistic programming). I'm trying to profile the relationship between inference time and number of observations. I'd like to measure inference time not in the wall time required to perform a fixed number of iterations, but in the wall time required to achieve a certain level of inference quality. Inference quality here could be measured, for instance, as information divergence between the approximate posterior from N approximate samples, vs. the true posterior computed from exhaustive enumeration. That's why it would be nice to be able to compute said true posterior. Let me know if it's possible.

Thanks for the help!

[Lei Li]

For urn-ball, we have code.

For general models, it might be difficult.

Cheers,

Lei

To view this discussion on the web visit https://groups.google.com/d/msgid/blog-user/3c01066d-c60e-4478-acd2-bc992a1d0970%40googlegroups.com.

[David Wadden]

Urn-ball would be perfect.

Is there any way I could use it?