Report : The likelihood calculation of Particle filter in Nimble

[kei]

Hi all nimble users!

Thank you for answering questions every day and maintaining nimble.

It is very helpful and I appreciate it very much.

This is a report that can make nimble more efficient tool.

In this nimbleSMC version, log_likelihood in ParticleFilter is calculated by

log_likelihood = log(1/M * sum(weight)). # M and weight mean the number of particles

# and the weight of each particles respectively

It is from p.6~7 in paper below.

But actually, it sometimes causes the underflow problem and I got poor-estimation problems because of this underflow.

So I suggest that nimble should use this calculation,

weight_log = log(weight)  # weight means the weight for each particles

max_w = max(weight_log)

weight = exp(weight_log - max_w)

log_likelihood = log(1/M * sum(weight_log)) + max_w

This is the same meaning as the calculation that is used in nimble now.

You can modify these formulas and check easily.

I've attached the test code that show the problem.

This underflow problem can make ParticleFilter less effective because of the lower-weight particles. And in some cases, underflow may occur for most particles due to the larger absolute value of the log-likelihood in higher dimensional estimation.

This problem is discussed in the book below(sorry this is written in Japanese).

Thank you for reading and I'm glad if you think about this report.

Thanks,

Kei

Paper : Michaud N, De Valpine P, Turek D, Paciorek CJ, Nguyen D (2021). “Sequential Monte Carlo Methods in the nimble and nimbleSMC R Packages.” Journal of Statistical Software, 100(3), 1–39. doi:10.18637/jss.v100.i03.

Book : 樋口知之(2011), データ同化入門,   (https://www.amazon.co.jp/-/en/%E6%A8%8B%E5%8F%A3-%E7%9F%A5%E4%B9%8B/dp/4254127863)

[Perry de Valpine]

Dear Kei,

Thanks for this message. Yes, the "log-sum-exp" trick as you've outlined is important for numerical stability, to avoid underflows. We do use the approach just as you've outlined in many places in nimble code, and I believe it is used in relevant places in the particle filter code in nimbleSMC. From your message, it sounds like you were referring to the paper about nimbleSMC, but were you also looking at the code? The paper describes the math of the algorithm steps but does not describe every implementation detail such as the log-sum-exp trick for numerical stability. Can you please let us know if you find a lack of using this trick in the code of the currently released version of nimbleSMC? I took a quick look to check on this but might have missed something. Thank you for giving attention to this issue.

Best wishes,

Perry

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/6dde2cfc-63f9-412a-8601-f066e51b4712n%40googlegroups.com.