Particle deprivation

[ganlu...@gmail.com]

How do we introduce randomness in the particle filter, ie. prevent particle deprivation? Our particles keep collapsing to a single terminal state without reaching the true terminal state. We implemented a default_action in the BasicPOMCP solver, but it does not seem to be solving this issue.

Particle filter update error: all states in the particle collection were terminal.

[Mykel Kochenderfer]

Try more particles or a broader observation and transition model.

[Zachary Sunberg]

Can you share your code with me so I can see where you're starting from (perhaps in a gist https://gist.github.com/ if you don't know how else to share)?

default_action won't solve this problem because the error is in the particle filter update, not POMCP itself.

All of the simulation functions (simulate(...), stepthrough(...), sim(...), etc.) should take an updater as an argument after the policy/planner. You can create a particle filter with N particles with

SIRParticleFilter(pomdp, N)

where pomdp is your pomdp object. Try increasing N first.

You should also investigate what is going on more closely. Use the stepthrough function to simulate. Examine the beliefs to see what is happening. The error should occur after

all(isterminal(pomdp, s) for s in particles(b))

is true.

If you do need to add noise, you can implement a custom resampler for a SimpleParticleFilter in ParicleFilters.jl (https://github.com/JuliaPOMDP/ParticleFilters.jl), or you can  implement your own more robust particle filter. Examples of both resamplers and a custom particle filter are here: https://github.com/zsunberg/ContinuousPOMDPTreeSearchExperiments.jl/blob/master/src/updaters.jl

[ganlu...@gmail.com]

Thanks! It seems to have worked with an increased number of particles for now.