Quantifying noise in stochastic simulations

[Sebastián Espinel Ríos]

Hello,

We have been running stochastic simulations using COPASI-BasiCO. So far, we have been quantifying noise as the fano factor; that is, the variance divided by the mean. We have been doing this at equidistant sampling times; for example, for a simulation of  2,500 s, we compute the mean, standard deviation, and thus the fano factor, every 10 s.

At steady state, we have seen some unexpected fluctuations in the fano factor, which become less prominent when we increase the number of trajectories. Nevertheless, they are always there. We would like to ask the community if you could share your experience about the best way to calculate the statistics of stochastic simulations to avoid such things.

Any comment would be highly appreciated.

Best,

Sebastián

[juergen]

Hi Sebastián,

I am not sure whether I fully understand what you are trying to do. What do you mean by "unexpected fluctuations" in the Fano factor at steady state? As you are estimating the Fano factor numerically from the sample standard deviation and sample mean, you would expect it to fluctuate and the fluctuations to become less with increasing number of tractories, would you not? This is due to the sampling error.

Best wishes,
Juergen

[Sebastián]

Thanks Juergen. Yes, that is what we also suspected. I have two additional questions:

1) in your opinion, what is the minimum number of trajectories for "acceptable" results and 

2) are these fluctuations in the fano factor related to other parts of the computational process, i.e., how we calculate the mean and the variance or how we divide these two things (besides the number of trajectories)? If yes, there is a computational way to minimize such errors. 

Best,

Sebastián

[juergen]

Dear Sebastián,

To the best of my knowledge,

(1) there is no one minimum number of trajectories for "acceptable" results, as this is dependent on the specific model and the resulting underlying probability distributions, and

(2) it could very well be that there is differences between different ways of calculating the Fano factor from a sample, i.e. estimators for the Fano factor. I would intuitively also just divide the sample variance by the sample mean, but I would advise to ask a statistician for potential better estimators or have a look in to the literature (e.g. https://doi.org/10.1016/j.jneumeth.2010.04.012).

Hope this helps.

Best wishes,

Juergen