Calculation of Confidence Interval in Castalia

[Jay Yu]

Hi, I am trying to understand how the confidence interval is calculated in Castalia when we run multiple repetitions of the same simulation. 

Below I have the outputs of the value propagation example with 5 nodes and 5 repetitions. 

	Node 0	Node 1	Node 2	Node 3	Node 4
Seed 0	1.8125	1.5625	1.57812	1.60938	1.8125
Seed 3	1	2	2	2	1
Seed 4	0	2.625	2.3125	2.75	2.3125
Seed 1	0	0	0	0	0
Seed 2	0	0	0	0	0

Using CastaliaResults -i ... -c 95 shows the confidence interval to be 0.3821, but for the life of me, I can not manage to replicate the number using manual calculations. 

Other attempts reveal the following

CastaliaResults -i ... -c 95 -n

Confidence Interval

Node 0	Node 1	Node 2	Node 3	Node 4
0.85263	1.39193	1.30525	1.43578	1.19708

Any help would be appreciated.  

[Thanassis Boulis]

Confidence interval is a complex statistical notion.

You can study http://en.wikipedia.org/wiki/Confidence_interval 

And you can study the code of CastaliaResults to find out the formula used in calculating it.

Confidence intervals make sense to use when we know some statistical properties of the underlying process. 

If the results can be al over the place then you are better off seeing all individual points (from all seeds) and try to make sense of the results.

For example in this case you see these zeros which skew the result considerably. Could be a  bug, or maybe expected behaviour. If you take more seeds, the confidence interval will shrink, but this does not reveal anything about your confidence on these being the correct results (e.g. bug free).

--
You received this message because you are subscribed to the Google Groups "Castalia Simulator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to castalia-simula...@googlegroups.com.
To post to this group, send email to castalia-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/castalia-simulator/88bd09ce-12bc-4eb4-9e98-6f63f08260a6%40googlegroups.com?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.