bug in stddev_samp, stddev_pop, var_samp, var_pop

40 views
Skip to first unread message

ggrothendieck

unread,
Feb 2, 2010, 1:18:29 PM2/2/10
to SpatiaLite Users
var_samp,and var_pop are reversed as are stddev_samp and stddev_pop.

C:\tmp2>sqlite3
SQLite version 3.6.18
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .load libspatialite-1.dll
SpatiaLite version ..: 2.3.1 Supported Extensions:
- 'VirtualShape' [direct Shapefile access]
- 'VirtualText' [direct CSV/TXT access]
- 'VirtualNetwork [Dijkstra shortest path]
- 'RTree' [Spatial Index - R*Tree]
- 'MbrCache' [Spatial Index - MBR cache]
- 'VirtualFDO' [FDO-OGR interoperability]
- 'SpatiaLite' [Spatial SQL - OGC]
PROJ.4 Rel. 4.6.1, 21 August 2008
GEOS version 3.1.1-CAPI-1.6.0
sqlite> create table d (x);
sqlite> insert into d values(1);
sqlite> insert into d values(2);
sqlite> insert into d values(3);
sqlite> select var_samp(x), var_pop(x) from d;
0.666666666666667|1.0
sqlite> select stddev_samp(x), stddev_pop(x) from d;
0.816496580927726|1.0

In both cases the sample and population figures are reversed from what
they should be.

The sample variance of 1, 2, 3 is ((1-2)^2 + (2-2)^2 + (3-2)^2)/2 = 1
and the population variance of 1, 2, 3 is
((1-2)^2 + (2-2)^2 + (3-2)^2)/3 = 0.666666666666667.

The sample and population standard deviations are the square roots of
those which are 1 and 0.816496580927726.

a.furieri

unread,
Mar 31, 2010, 6:11:39 AM3/31/10
to SpatiaLite Users
Hi,

I can confirm this issue: the 'population' and 'sample'
functions where badly swapped for both variance and
standard deviation: I've already fixed into SVN

Thanks a lot for noticing and signaling this bug

bye
Sandro580927726.

Reply all
Reply to author
Forward
0 new messages