new release: 4.4.0-RC0

[sandro furieri]

Hi list,

I'm glad to announce that version 4.4.0 (Release Candidate 0) is now available.

http://www.gaia-gis.it/gaia-sins/libspatialite-4.4.0-RC0.tar.gz
http://www.gaia-gis.it/gaia-sins/libspatialite-4.4.0-RC0.zip

http://www.gaia-gis.it/gaia-sins/spatialite-tools-4.4.0-RC0.tar.gz
http://www.gaia-gis.it/gaia-sins/spatialite-tools-4.4.0-RC0.zip

and here is the accompanying documentation:

http://www.gaia-gis.it/gaia-sins/spatialite-sql-4.4.0.html
https://www.gaia-gis.it/fossil/libspatialite/wiki?name=ST_Cutter
https://www.gaia-gis.it/fossil/libspatialite/wiki?name=GPX+tracks
https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN
https://www.gaia-gis.it/fossil/libspatialite/wiki?name=topo-intro

Short summary of the coolest new features introduced by 4.4.0
-------------------------------------------------------------------------------------------
1. ISO Topology-Geometry and Topology-Network support
2. KNN (K-Nearest Neighbors) support based on VirtualKNN
3. ST_Cutter(), an advanced function intended to precisely
    cut a full set of Input Geometries against another set of
    Blade Geometries.
4. a new SQL function for creating MultiLinestrings from
    GPX tracks fully preserving GPS timestamps.
5. several bugfixes and performance optimizations.

happy testing,
Sandro

[robertoangeletti]

Hi Sandro,

> 4. a new SQL function for creating MultiLinestrings from
> GPX tracks fully preserving GPS timestamps.

It also mantains gps precision for each point ? It should be great !


Thank you

Roberto

[Roberto Angeletti]

Hi Sandro,

attached, you can find a GPX example  that has also   HDOP   and  VDOP   fields, that are  the horizontal and vertical   error  of the GPS point.

            <trkpt lat="41.903897" lon="12.502165">
                <ele>98.934204</ele>
                <time>2015-10-13T05:45:12Z</time>
                <hdop>2.000000</hdop>
                <vdop>9.600000</vdop>
            </trkpt>

I created it with  "EasyTrails"  app  for iPhone.

I hope my GPX  can be useful to implement the GPS error  in SpatiaLite.

A presto

Roberto

[a.fu...@lqt.it]

On Mon, 14 Dec 2015 06:59:46 +0100, robertoangeletti wrote:
>> 4. a new SQL function for creating MultiLinestrings from
>> GPX tracks fully preserving GPS timestamps.
>
> It also mantains gps precision for each point ? It should be great !
>

ciao Roberto,

GPS spatial coordinates XY are expressed in decimal degrees,
the Z elevation is expressed in meters.
the expected accuracy usually is about 2-5m.
GPS timestamps have a resolution of 1 second.

SpatiaLite internally stores all coordinates as IEEE 754
floating points double precision (64 bit), this corresponding
to a precision of 17 digits.

All this considered we can safely assume that both the spatial
coordinated and the timestamps will be correctly represented
as XYZM points without introducing any rounding or truncation.

bye Sandro

[a.fu...@lqt.it]

On Mon, 14 Dec 2015 11:22:36 +0100, Roberto Angeletti wrote:
> Hi Sandro,
>
> attached, you can find a GPX example  that has also   HDOP  
> and  VDOP   fields, that are  the horizontal and vertical  
> error  of the GPS point.
>

> I hope my GPX  can be useful to implement the GPS error  in
> SpatiaLite.
>

ciao Roberto,

the intended scope of the new GPX related function is the
one to transform GPX tracks into standard MultiLinestrings
geometries of XYZM dimensions (the M coordinate corresponding
to the timestamp encoded as a julian day number).

parsing both HDOP and VDOP (if eventually declared, because
they are not mandatory attributes and many real world
implementations does not support them) is not a problem.
the problem is that we absolutely have no further room
to store them somewhere within the MultiLinestring Geometry,
because we've already used all the four available coordinates:
X (longitude), Y (latitude), Z (elevation) and M (timestamp
encoded as a julian day number).

bye Sandro

[linux...@gmail.com]

On Sunday, December 13, 2015 at 8:33:56 PM UTC+1, sandro furieri wrote:

happy testing

The splite_connection_pool symbol introduced in 4.2.0 is no longer provided, this is an ABI break that strictly speaking requires a SONAME bump. It doesn't look like any spatialite reverse dependencies use it, so you may get away with just the incremented revision.

The new check_topology2d, check_topology3d & check_topoplus tests segfault on Debian unstable.

gdb output for all three is pretty much the same, I'm not quite sure what to make of it:

 Program received signal SIGSEGV, Segmentation fault.
 do_read_edge_by_face (callback_name=0x7ffff77b58b7 "callback_getEdgeByFace", errmsg=0x7fffffffd610, box=0x99, fields=153, face_id=0, list=0x762220, stmt=0x762038) at topo_callbacks.c:1313
 1313              sqlite3_bind_double (stmt, 3, box->xmin);
 (gdb) bt
 #0  do_read_edge_by_face (callback_name=0x7ffff77b58b7 "callback_getEdgeByFace", errmsg=0x7fffffffd620, box=0x99, fields=153, face_id=0, list=0x762220, stmt=0x762038) at topo_callbacks.c:1313
 #1  callback_getEdgeByFace (lwt_topo=0x747c90, ids=<optimized out>, numelems=0x7fffffffd6e4, fields=153, box=0x99) at topo_callbacks.c:4691
 #2  0x00007ffff6195115 in ?? () from /usr/lib/liblwgeom-2.2.so.2
 #3  0x00007ffff61967bf in ?? () from /usr/lib/liblwgeom-2.2.so.2
 #4  0x00007ffff753f9c1 in gaiaAddEdgeModFace (accessor=accessor@entry=0x747c90, start_node=start_node@entry=8, end_node=end_node@entry=9, ln=ln@entry=0x6f9950, skip_checks=skip_checks@entry=0) at gaia_auxtopo.c:2863
 #5  0x00007ffff7534c3a in fnctaux_AddEdgeModFace (xcontext=0x762348, argc=<optimized out>, xargv=<optimized out>) at gaia_topology.c:1554
 #6  0x00007ffff6433da7 in ?? () from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
 #7  0x00007ffff643cfa7 in sqlite3_step () from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
 #8  0x00007ffff643e01a in sqlite3_exec () from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
 #9  0x0000000000402b66 in do_level3_tests (retcode=0x7fffffffdebc, handle=0x681ff8) at check_topology2d.c:1345
 #10 main (argc=<optimized out>, argv=<optimized out>) at check_topology2d.c:3037

The test-suite.log is attached.

[linux...@gmail.com]

On Tuesday, December 15, 2015 at 10:43:24 PM UTC+1, linux...@gmail.com wrote:

On Sunday, December 13, 2015 at 8:33:56 PM UTC+1, sandro furieri wrote:

happy testing

The splite_connection_pool symbol introduced in 4.2.0 is no longer provided, this is an ABI break that strictly speaking requires a SONAME bump. It doesn't look like any spatialite reverse dependencies use it, so you may get away with just the incremented revision.

The new check_topology2d, check_topology3d & check_topoplus tests segfault on Debian unstable.

On i386 two additional tests fail:

 FAIL: check_sql_stmt
 ====================
 ...
 Unexpected value at 1: 24.0|
 Expected value was   : 22.0|

 FAIL: check_control_points
 ==========================

 Unexpected result E{-161.0000000000,1.0000000000,0.0000000000}, N{-64.0000000000,-0.0000000000,1.0000000000}
 Expected: E{-161.0000000000,1.0000000000,-0.0000000000}, N{-64.0000000000,-0.0000000000,1.0000000000} 

Full build log at: https://buildd.debian.org/status/fetch.php?pkg=spatialite&arch=i386&ver=4.4.0~rc0-1~exp1&stamp=1450220426

On powerpc an additional test fails because spatialite is configure with --disable-epsg there due to Debian Bug #649302 (https://bugs.debian.org/649302):

 FAIL: check_virtualknn
 ======================
 
 CREATE VIRTUAL TABLE "knn" error: table knn already exists
 unknown SRID: 32532
 unknown SRID: 32532	<no such table: gpkg_spatial_ref_sys>
 ...
 unknown SRID: 32532
 unknown SRID: 32532	<no such table: gpkg_spatial_ref_sys>
 Check KNN #10: unexpected failure
 FAIL check_virtualknn (exit status: 237)

 
Full build log at: https://buildd.debian.org/status/fetch.php?pkg=spatialite&arch=powerpc&ver=4.4.0~rc0-1~exp1&stamp=1450220442

[sandro furieri]

Il giorno martedì 15 dicembre 2015 22:43:24 UTC+1, linux...@gmail.com ha scritto:

On Sunday, December 13, 2015 at 8:33:56 PM UTC+1, sandro furieri wrote:

happy testing

The new check_topology2d, check_topology3d & check_topoplus tests segfault on Debian unstable.

Hi Bas,

I easily imagine that on Debian unstable you are using the infamous lwgeom-2.2.0, that was
prematurely released and that surely contains several critical bugs affecting lwgeom-topology.
all them were definitely fixed only after the official release of the buggish 2.2.0, so the
correctly working patched version is currently supported by postgis-trunk only.

a practical test I've checked right now on Debian 8 Jessie (64 bit):

1. building libspatialite-4.4.0-RC0 against liblwgeom-2.1.4 (system package); the
    test coverage successfully completes with zero errors (anyway Topology was
    automatically disabled because this obsolete version completely lacks any
   lwgeom-topology support)

2. building against liblwgeom-2.2.0
   I've got exactly the same failures you've already noticed.

3. building against the latest liblwgeom-trunk [1]
    all test successfully pass, zero errors.

conclusion: lwgeom-2.2.0 is completely unfit to correctly support lwgeom-topology.

[1] https://github.com/postgis/postgis/archive/svn-trunk.zip

bye Sandro

[sandro furieri]

Il giorno mercoledì 16 dicembre 2015 01:24:24 UTC+1, linux...@gmail.com ha scritto:

On Tuesday, December 15, 2015 at 10:43:24 PM UTC+1, linux...@gmail.com wrote:

On Sunday, December 13, 2015 at 8:33:56 PM UTC+1, sandro furieri wrote:

happy testing

On i386 two additional tests fail:

 FAIL: check_sql_stmt
 ====================
 ...
 Unexpected value at 1: 24.0|
 Expected value was   : 22.0|

 FAIL: check_control_points
 ==========================

 Unexpected result E{-161.0000000000,1.0000000000,0.0000000000}, N{-64.0000000000,-0.0000000000,1.0000000000}
 Expected: E{-161.0000000000,1.0000000000,-0.0000000000}, N{-64.0000000000,-0.0000000000,1.0000000000} 

Full build log at: https://buildd.debian.org/status/fetch.php?pkg=spatialite&arch=i386&ver=4.4.0~rc0-1~exp1&stamp=1450220426

Hi Bas,

I'm unable to confirm any issue specifically related to i386.

I've just set up a Debian 8 Jessie Virtual Machine (kernel: 3.16.0-4-686-pae) and
all tests pass without any problem.
Just to be absolutely sure I've tested both geos-4.2.0 and geos-4.5.0, and it
works nicely in both configurations..

bye Sandro

[a.fu...@lqt.it]

On Tue, 22 Dec 2015 14:37:43 -0800 (PST), sandro furieri wrote:
> Il giorno mercoledì 16 dicembre 2015 01:24:24 UTC+1,
> linux...@gmail.com ha scritto:
>
>> On Tuesday, December 15, 2015 at 10:43:24 PM UTC+1,
>> linux...@gmail.com wrote:
>>
>>> On Sunday, December 13, 2015 at 8:33:56 PM UTC+1, sandro furieri
>>> wrote:
>>>
>>>> happy testing
>>>
>>> On i386 two additional tests fail:
>>>
>>> FAIL: check_sql_stmt
>>> ====================
>>> ...
>>> Unexpected value at 1: 24.0|
>>> Expected value was : 22.0|
>>>
>>> FAIL: check_control_points
>>> ==========================
>>>
>>> Unexpected result E{-161.0000000000,1.0000000000,0.0000000000},
>>> N{-64.0000000000,-0.0000000000,1.0000000000}
>>> Expected: E{-161.0000000000,1.0000000000,-0.0000000000},
>>> N{-64.0000000000,-0.0000000000,1.0000000000} 
>>
>> Full build log at:
>>
>
> https://buildd.debian.org/status/fetch.php?pkg=spatialite&arch=i386&ver=4.4.0~rc0-1~exp1&stamp=1450220426

>> [1]

>
> Hi Bas,
>
> I'm unable to confirm any issue specifically related to i386.
>
> I've just set up a Debian 8 Jessie Virtual Machine (kernel:
> 3.16.0-4-686-pae) and
> all tests pass without any problem.
> Just to be absolutely sure I've tested both geos-4.2.0 and
> geos-4.5.0,
> and it
> works nicely in both configurations..
>

an interesting follow up.
I've performed yet another test, and this time I was finally able to
reproduce the failures you already detected, anyway what I've
discovered is completely unexpected and really puzzling.

1. compiling libspatialite passing to gcc the "-g -O0" flags (no
code optimization at all): full success

2. if instead we pass "-g -O2" (standard optimization) the test
coverage reports two failures.

it's probably interesting to note that on Win32 the MinGW compiler
(a strict gcc derivative) always returns the expected results even
when "-O2" is specified.

it seems to be an artifact specifically caused by the Debian i386
compiler, possibly because it's attempting to apply too aggressive
and unsafe code optimizations.

bye Sandro

[sandro furieri]

Il giorno martedì 15 dicembre 2015 22:43:24 UTC+1, linux...@gmail.com ha scritto:

On Sunday, December 13, 2015 at 8:33:56 PM UTC+1, sandro furieri wrote:

happy testing

The splite_connection_pool symbol introduced in 4.2.0 is no longer provided, this is an ABI break that strictly speaking requires a SONAME bump. It doesn't look like any spatialite reverse dependencies use it, so you may get away with just the incremented revision.

Hi Bas,

"splite_connection_pool" simply was a global array used by earlier versions in order to partially
circumvent the thread-safety limitations imposed by GEOS (< 3.5.0).
the intended scope of such an array was to safely support a maximum of 64 concurrent threads,
and it was never intended to be directly accessed, it was just a common storage area supporting
all the connection-oriented API.

now finally GEOS 3.5.0 is fully reentrant and can safely support any arbitrary number of concurrent
threads, so the intermediate array is no longer required and  consequently has been completely
removed when the --enable-geosreentrant ./configure option is activated (and this is the default
setting when GEOS 3.5.0 is actually detected).
 
I imagine that this shouldn't be considered as an API break, because after all any C API will
still continue to correctly work using the same identical signatures as before.
What really changes is simply the internal implementation of few connection-oriented API,
but this is completely transparent and can never have harmful consequences.

bye Sandro

[sandro furieri]

Il giorno mercoledì 16 dicembre 2015 01:24:24 UTC+1, linux...@gmail.com ha scritto:

On powerpc an additional test fails because spatialite is configure with --disable-epsg there due to Debian Bug #649302 (https://bugs.debian.org/649302):

 FAIL: check_virtualknn
 ======================
 
 CREATE VIRTUAL TABLE "knn" error: table knn already exists
 unknown SRID: 32532
 unknown SRID: 32532	<no such table: gpkg_spatial_ref_sys>
 ...
 unknown SRID: 32532
 unknown SRID: 32532	<no such table: gpkg_spatial_ref_sys>
 Check KNN #10: unexpected failure
 FAIL check_virtualknn (exit status: 237)

 

fixed into the Fossil repository since the more recent commit.
now it correctly supports the --disable-epsg option

bye Sandro

[linux...@gmail.com]

Hi Sandro,

Thanks for the comprehensive follow-up and the powerpc fix.

PostGIS 2.2.1 is expected to be released in the next two weeks, that should take care of the topology test failures.

I look forward to the separate rt-topology library to get rid of PostGIS as a dependency for SpatiaLite.

Kind Regards,

Bas

[linux...@gmail.com]

libspatialite & spatialite-tools 4.4.0-RC1 with support for librttopo were released in early May (but not announced). What is the expected timeframe for the final release?

Kind Regards,

Bas