new release: spatialite 5.0.1 RC0

[Alessandro Furieri]

Hi list,

the release cycle of a new updated version (5.0.1) has started 

and a first Release Candidate (RC0) is now available for 

community testing.

please quickly report any issue related to the Release Candidate;

if no problems will be found the final stable release is expected 

during next week.

this simply is a very quiet bug fixing release; no new 

functionalities were introduced but many annoying bugs have

been definitely fixed. 

even more important, now SpatiaLite nicely supportes the very 

recent PROJ-7.2.1 and GEOS-3.9.0

upgrading from version 5.0.0 is warmly reccommended.

the source code is available from here:

=======================================

http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1-rc0.tar.gz

http://www.gaia-gis.it/gaia-sins/libspatialite-5.0.1-rc0.zip

http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-5.0.1-rc0.zip

http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-5.0.1-rc0.tar.gz

the pre-compiled binaries for Windows are here:

===============================================

http://www.gaia-gis.it/gaia-sins/windows-bin-NEXTGEN-x86/spatialite-tools-5.0.1-rc0-win-x86.7z

http://www.gaia-gis.it/gaia-sins/windows-bin-NEXTGEN-amd64/spatialite-tools-5.0.1-rc0-win-amd64.7z

http://www.gaia-gis.it/gaia-sins/windows-bin-NEXTGEN-x86/mod_spatialite-5.0.1-rc0-win-x86.7z

http://www.gaia-gis.it/gaia-sins/windows-bin-NEXTGEN-amd64/mod_spatialite-5.0.1-rc0-win-amd64.7z

IMPORTANT NOTICE:

=================

there is no GUI tool supporting this Release Candidate.

I'll explain in a following message the road-map leading to the

release of the next versions of librasterlite2 and GUI tool.

bye Sandro

[Bas Couwenberg]

spatialite-tools 5.0.1-rc0 has readosm enabled, but the spatialite_osm_{map,net,raw} are not built:

"""

checking for readosm... yes

checking for library containing readosm_version... -lreadosm

"""

Only spatialite_osm_{filter,overpass} get built.

[Bas Couwenberg]

This patch works for me:

https://salsa.debian.org/debian-gis-team/spatialite-tools/-/blob/experimental/debian/patches/readosm.patch

[Alan Snow]

The conda-forge builds are failing ref. Any suggestions would be appreciated.

[a.fu...@lqt.it]

On Tue, 26 Jan 2021 13:38:41 -0800 (PST), Alan Snow wrote:
> The conda-forge builds are failing ref [10]. Any suggestions would be
> appreciated.
>

Hi Alan,

remark #1: a failing test says very little aboute the underlaying
cause.
in these cases it's always useful carefully checking the logfiles
generated by each failing test so to discover the real issue.
you'll find such logfiles in the "test" directory.
example: the "check_init2" test corresponds to "check_init2.log"

remark #2: as rule-of-the-thumb any massive failure is usually
caused by a single critical issue propagating everywhere.
and this exactly is our current case.

I've been able to exactly replicate your issue; the offending
cause is in that you've not enabled Topology support in your
local build.

going in more depth:
- SpatiaLite 5.0.0 introduced a new extended DB layout
intended to fully support RasterLite2 integration.
please see:

https://www.gaia-gis.it/fossil/libspatialite/wiki?name=Upgrading+existing+DB-files+to+5.0.0

- If Topology support is not enabled creating a genuine
SpatiaLite-5 DB layout is simply impossible, and
consequently an old-style SpatiaLite-4 schema
will be created instead.
this is not a risky situation, because Spatialite 5
can safely operate even on obsolete DB schemas,
but as an obvious consequence many SQL functions
will then start to unexpectedly fail.

- the net final outcome will be a massive series
of failures in the testcoverage.
shielding against this is extremely difficult
if not impossible, because the consequences of
an invalid DB schema will propagate practically
everywhere.


HOW TO RESOLVE THIS ISSUE
==========================
you are just required to explicitly enable
Topology support when building libspatialite:

./configure --enable-rttopo --enable-gcp

this in turn requires compiling and installing
first librttopo, that is not difficult at all.

https://git.osgeo.org/gitea/rttopo/librttopo/releases
https://git.osgeo.org/gitea/rttopo/librttopo/archive/librttopo-1.1.0.tar.gz

------------

conclusion: it would be probably wise changing the ./configure
default settings; I'll start a discussion about this topic in a
separate thread for the sake of clarity.

bye Sandro

[Xylar Asay-Davis]

Hi Sandro,

Thanks for your detailed reply, it's been really helpful and is very much appreciated.

Sorry for not getting further with our own debugging.  I took a look at the logs this morning and realized the same thing you suggested: that we needed rttopo.

The Linux build is now working great!  And I think I have a handle on the Windows build.

Three tests are still failing for us on the OSX build:

==================================================

   libspatialite 5.0.1-rc0: test/test-suite.log

==================================================

# TOTAL: 107

# PASS:  104

# SKIP:  0

# XFAIL: 0

# FAIL:  3

# XPASS: 0

# ERROR: 0

.. contents:: :depth: 2

FAIL: check_extension

=====================

load_extension() error: dlopen(mod_spatialite.dylib, 10): image not found

FAIL: check_sql_stmt_extension

==============================

****************** testing in load_extension mode

Test case: NumPoints - bad blob (error)

load_extension() error: dlopen(mod_spatialite.dylib, 10): image not found

FAIL: check_get_normal_zoom_extension_load

==========================================

load_extension(mod_spatialite) error: dlopen(mod_spatialite.dylib, 10): image not found

Indeed, if I take a look in src/.libs, I don't see mod_spatialite.dylib:

$ ls src/.libs/

libspatialite.7.dylib libspatialite.lai mod_spatialite.lai

libspatialite.dylib mod_spatialite.7.so mod_spatialite.so

libspatialite.la mod_spatialite.la

I think we were seeing this in 5.0.0 but we were in kind of a rush and decided to just disable these 3 tests.  but now that we have a chance to work with you, it would be great to see if these can be fixed or if we're doing something wrong.

Cheers and thanks again,

Xylar

[mj10777]

 mod_spatialite.so was created instead of mod_spatialite.dylib. 

I don't remember if 'dylib' is also looked for. 

Sandro: isn't the checking of the dll, so extension done in sqlite3?  

[a.fu...@lqt.it]

On Tue, 26 Jan 2021 09:42:53 -0800 (PST), Bas Couwenberg wrote:
> This patch works for me:
>

Hi Bas,
thanks a lot; your patch definitely sanes the issue.
there was an unnoticed regression caused by a recent
commit.

updated sources (RC1) are available from here:

http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-5.0.1-rc1.zip
http://www.gaia-gis.it/gaia-sins/spatialite-tools-sources/spatialite-tools-5.0.1-rc1.tar.gz

bye Sandro

[a.fu...@lqt.it]

On Wed, 27 Jan 2021 02:07:13 -0800 (PST), 'mj10777' via SpatiaLite
Users wrote:
> On Wednesday, 27 January 2021 at 10:26:00 UTC+1 xylar...@gmail.com
> wrote:
>
>> The Linux build is now working great! And I think I have a handle on
>> the Windows build.
>>

note: as per my personal direct experience with mingw32/mingw64
compilers the whole testcoverage runs smoothly on Windows, but
only when compiling 64bit code.
few tests fail in 32bit mode due to some slightly different numeric
results returned by PROJ, GEOS and GCP.

the problem is in that x86 floating point registers (inherited from
the very old design of the initial 8087 coprocessor) have an odd
intrinsic precision of 80 bits, that doesn't matches the standard
64 bits usually expected for doubles, and this could easily lead
to a nightmare of rounding and truncation errors.

the more modern and rational design of x86_64 has 64 bits floating
point registers, so the problem never arise.

>> Three tests are still failing for us on the OSX build:
>>
>

> mod_spatialite.so was created instead of mod_spatialite.dylib.
> I don't remember if 'dylib' is also looked for.
> Sandro: isn't the checking of the dll, so extension done in sqlite3?
>

my personal experience about MacOsX is very limited.

in a very general way there is a conceptual difference
between a shared (aka dynamic) library and a dynamically
loadable module.
- a shared library is subject to "early binding", and
is immediately loaded when the main application starts;
if it cannot be loaded for any reason the app can't start.
- a dynamically loadable module is subject to "late binding",
and is effectively loaded only when strictly required.
if it cannot be loaded the main application can survive
unharmed (usually by suppressing some functionalities).
it's obviously intended to support a flexible architecture
based on plug-ins and dynamic modules.

in both Unix/Linux and Windows there is no practical
difference; in both cases we'll always have a .so (Linux)
or a .dll (Windows).

this is not true on MacOsX, that applies a strong
distinction between .so and .dylib, but if I remember
well loading mod_spatialite.so should work also
on MacOsX

a desperate workaround (strongly discouraged) could
be the one to manually rename mod_spatialite.so
to mod_spatialite.dylib (or even better create
a link with a different name)

bye Sandro

[Jeff McKenna]

Hi Sandro,

No issues with building on Windows (Visual Studio) for
libspatialite-5.0.1-rc0 and spatialite-tools-5.0.1-rc1

Seems to work well with GDAL 3.2.1, PROJ 7.2.1, GEOS 3.9.0, SQLite
3.34.1, etc.

Thanks again, from the east coast of Canada,

-jeff




--
Jeff McKenna
GatewayGeo: MapServer Consulting and Training Services
co-founder of FOSS4G
http://gatewaygeo.com/

[David Anderson]

I am unable to load the Windows binaries.

I am getting a procedure entry point _ZNSt12runtime_errorC1EOS  could not be located in the dynamic link library libgeos-3-9-0 when trying to load the 64 bit mod_spatialite extension in Python 3.8.3 (MSC v.1916 64 bit (AMD64)).  More or less the same thing when trying to load the 32 bit extension in SQLite Studio.

[a.fu...@lqt.it]

On Wed, 3 Feb 2021 22:40:12 -0800 (PST), David Anderson wrote:
> I am unable to load the Windows binaries.I am getting a procedure

> entry point _ZNSt12runtime_errorC1EOS could not be located in the
> dynamic link library libgeos-3-9-0 when trying to load the 64 bit
> mod_spatialite extension in Python 3.8.3 (MSC v.1916 64 bit (AMD64)).
> More or less the same thing when trying to load the 32 bit extension
> in SQLite Studio.
>

Hi David,

this seems to be the same issue discussed here in full detail:

https://groups.google.com/g/spatialite-users/c/ecEEpxbDROk/m/GE13jzg6AAAJ

very short summary:
mod_spatialite (more precisely GEOS and PROJ) strictly depends on the
platform C++ runtime support.

the pre-built binary packages for Windows are always based on the
latest version, but complex frameworks based on Java, Python, PHP etc
usually deploy their own version of the C++ runtime, that could be
easily obsolete.

it's absolutely not a SpatiaLite issue, it's basically a problem
of badly mixing DLLs of different generations.

suggested hint: updating the frameworks (Java, Python etc) to
their more recent version usually resolves this kind of issues.

bye Sandro