.loadshp parameters

[markb]

Hi Sandro,

Now that the Load Shapefile dialog in the SL-gui has new "Geometry
storage" options, will these be available from the CLI? Maybe as
additional optional parameters or something. I looked in the
CLI .help just to make sure, but I didn't notice any changes. BTW,
things are going really well for me with the RC's so far, great job.

Thanks,
Mark

[a.furieri]

Hi Mark,

thanks for your contribute: you are obviously
right.
.loadshp must support the same options available
in GIS and GUI tools.
and this must apply to spatialite_tool as well.

------------------

But now it's my turn to pose a *QUESTION*:

I recently noticed that several SHPs do actually
contains 'crazy' geometries, such as:
- linestrings containing only 1 vertex
[not at all a line]
- rings containing less than 4 vertices
[not at all a ring]
- unclosed rings [first vertex does not
coincides with the last one]

such 'crazy' geoms not only represents
conceptual aberrations: they are *harmful*,
because they easily causes GEOS functions
to crash.

PostGIS simply refuses to import such oddities:
you'll get a fatal message error, and the 'crazy'
geom isn't inserted into the DBMS.
This seems to be a sound and safe approach.

But this may cause concern to users, because
they can be frustrated while attempting to
load some SHP *apparently* valid [AFAIK you
can import these bad SHPs in ArcGIS without
any complaint]

So standing things, my trouble is:
- ignoring this issue at all, this way supporting
full compatibility with other widespread GIS tools,
but leaving users alone while fighting against
'mysterious' sporadic crashes.
- completely refusing to import a whole SHP if a
single offending geom is found
- partially importing the SHP, simply ignoring
any offending geom but accepting the good ones
- silently curing unclosed rings when possible,
implicitly inserting the missing last vertex

I'm not really sure about 'best strategy'.
I'm eventually evaluating to introduce a set
of options such as:

--import-strong-check *DEFAULT*
--import-relaxed-check
--import-ignore-check [corresponding to current implementation]

Anyway, I've already implemented 'strong' checking
in shp_doctor; but this is simply useful for diagnostic
purposes, so it's not a 'care' for this specific issue.

Any useful help, hint consideration etc coming from
the SpatiaLite community is welcome.

bye,
Sandro

[markb]

As far as importing invalid shapes into ArcGIS goes, that's "backwards
compatibility"! ;-)
3.x was really horrible (others too), 9.x covers more scenarios, but
all corner cases, who knows.

It seems to me that a two prong strategy would be required against
these assaults. The 3 or 4 "check"ing methods as you suggested, but
also some basic "repair" options via geom sql. In PostGIS these
"repair" functions might not be very useful, but with SQLite's
VirtualTable functionality utilized by your VirtualShape (VS)
extension, an equally sized hole is left in the "firewall". I'm
guessing that anything other than "ignoring" or "refusing" errors via
VS wouldn't work (maybe all or nothing), and possibly not even those?
The "silently correct" you suggest as a "check" method would also be
very useful separated into spatial sql functions that return recovered
geometries; something that could "fix" bow-ties, invalid overlaps and
the other crazy geometries you mentioned. These would not only be
useful for VS geometries, but maybe also indirectly for converted WKT
and WKB strings.

We definitely need that "import-ignore-check" though, since if
SpatiaLite is going to be the Open GeoDB standard were advocating and
priming it for, it will need to be a lossless and easy exchange format
(at the price of a few errors if necessary). If you provide
integrated diagnostics upon import, and users don't choose to use
them, then that is there fault.

Is this partly a problem with GEOS (i.e. the crashing), or are they
taking the premise that the burden of trash checking should be on the
library user? I can certainly appreciate either perspective, or
preferably a little of both (e.g. GEOS users should check, but GEOS
might handle errors more gracefully).

The flexibility that SpatiaLite provides with its VirtualShape
functionality definitely complicates "imports" relative to what
PostGIS faces, but maybe in the end PostGIS will adopt some SL
concepts in return.

Since yesterday we finally broke 200 users, nice milestone. The
community here is progressing pretty well.

Now I need to get ready for a snow storm (YES!), shin high is a very
bid deal where I'm at.

-Mark

[Andrea P.]

I think the best strategy is probably
"do not silently correct".
To avoid the changes was not known from the user.

The other solution to import all the geometry is also good but only if
there is a function to test and find the bugged geometry.

Like the postgis DB where there is a function named
ST_IsValid(geometry) .

So if the DB have some bad-geometry loaded is always possible to do a

SELECT some-idi FROM table WHERE ST_IsValid(geometry)=false;

to find is and what geometry are bad.

Andrea.

[markb]

IsValid() function:
http://www.gaia-gis.it/spatialite/spatialite-sql-2.3.1.html#p4