Rethinking "amphibious mode"

74 views
Skip to first unread message

Brad Hards

unread,
May 25, 2015, 7:21:02 AM5/25/15
to spatiali...@googlegroups.com, even.r...@spatialys.com, Pepijn Van Eeckhoudt, Jukka Rahkonen
Even / Pepijn / Jukka (and other list denizens)

Sandro and I had a short discussion off-list about amphibious mode, which I'm
starting to think may not be such a good idea. A relevant part follows, but
the key question (I think) is whether there is ever a time where you have to
be able to read the file without knowing (at the application level) whether it
is a geopackage or spatialite file?

If not, then explicit selection of mode is better.


On Sun, 24 May 2015 02:06:20 PM [Sandro] wrote:
> the real problem is that we absolutely need to carefully preserve
> some clearly marked distinction differentiating SpatiaLite and GPKG,
> otherwise we'll surely end up before or later with lots of bizarre
> hybrids half splite and half gpkg proliferating on the wild, and
> this certainly isn't in the interest neither of splite nor of gpkg.
[Which I agree with]

> I've absolutely nothing against implementing full gpkg support directly
> into libspatialite, but this must necessarily be done in such a way so
> to make absolutely clear for everyone that they are two separate
> environments presenting different scopes, different requirements and
> (most important than all) different binary encodings.
[On this too!]

> I wasn't exactly enthusiastic when accepting the proposed "amphibious
> mode"; anyway since it's strictly limited to input/read methods it
> doesn't seems to pose any critical danger, and it could potentially
> be useful for several applications (mainly conversion tools and
> universal viewers).
Indeed, and that was the kind of thing I was working on when I suggested it.

> Allowing to completely relax any distinction about binary formats on
> the output/write side is a completely different affair.
> This will effectively open the doors to a couple of very dangerous
> pitfalls:
> - users/developers could then be able (may be inadvertently) to insert
> splite's own geometries in a gpkg file.
> - and the opposite could happen in a splite "vanilla" db; ok, splite
> have several triggers actively preventing to do such a thing, anyway
> triggers can be easily disabled, so they are not at all a bomb-proof
> shield.
Indeed, and I hadn't considered this adverse implication when first suggesting
the idea.

> It's not too much difficult forecasting that after adopting such a
> design choice we'll be then submerged in a very short time by massive
> waves of complaints and recriminations about severe malfunctions
> forbidding any real interoperability, most notably when using
> different versions of libspatialite and may be combined with
> deploying different versions of QGIS, GDAL and alike.
Well, they could all work as long as they all use some nice new form of
spatialite. It'll be broken for everyone else though, and we'd be at fault
(according to some, and probably a reasonable claim).

> In short conclusion: a small change initially intended to facilitate
> a better and simpler interoperability could sadly convert into a
> painful nightmare preventing any safe interoperability.
I agree, and I (after reading your email and thinking this through a bit) now
think that amphibious mode is a bad idea. The problem is that the mode really
only works on read. It has no effect on write. In particular, it doesn't help
prevent putting spatialite binary into geopackage files (as you identified
above).

Instead of amphibious mode, we should have "use geopackage mode" where the
input *has* to be geopackage binary (i.e. spatialite binary is an error), and
the output is geopackage binary.

The default would be "use current spatialite mode" (as for spatialite 4.2.x
behaviour), and surely we'd need a way to swap back, even if that meant
dropping the connection, but an explicit function might be useful for running
tests.

Brad

Even Rouault

unread,
May 25, 2015, 7:42:04 AM5/25/15
to spatiali...@googlegroups.com, Brad Hards, Pepijn Van Eeckhoudt, Jukka Rahkonen
Le lundi 25 mai 2015 13:20:56, Brad Hards a écrit :
> Even / Pepijn / Jukka (and other list denizens)
>
> Sandro and I had a short discussion off-list about amphibious mode, which
> I'm starting to think may not be such a good idea. A relevant part
> follows, but the key question (I think) is whether there is ever a time
> where you have to be able to read the file without knowing (at the
> application level) whether it is a geopackage or spatialite file?
>
> If not, then explicit selection of mode is better.

From GDAL point of view, as there are 2 distinct drivers for GPKG and
Spatialite, GDAL certainly knows what kind of geometry format it will provide
to spatialite. So would be OK if EnableGpkgAmphibiousMode() just become
EnableGpkgMode() (provided that in that mode, the output of Spatialite
functions that return geometries is also GPKG geometries so that you can chain
calls like ST_Buffer(ST_Buffer(...)) )
Would be good to settle on that in the following days so that it can go into
GDAL 2.0 final.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com

Pepijn Van Eeckhoudt

unread,
May 25, 2015, 2:17:12 PM5/25/15
to Brad Hards, spatiali...@googlegroups.com, even.r...@spatialys.com, Jukka Rahkonen
I agree with everything you and Sandro said Brad. What libgpkg does to make things work automagically is detect the type of database based on the metadata tables, kind of like the spatialite variant detection. I think I posted an explanation of the technical details already so I won't bore you with a repeat of that. In a nutshell you detect the database type in the sqlite extension init method and store that information in the user data pointer associated with each function. Every geometry consuming/producing function can then use that info to automatically do the right thing. There's no need for explicit sql level casts that way.

The only moment when you need to be explicit is when you're creating a new database. I solved that with multiple extension entry points. The default one is auto detection with fallback to gpkg for new databases. The other ones choose explicitly for spl2, spl3, spl4 or gpkg and produce an error if used with an existing database and the type is incorrect.

I agree entirely with Sandro that allowing the different encodings to be mixed and matched is a support nightmare waiting to happen. The only useful use case for this I can imagine is when you want to open a spl db, attach a gpkg db and then select from one to insert into the other. I decided to not support that in my code. People can use ogr2ogr for transcoding purposes if necessary.

Hth,

Pepijn
Reply all
Reply to author
Forward
0 new messages