Starting with topologies

[ckgoo...@gmail.com]

Hi, I’ve been using Spatialite for a while now to implement my atlas application for visually impaired users.  I’m very happy with what I’ve done so far but I’ve come across the classic problem when using standard SFS geometries of gaps appearing between my polygons when I simplify my geometries to improve performance at low zoom levels.

And so I’ve just started reading about topologies.  This is a whole new area of Spatialite for me and it looks like quite a shift in my thinking and my database design though what I’ve read makes sense so far.

Can I just check the best documentation to work through is at:

https://www.gaia-gis.it/fossil/libspatialite/wiki?name=ISO+Topology

I have made the mistake in the past of reading outdated Spatialite documentation so thought it worth checking before I go too deep into this topic.

Best, Chris

[a.fu...@lqt.it]

On Wed, 13 Aug 2025 21:25:10 +0100, ckgoo...@gmail.com wrote:
> Hi, I've been using Spatialite for a while now to implement my atlas
> application for visually impaired users. I'm very happy with what
> I've
> done so far but I've come across the classic problem when using
> standard SFS geometries of gaps appearing between my polygons when I
> simplify my geometries to improve performance at low zoom levels.
>
> And so I've just started reading about topologies. This is a whole
> new
> area of Spatialite for me and it looks like quite a shift in my
> thinking and my database design though what I've read makes sense so
> far.
>
> Can I just check the best documentation to work through is at:
>
> https://www.gaia-gis.it/fossil/libspatialite/wiki?name=ISO+Topology

> [1]

>
> I have made the mistake in the past of reading outdated Spatialite
> documentation so thought it worth checking before I go too deep into
> this topic.
>

Hi Chris,

I confirm that the URL you cited is the only documentation
available on the SpatiaLite Topology; as far as I know,
nothing else exists.

I must warn you, however, that you are venturing in a very
dangerous direction.

First point: processing a non-trivial Topology is a complex
operation requiring a lot of time; processing times are
usually measured not in seconds or minutes, but in hours
or days (literally).

Second point: Topology requires absolute precision in numerical
calculations, but unfortunately Floating Point numbers
intrinsically only have a finite precision.
To try to get around this conceptual obstacle, Topology
adopts a tolerance criterion that often helps but not always.
In a non-trivial Topology it is inevitable that situations
of irreconcilable conflict will arise.
Simply put; most geometries will fit correctly into the
Topology, but you must be prepared to accept that a small
minority will be discarded due to insurmountable exceptions.
Resolving all these exceptions is not impossible, but it
requires a patient and careful series of corrective actions.
Which is something that requires a lot of time as well as
excellent expertise.

-----------------------------------------------

Just to help you understand better.
Topology is not simply an alternative model to SFS for
representing Geometries.
Instead, it is something very heavy and complex that is
used by professionals to produce high-quality digital maps
that will then be used for practical purposes in the form
of SFS datasets.

In other words: Topology is something that makes sense to
use in the context of some Public Administration (central
or local Govt.) called upon to publish digital maps.
In this context, speed is not a requirement, because publishing
a good regional or national map takes months, if not years.
However, ensuring excellent data quality is absolutely essential.

Topology is not something that can be used in a real-time context,
it is rather a typical support tool for back office activities.
Possibly within a well-structured organization capable of adequately
supporting this type of activities.

If you hope to use Topology to improve the performance of your app,
carefully consider that this would imply a decidedly intolerable
slowness.

bye Sandro

[ckgoo...@gmail.com]

Hi Sandro,
Thanks, as always, for your advice.
I'll read the documentation with your points in mind. It sounds like an interesting topic to find out about. It may well be over the top for what I need. I will need to solve the problem of gaps between my polygons somehow though but I have thoughts on how I can do this without topologies.
Best, Chris

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/spatialite-users/754640e6388a7ef30ce53b3c22d448b0%40lqt.it.

[a.fu...@lqt.it]

On Thu, 14 Aug 2025 16:35:11 +0100, ckgoo...@gmail.com wrote:
> I will need to solve the problem of gaps between my polygons
> somehow though but I have thoughts on how I can do this
> without topologies.
>

Hi Chris,

if you are noticing too many gaps between polygons, this
surely means that you are using very poor quality data.

If I remember correctly, you are using the national borders
derived from OSM, which are always expected to be topologically
correct.
If this doesn't happen, it's almost certainly because there's
some phase in your process chain that catastrophically lowers
the quality of the data, irreversibly destroying the initial
topological coherence.
The most obvious suspicion is that you are applying some very
aggressiva generalization/simplification of polygons so to
reduce the size of the required storage.

There's nothing wrong with this; it's something that's
used frequently, but obviously, this way you'll end up with
a dataset that can only be viewed at the largest scales
(continents) but will be completely unusable at the most
detailed scales (single country, region, province etc).
In other words: you have suppressed so much information that
what remains is insufficient to show detailed maps with an
acceptable level of accuracy.

I easily understand that a worldwide coverage of high-resolution
data requires exaggerated amounts of storage that are difficult
if not impossible to manage, especially on portable devices.
You're struggling with two irreconcilable needs: using minimal
storage while still maintaining good data quality.
Two conflicting needs that simply can't exist together.

Here are some possible solutions to solve the problem.

1) Adopt the most canonical solution.
Geographic data are never stored on the client; they are hosted
on the server, where there are no storage limitations.
The two WEB protocols WMS and WFS were invented specifically to
discipline the exchange of geographical information between client
and server.
https://en.wikipedia.org/wiki/Web_Map_Service
https://en.wikipedia.org/wiki/Web_Feature_Service
I assume, however, that this is too complicated a solution for
your purposes which seem quite limited; you are probably looking
for something much simpler.

2) Partition and slice data into multiple levels.
I understand that what you've already achieved is adequate for
continental scales, but to move to higher levels of detail, however,
further information is needed.
The simplest solution might be to add a second full-resolution level
(not generalized/simplified) to be used at more detailed scales.
To keep storage demand within reasonable limits, this second level
will not have global coverage, but will instead be broken down into
discrete packages (single country, region, province).
It's virtually impossible for a user to be in multiple parts of the
world at the same time, so it would be sufficient to upload only the
data needed for the area in which the user is currently located;
detailed data will be then deleted when no longer needed to free
up space.
In short, a sort of dynamic plug-in system, simple and easy to
manage
for the usert that should never require the download of excessive
amounts of data.
To better understand, take a look at how GeoFabrik organizes its
OSM data packages offered for downloading:
https://download.geofabrik.de/

3) Change data provider.
OSM datasets are certainly valid, but they are also very large.
Fortunately, there is an alternative provider of worldwide
geographic
open data:
https://www.naturalearthdata.com/downloads/
Natural Earth datasets are smaller in size, are immediately ready
for
use without any need for preliminary cleanup, and are topologically
correct.
But above all they are offered with three different levels of
generalization and size:
* Large Scale
* Medium Scale
* Small Scale
Evaluate them carefull and without prejudicey, because IMHO they
could be the optimal solution for your needs.

bye Sandro

[ckgoo...@gmail.com]

Hi Sandro,
I know exactly why I'm getting the gaps. It is as you said, I'm simplifying my data.
I have taken the OSM country level data and intersected it with the land Shapefiles which are derived from the OSM coastline data to remove the marine territorial waters from the country polygons. This has worked well and I end up with a mere 766,000 polygons which are all valid geometries representing every country and island on the planet. Or near enough. This takes up about 1.5Gb of disk space. And actually, this is fine.

The problem comes when I start drawing the vector map. At the OSM full resolution it is far too slow. So I have to simplify the polygons.
I actually have 4 different levels of detail - the full resolution and then 3 increasingly simplified versions. The most simplified version used 5000m as the tolerance in 6933. My map now renders in a perfectly acceptable time on a range of zoom levels.

When I'm zoomed in and using the full res version of the map there are no gaps. It is the base data. But zooming out shows gaps between the polygons.

My problem is that I have a polygon for each country and the simplifications are different on two neighbouring polygons.

Hence my interest in topology which stores the data as shared edges with fixed nodes. I can simplify the edges and both neighbouring countries will undergo the same simplification and there will be no gaps.

But I realise now that applying a topology is more a data validation tool than something I need to implement. I can still though remodel my data to use OSM ways stored directly and storing how to construct the polygon from a list of ways. I can apply different levels of simplification to the ways and I should get no gaps.

The problem is I still need to trim off the marine territorial waters which I currently do using whole polygon geometries - not collections of ways / line geometries. So I'm currently looking at the feasibility of downloading all the coastline ways constructing my own continental and island polygons from them, and doing the intersection trick with that data. But I'll have to do my own intersection algorithm so I can maintain the ways. I can't use polygon geometries.

It is quite an appalling prospect but also quite an interesting challenge.

Maybe there is an easier way. If there is, I don't know what it is at the moment.

As for switching to other map data. Well, yes, possibly. What I like about OSM though is the detail and what I've enjoyed doing is loading in different types of data and having the option to do this e.g. Cities, towns, villages, hamlets, districts, castles, lakes, rivers, bins. And then there is all the network data for road, rail, tracks and so on.

So I know the problem. I have some idea of how to solve it. It is just tricky.

Best, Chris


-----Original Message-----
From: spatiali...@googlegroups.com <spatiali...@googlegroups.com> On Behalf Of a.fu...@lqt.it

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/spatialite-users/fa4ad6069232e087982c914430241440%40lqt.it.

[a.fu...@lqt.it]

On Fri, 15 Aug 2025 20:24:05 +0100, ckgoo...@gmail.com wrote:
> Hi Sandro,
> I know exactly why I'm getting the gaps. It is as you said, I'm
> simplifying my data.
> I have taken the OSM country level data and intersected it with the
> land Shapefiles which are derived from the OSM coastline data to
> remove the marine territorial waters from the country polygons. This
> has worked well and I end up with a mere 766,000 polygons which are
> all valid geometries representing every country and island on the
> planet. Or near enough. This takes up about 1.5Gb of disk space.
> And actually, this is fine.
>

> Hence my interest in topology which stores the data as shared edges
> with fixed nodes. I can simplify the edges and both neighbouring
> countries will undergo the same simplification and there will be no
> gaps.
>
> But I realise now that applying a topology is more a data validation
> tool than something I need to implement. I can still though remodel
> my data to use OSM ways stored directly and storing how to construct
> the polygon from a list of ways. I can apply different levels of
> simplification to the ways and I should get no gaps.
>

Hi Chris,

These clarifications of yours change the whole scenario.

The OSM data model is already topological in itself, even if it does
not comply with the ISO standard, but it is still possible to
highlight some equivalences.

1. The concept of NODE is common in both models.
2. What OSM defines as WAYs are equivalent to what ISO calls EDGEs
3. In both models, polygonal entities are never directly defined
as such and instead result from the assembly of the various
sections of their external boundary that are common to some
other adjacent polygon.
in essence, OSM AREAs are equivalent to ISO FACEs.

The problem of building and validating a Topology starting from
a set of polygons is decidedly complex, long and not without
pitfalls.
But this happens mainly because a lot of preliminary work must be
done so to identify and normalize all the EDGEs common to two
adjacent polygons.
At this point, we will also have all the NODEs we need, and it will
therefore be possible to identify all the FACEs that make up our
Topology.
A very long process, certainly, keeping fingers crossed in the
hope of encountering only a few irresolvable exceptions.

But your case is completely different, because you can start
not from polygons but from OSM WAYs, which are already ready
to be used as EDGEs without requiring complex preparatory
operations.
It's an absolutely fundamental difference that changes the game.

You don't have to waste time studying the most sophisticated methods
for building a Topology like TopoGeo_FromGeoTable() and friends which
are the most common way to build a Topology.

You just need two of the most basic functions:
A) ST_AddIsoNetNode() for inserting NODEs
B) ST_AddIsoEdge() for inserting EDGEs.
C) You don't have to do anything at all to insert the FACEs, because
the Topology itself will automatically create them as soon as the
insertion of some EDGE forms a closed boundary that identifies an
area.

The transformation of the Topology into a set of polygons will only
take
place at the very end of the process, and naturally you will be free to
apply any generalization/simplification criterion (even very
aggressive
ones) with the certainty of always maintaining perfect topological
coherence because you will be working on the EDGEs that are upstream
of the polygons.

I still recommend that you build the polygons beforehand, because
materializing the FACEs starting from their EDGEs is a slow process,
a real performance killer.
Basically you will use Topology in back office mode to prepare the
polygons that will be displayed in real time.
But applying all the above suggestions it should turn out to be
something quite simple and not particularly demanding, much simpler
and faster than the more usual approach.

One last piece of advice: perhaps before venturing onto the global
scale, it might be wise to do some preliminary tests on a smaller
scale such as the Western Europe, South America or Africa.

bye Sandro

[ckgoo...@gmail.com]

Hi Sandro,
Thank you for the email below about topologies. I've taken a good look at it today.
I ended up using AddISONode and AddEdgeModFace as otherwise I started throwing exceptions. But with manually entered data I have created a very simple network made up of 3 bordering polygons which are then overlaid with coastline data.
I was then able to use Polygonize to create a polygon geometry for each face created.
Another step I've done manually is to determine if each polygon is clockwise or counterclockwise using the Shoelace formula but I have not yet done this programmatically. But it works.
If I do it for real I will also have to maintain a mapping from OSM way ID's to topology edge ID's.
I have a question though. Either I might be doing it wrong, but I seem permitted to create a polygon inside another polygon. I was hoping I would be able to create a counterclockwise hole in a polygon and fill this hole with a clockwise polygon. But I throw exceptions for overlapping edges.
I want to do this to show, Italy for example, has two holes in it for San Marino and Vatican City. And also then have polygons for these two countries.
Any thoughts on this?

Best, Chris
-----Original Message-----
From: spatiali...@googlegroups.com <spatiali...@googlegroups.com> On Behalf Of a.fu...@lqt.it
Sent: 16 August 2025 16:47
To: spatiali...@googlegroups.com
Subject: RE: [SpatiaLite-Users] Starting with topologies

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/spatialite-users/2f892e97b915b9b3e60f716ea8feda35%40lqt.it.

[a.fu...@lqt.it]

On Fri, 22 Aug 2025 17:32:11 +0100, ckgoo...@gmail.com wrote:
> Hi Sandro,

> Thank you for the email below about topologies. I've taken a good
> look at it today.
> I ended up using AddISONode and AddEdgeModFace as otherwise I started
> throwing exceptions. But with manually entered data I have created a
> very simple network made up of 3 bordering polygons which are then
> overlaid with coastline data.
> I was then able to use Polygonize to create a polygon geometry for
> each face created.
> Another step I've done manually is to determine if each polygon is
> clockwise or counterclockwise using the Shoelace formula but I have
> not yet done this programmatically. But it works.
> If I do it for real I will also have to maintain a mapping from OSM
> way ID's to topology edge ID's.
> I have a question though. Either I might be doing it wrong, but I
> seem permitted to create a polygon inside another polygon. I was
> hoping I would be able to create a counterclockwise hole in a polygon
> and fill this hole with a clockwise polygon. But I throw exceptions
> for overlapping edges.
> I want to do this to show, Italy for example, has two holes in it for
> San Marino and Vatican City. And also then have polygons for these
> two countries.
> Any thoughts on this?
>

Hi Chris,

Why are you so obsessed with the polygonal direction?
It's an archaic concept that smells as musty as mammoths and dinosaurs.

There is nothing in the OGC specification that says that the direction
of a polygon matters, it is an absolutely irrelevant detail of no
importance whatsoever.

I presume you're still influenced by the outdated ShapeFiles data
model,
where the direction of Polygon Rings was crucial to determining whether
they were External or Internal Rings.
We were still in the 80s, practically in the Stone Age of Spatial Data;
fortunately a lot of water has passed under the bridge, and today we
have
a much better conceptual model for Geometries, the OGC one.

SpatiaLite offer some functions to identify the direction of the Rings,
and possibly to reverse it, but they are never used except to export
the Geometries in the ShapeFile format, which is however a detail that
is managed internally in a completely automatic way.
please see:
ST_IsPolygonCW()
ST_IsPolygonCCW()
ST_ForcePolygonCW()
ST_ForcePolygonCCW()



Don't worry about San Marino and the Vatican; there's nothing special
about them. they're perfectly normal OGC polygons.
Let SpatiaLite (and especially the OGC model behind it) work, and you
will see that everything will automatically fall into place in the
most painless way.

bye Sandro

[ckgoo...@gmail.com]

Hi Sandro,
I am not a professional GIS user or developer so I only go on what I read. But good to know polygon direction is not important.
Having holes in Italy is useful though as otherwise, when I query what polygons I am in when my user is at St Peter's Square will return two results. Italy and Vatican City where really would prefer just the one. Though it isn't hard to have a rule to defer to the smallest area polygon.

Best, Chris

-----Original Message-----
From: spatiali...@googlegroups.com <spatiali...@googlegroups.com> On Behalf Of a.fu...@lqt.it
Sent: 22 August 2025 18:18
To: spatiali...@googlegroups.com
Subject: RE: [SpatiaLite-Users] Starting with topologies

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/spatialite-users/75a7328fc73852044ef9c8c8b2dbfd1e%40lqt.it.

[a.fu...@lqt.it]

On Fri, 22 Aug 2025 19:06:42 +0100, ckgoo...@gmail.com wrote:
> Hi Sandro,

> I am not a professional GIS user or developer so I only go on what I
> read. But good to know polygon direction is not important.
> Having holes in Italy is useful though as otherwise, when I query
> what polygons I am in when my user is at St Peter's Square will
> return two results. Italy and Vatican City where really would prefer
> just the one. Though it isn't hard to have a rule to defer to the
> smallest area polygon.
>

In the OGC model, direction is not important, nor is size.
Each polygon must be bounded by an Exterior Ring; but a polygon
(it is not mandatory, it is completely optional) can also
declare one or more Internal Rings that allow to identify
any "holes" that may be present.

These are exclusion areas, so no spatial function will
ever tell you that St. Peter's Square is located in
Italy, if the boundaries have been drawn correctly
(including all the holes).

If this is not the case at the moment, it means that
your current geometries are not topologically consistent.
It's a problem that will disappear when you finally will
create your polygons from a verified topology.

bye Sandro