topology-based simplification

[Jan Vaillant]

Hello Sandro,

I was looking through your tutorials on the new topology features and
tried to reproduce this one:
https://www.gaia-gis.it/fossil/libspatialite/wiki?name=topo-intermediate

I have trouble to achieve the level of simplification/generalization for
the comuni table as shown in the images.

I went through the commands step by step (see below) and then exported
the result as svg (attached). I suppose the sample DB from here
https://www.gaia-gis.it/gaia-sins/toposamples/tuscany-admin.7z has been
updated since you have created the tutorial. I get different results for
the no of edges etc. You may have achieved the level of simplification
in the 5th Elba image with something higher then a 500m radius. So I
tried to increase the radius further but then some polygons started
disappearing.

Any idea what could have gone wrong in my case?

Thank you,

Jan


SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> select load_extension('/usr/local/lib/mod_spatialite');

sqlite> select initspatialmetadatafull(1);
1
sqlite> SELECT CreateTopology('topo', 3003, 0, 0);
1
sqlite> ATTACH DATABASE "./tuscany-admin.sqlite" AS admin;
sqlite> SELECT TopoGeo_FromGeoTable('topo', 'admin', 'comuni', NULL,
NULL, 512);
1
sqlite> DETACH DATABASE admin;
sqlite> SELECT Count(*) FROM topo_node;
27235
sqlite> SELECT Count(*) FROM topo_edge;
27518
sqlite> SELECT Count(*) FROM topo_face;
1328
sqlite> SELECT ST_ValidateTopoGeo('topo');

sqlite> SELECT * FROM TEMP.topo_validate_topogeo;
sqlite> SELECT TopoGeo_UpdateSeeds('topo');
1
sqlite> ATTACH DATABASE "./tuscany-admin.sqlite" AS admin;
sqlite> SELECT TopoGeo_ToGeoTableGeneralize('topo', 'admin', 'comuni',
NULL, 'out_comuni_500m', 500);
1
sqlite> SELECT TopoGeo_ToGeoTableGeneralize('topo', 'admin', 'comuni',
NULL, 'out_comuni_5000m', 5000);
1
sqlite> .output ./out_comuni_500m
sqlite> select '<path d="' ||  assvg(geom, 0, 1) || '"/>' from
out_comuni_500m where geom not null;
sqlite> .output ./out_comuni_5000m
sqlite> select '<path d="' ||  assvg(geom, 0, 1) || '"/>' from
out_comuni_5000m where geom not null;

[Jan Vaillant]

Hi,

finally I figured it out. The problem was this command in the tutorial:

On 8/7/21 5:01 PM, Jan Vaillant wrote:
> sqlite> SELECT TopoGeo_FromGeoTable('topo', 'admin', 'comuni', NULL,
> NULL, 512);

This command gave me the good result:

SELECT TopoGeo_FromGeoTable('topo', 'admin', 'comuni', NULL, 512);

Apparently the 6th argument (512) is not the line_max_points as stated
in the tutorial but line_max_length.

[Andrea Peri]

Afaik. It should split the line when it reach that values of vertexes.

If you have a line with 400 vertex, it is loaded as is.

But if the lines has 1000 vertexes.

It load the lines as two lines.

The first with 511 vertexes and the second with 489 vertexes.

A 

One with 

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spatialite-users/f3c833fc-4f69-1820-2a2c-85c6eae76137%40gmx.de.

[a.fu...@lqt.it]

Sorry Jan,

I hadn't yet the time to examine your post (I'm rather busy during
these days).

yes, you are absolutely right: the Wiki page was written during
the very first days of Topology, but during the final steps
there was a substantial refactoring causing several changes
in the signatures of few SQL functions.
unhappily not always the documentation has been updated.

hint: please always carefully check the examples by
verifying the actual updated signatures of SQL functions
that you'll find here:

http://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.1.html

bye Sandro

[Jan Vaillant]

Bonjour list,

On 8/13/21 10:35 AM, a.fu...@lqt.it wrote:
> Sorry Jan,
>
> I hadn't yet the time to examine your post (I'm rather busy during
> these days).

no problem. I am happy that I finally found the "issue" myself.

That the "512" was not the vertex number but max length explains well
why I could not find any difference

in the polygons when I went from 500m to something larger than 500m for
the generalization. When I inspected the nodes in QGIS

they where identical and the level of simplification remained unchanged.
Still, there is one thing that I find puzzling: Why did some polygons
disappear while most of them remained unchanged?


Thank you,

Jan

[a.fu...@lqt.it]

On Fri, 13 Aug 2021 13:03:30 +0200, Jan Vaillant wrote:
> Still, there is one thing that I find puzzling: Why did some polygons
> disappear while most of them remained unchanged?
>

Hi Jan,

your first query used the "wrong" arguments (sorry, my fault;
the error was in the not updated Wiki page)

SELECT TopoGeo_FromGeoTable('topo', 'admin', 'comuni', NULL, NULL,
512);

this actually means that 512 will be interpreted as the tolerance
radius (that expressed in meters is a definitely excessive value)

when computing generalization SpatiaLite always applies an internal
rule: all polygons will a very neglibile area will be suppressed.
the rule for identifying such microscopic polygons is:

polygon_area < tolerance * tolerance

short conclusion: all polygons presenting an area of less than
512x512 square meters were suppressed (as you actually
noticed). Mystery solved.

bye Sandro

[Jan Vaillant]

Hello Sandro,

On 8/13/21 8:38 PM, a.fu...@lqt.it wrote:
> short conclusion: all polygons presenting an area of less than
> 512x512 square meters were suppressed (as you actually
> noticed). Mystery solved.

I understand that holes & islands are suppressed according to such a
rule. But wouldn't it make some sense if at least every polygon that
shares a border with another polygon would remain? Then the
generalization would preserve the topology as much as possible.

I took a look at postgis: The simplify function has a
"preserveCollapsed" option. ST_SimplifyPreserveTopology does not have
the option - I suppose because it never collapses a polygon. I
understand this is a design/api choice. Maybe you could consider adding
a user option to avoid suppressing bordering polygons (and/or holes &
islands)?

The topological generalization is really a very nice new feature of
spatialite. But as far as I understand it would require currently to
first compute the area of all polygons to pick a radius that would not
remove (bordering) polygons.


Bye,

Jan

[Andrea Peri]

The preservetopology function of postgis os not true topology function.

Is a kind of preserve shape.

Pay attention theyxare not polygon but faces. As lines are edges.

A.

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/spatialite-users/d83a28d0-6555-9b3b-f12c-0474c882c715%40gmx.de.

[Jan Vaillant]

Hi,

On 8/14/21 8:14 AM, Andrea Peri wrote:
> The preservetopology function of postgis os not true topology function.
> Is a kind of preserve shape.

Yes, right, I know. Though there is also a ST_Simplify(TopoGeometry tg)
- I admit,  I do not know how this functions treats collapsing. I just
wanted to point out that there is a sort of succession from Simplify to
SimplifyPreserveTopology to TopoGeo_ToGeoTableGeneralize improving the
"correctness" of the simplification.

Jan

[a.fu...@lqt.it]

On Sat, 14 Aug 2021 08:40:27 +0200, Jan Vaillant wrote:
> Hi,
>

> On 8/14/21 8:14 AM, Andrea Peri wrote:
>> The preservetopology function of postgis os not true topology
>> function.
>> Is a kind of preserve shape.
>

> On Sat, 14 Aug 2021 07:56:28 +0200, Jan Vaillant wrote:
> Yes, right, I know. Though there is also a ST_Simplify(TopoGeometry
> tg)
> - I admit,  I do not know how this functions treats collapsing. I
> just
> wanted to point out that there is a sort of succession from Simplify
> to
> SimplifyPreserveTopology to TopoGeo_ToGeoTableGeneralize improving
> the
> "correctness" of the simplification.
>

it's not too difficult to be explained.

- non topological functions: ST-Simplify() and
ST_SimplifyPreserveTopology()
they are supported by both PostGIS and SpatiaLite, and they are
exactly
the same, because in both cases all the hard work is delegated to the
GEOS library that implements the Douglas–Peucker algorithm [1]
note: the only difference of the PreserveTopology flavor is that
polygons will never collapse (i.e. they'll never become Points or
Linestrings: at least a triangle will be preserved even in the
case of an extreme simplification)

[1]
https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm

- the topological approach: TopoGeo_ToGeoTableGeneralize()
this too is based on the Douglas–Peucker support of GEOS, but
in this case only the EDGES of the Topology will be
simplified. FACES by definition are delimited by EDGES, so
the PreserveTopology variant is not required because it's
always implicit.

why removing microscopic faces (aka polygons) is a good idea.
Generalizing is almost always an useful trik for deriving
another "light-weight" layer much more fast to be painted
on the sceen or printed on paper.

too many vertices are real performance killer, and all too
short segments will simply be rendered as single pixels
during rasterization.
in other words: they'll be absolutely invisible on the
graphic map, but they'll require a substantial time to
be processed.

final note: you noticed many disappearing polygons
after generalizing only because you had cretated your
topology with an absurd tolerance value of 512 meters.
usually tolerance is expected to be in the centimeters
or millimeters range, and possibly even less.
if you had set a more reasonable tolerance your eye
would be absolutely unable to notice any "missing
polygon", except may be for some microscopic
fly poop.

bye Sandro

[Andrea Peri]

No,

There isnt any succession.

They are two totally differentes lineages.

The simplifypreservetopology (afaik) come from geos that Is a porting of java jts library.

Instead the topogeo_* functipns are an evolutions of the porting on spatialite of the postgis topology suite. The postgis topology suite born from the ISO19xxx (i dont remember number) topology specs.

The iso describe a true topology universe where there is faces and edges and nodes. And there are rules to manage add/delete/modify of them.

The geos/jts is instead a library to manage simplefeature geometry.

Please note that a face cannot ignore where it is in its universe and so where thereos a face cannot exist another face.

Instead in simplefeature world could exist.without problem many polygons that are averlap or partially overlapped.

The simplifypreservetopology try to maintain the rules.between polygon on a chaotic simplefeature universe.

Instead in a topplogy universe the rules between faces Is implicit and so there os no need for a preservetopology function.

:)

A.

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/spatialite-users/2f8b6c7a-017f-eac6-c837-fe4ce8cb35ac%40gmx.de.

[a.fu...@lqt.it]

On Sat, 14 Aug 2021 09:30:25 +0200, Andrea Peri wrote:
> The postgis topology suite
> born from the ISO19xxx (i dont remember number) topology specs.
>

just for the precision, the complete reference is

ISO/IEC 13249-3
(Information technology -- Database languages
-- SQL multimedia and application packages
-- Part 3: Spatial aka SQL/MM)

bye Sando

[Andrea Peri]

Hi sandro.

Just to better understand.

Your TopoGeo_ToGeoTableGeneralize()

Apply only to every esges taking them one by one. I understand ?

Thia.is only point of contact between the somplefeature function and the topology versipn: The geometry of every single edge.

I dont know the Douglas peucher algorithm and dont know of it see also at the boundary geometry or not.

But if the geneealization break the actual topology (intersection of near edges as examples) your functipn reassemble.all the topology ?

A.

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/spatialite-users/703dbaafbd316ce642d000e3f58f7e63%40lqt.it.

[a.fu...@lqt.it]

On Sat, 14 Aug 2021 09:40:06 +0200, Andrea Peri wrote:
> Hi sandro.
> Just to better understand.
>
> Your TopoGeo_ToGeoTableGeneralize()
> Apply only to every esges taking them one by one. I understand ?
>

yes, it's correct.

> Thia.is only point of contact between the somplefeature function and
> the topology versipn: The geometry of every single edge.
>

yes again

> I dont know the Douglas peucher algorithm and dont know of it see
> also
> at the boundary geometry or not.
>

the algorithm is basically simple: you'll find a very clean explanation
on Wikipedia with many figures:

IT: https://it.wikipedia.org/wiki/Algoritmo_Ramer-Douglas-Peucker
EN:
https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm

> But if the geneealization break the actual topology (intersection of
> near edges as examples) your functipn reassemble.all the topology ?
>

after generalizing all Edges any Face will be exported into the
destination table as a Polygon.
Nodes will never be affected by the simplification process,
so in the most extreme case an Edge could become a straight
segment directly connecting its Nodes, and topological
correctness will be preserved in any case.

bye Sandro

[Jan Vaillant]

Friends,

really, I have no doubt that your algorithmic insight is far superior to
mine.

I am just arguing on behalf of the friendly people of Marciana Marina
(quote: "one of the most important towns of Elba Island") and any other
tribe that is neglected existence and wiped of the map by a cruel
algorithm. I was hoping they could just happily live on in their tiny
triangle - instead of being nullified - regardless of the tolerance
applied by a heartless GIS users.

Bye,

Jan

[a.fu...@lqt.it]

Hi Jan,

you still continue to miss the question.

why and when we'll use a simplified/generalized dataset ?

surely not for data processing or data analysis, because only
a fool man will voluntary use a degraded dataset when he can
use the full quality one.

using simplified/generalized datasets make sense just in a
single case that is visual rendering (i.e. printing a map).

using the full resolution layer surely is the optimal choice
at low representation scales, but will become terrible at
higher scales because it will become deadly slow.
as higher the scale, as slower the graphic rendering.

a mildly generalized alternative layer can ensure an
acceptable rendering time at middle scales; a strongly
generalized layer can widely help at higher scales.
there is no good or bad generalization level; it fully
depends on the intended representation scale.

speaking in a general way simplification/generalization
is a form of destructive data compression; many small
details will be surely suppressed during the process, but
the result will be a derived layer.lighter (and faster).
the final result will be the same to the human eye,
because small details will become invisible.
conclusion: we can safely trade quality for speed.

complaining because very radical simplifications will
suppress too many details is self contradictory; it
simply means that a too much aggressive compression
level has been applied.

it's exactly the same kind of problem that we usually
have to face when using rasters compressed as JPEG;
the compression level must be carefully adjusted
depending on the intended usage.
a very aggressive compression could be optimal for
small size thumbnails but could be awfull for
full resolution images. Once again, it fully
dependes on the intended representation scale.

bye Sandro

[Jan Vaillant]

Salut Sandro & list,

On 8/16/21 9:29 AM, a.fu...@lqt.it wrote:
> On Sat, 14 Aug 2021 19:31:02 +0200, Jan Vaillant wrote:
>> Friends,
>>
>> really, I have no doubt that your algorithmic insight is far superior to
>> mine.
>>
>> I am just arguing on behalf of the friendly people of Marciana Marina
>> (quote: "one of the most important towns of Elba Island") and any other
>> tribe that is neglected existence and wiped of the map by a cruel
>> algorithm. I was hoping they could just happily live on in their tiny
>> triangle - instead of being nullified - regardless of the tolerance
>> applied by a heartless GIS users.
>>
>
> Hi Jan,
>
> you still continue to miss the question.
>
> why and when we'll use a simplified/generalized dataset ?
>
> surely not for data processing or data analysis, because only
> a fool man will voluntary use a degraded dataset when he can
> use the full quality one.
>

Thank you for the thorough  - as always - explanation.

I admit, I am a fool - and my question is indeed quite 'foolish'. It is
not about some theoretical, mathematical concept of topological
somethingness.

I am just trying to understand the following behavior that I find a bit
inconsistent given that there are three functions/ways to
"simplify/generalize":

SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.

Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.

sqlite> select load_extension('/usr/local/lib/mod_spatialite');

sqlite> select initspatialmetadatafull(1);
1

sqlite> create table polys (id integer primary key autoincrement);
sqlite> select AddGeometryColumn('polys', 'geom', 3003, 'POLYGON');
1
sqlite> insert into polys values (null, geomfromtext('polygon((0 0, 0
10, 10 0, 0 0))', 3003));
sqlite> insert into polys values (null, geomfromtext('polygon((10 0, 0
10, 10 10, 10 0))', 3003));
sqlite> select aswkt(simplify(geom, 5)) from polys;
POLYGON((0 0,0 10,10 0,0 0))
POLYGON((10 0,0 10,10 10,10 0))
sqlite> select aswkt(simplify(geom, 10)) from polys;
NULL
NULL
sqlite> select aswkt(simplifypreservetopology(geom, 10)) from polys;
POLYGON((0 0,0 10,10 0,0 0))
POLYGON((10 0,0 10,10 10,10 0))

sqlite> SELECT CreateTopology('topo', 3003, 0, 0);
1

sqlite> SELECT TopoGeo_FromGeoTable('topo', 'main', 'polys', NULL);
1
sqlite> SELECT TopoGeo_ToGeoTableGeneralize('topo', 'main', 'polys',
NULL, 'polys_5', 5);
1
sqlite> SELECT TopoGeo_ToGeoTableGeneralize('topo', 'main', 'polys',
NULL, 'polys_10', 10);
1
sqlite> select aswkt(geom) from polys_5;
MULTIPOLYGON(((0 0,0 10,10 0,0 0)))
MULTIPOLYGON(((0 10,10 10,10 0,0 10)))
sqlite> select aswkt(geom) from polys_10;
NULL
NULL

Observations:

1. simplify collapses the geom if the tolerance is >= 10m

2. simplifypreservetopology leaves them untouched if the tolerance is >=
10m (I guess, because it is supposed to preserve the topology - as far
as I understand)

3. TopoGeo_ToGeoTableGeneralize collapses the geom if the tolerance is
>= 10m


If the behavior of "TopoGeo_ToGeoTableGeneralize" is a choice (I guess
it is) - that's fine! I just argue that it is at least surprising and
maybe considering (<- notice my carefully phrased idea) an option to not
collapse (e.g. for the sake of the brave "Marciana Marinarians") might
be, if possible ... nice, useful.


Bye,

Jan

[a.fu...@lqt.it]

Jan,

the Administrative Boundaries of Tuscan Municipalies are a
very nice example of an high-quality and extremely precise
map; it's very rich of minute details, really useful for
data processing and spatial analysis tasks or for printing
a low scale map.
unhappily, if your goal is printing a middle scale or
even worst an high scale map a such detailed layer will
become a real performance bottleneck.
but in this case we can apply some kindk of generalization.

let's examine the results of simplifying using a distance
radius of respetively 5 and 10 metres.

original data: the Municipality of Marciana Marina is a
MultiPolygon with 28 individual polygons:
- the main polygon has 3.793 vertices, and obviously
is the main inhabited area.
- all the other polygons have about ten vertices and
a very small area ? what are they intended to represent ?
Marciana Marina is on a rocky coast, and this is a very
detailed layer. each one of this small polygons simply is
an individual rock or cliff emerging from the sea near
the main coast.

simplify 5m: now only 4 individual polygons have been preserved
- the main polygon (just 453 vertices after simplification)
- and the 3 bigger rocks or cliff

simplify 10m: only 2 individual polygons now survive:
- the main polyhon (only 251 vertices)
- and the bigger of all cliffs

conclusion: the friendly people of Marciana Marina
can safely continue to happily live in their not
so tiny mainland :-D
all the suppressed polygons simply are small rocks
into the sea, probably smaller than my own bedroom.
so small that I had to zoom a lot before being able
to visually identify them on the screen using the
full resolution layer.

you are free to believe that this is a capital sin,
but for me the discussion stops here.

bye Sandro

[Jan Vaillant]

Salut,

On 8/16/21 3:30 PM, a.fu...@lqt.it wrote:
> conclusion: the friendly people of Marciana Marina
> can safely continue to happily live in their not
> so tiny mainland :-D
> all the suppressed polygons simply are small rocks
> into the sea, probably smaller than my own bedroom.
> so small that I had to zoom a lot before being able
> to visually identify them on the screen using the
> full resolution layer.
>
> you are free to believe that this is a capital sin,
> but for me the discussion stops here.

No doubt, this is a very useful and nice feature of spatialite - and in
case I did not mention it: I really do appreciate your work on the library.

My conclusion is that in GIS - it seems - there is not a clear,
unambiguous concept of what "preserving a topology" actually means. I'll
take a look at GRASS and maybe also run the example through postgis to
get more insight in how it is handled in various GIS implementations.

Thank you,

Jan

[Andrea Peri]

Instead is clear if one have the patience to read the documentation.

The topolgy is a propriety of an mathematical environment.

So declaring that a single polygon is topologycally correct is a non-sense because the topology is the relation with every one polygon in the universe (universe is a well defined concept in GIS world).

So simply a function che is called ***topology*** but not act in a topological universe or which not see what happened at boundaries polygon is not a real topology function.

But only a function called **topology***

Obviously the topology is not a unique concept, but there are many possible definition.

In grass there is a concept of topolgy. In arcinfo ($25.000) there is another concept of topolgy and ISO try to define a standard of topology definition, but is however another definition of topology.

As example:

Oracle in its spatial module adopt the ISO topology so is near to the concept of topology as reported from ISO.

Postgis is supporting the ISO and also spatialite is supporting the ISO.

Grass use an own definition of topology and also arcinfo use own definition AFAIK.

A.

--
You received this message because you are subscribed to the Google Groups "SpatiaLite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spatialite-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/spatialite-users/8fd5325e-5703-8d77-6de9-1222a6b49c4f%40gmx.de.

--

-----------------
Andrea Peri
. . . . . . . . .
qwerty àèìòù
-----------------