[postgis-users] Clipping a raster and a polygon using ST_Intersection
liglio....@nexxa.com.br
Hi,
I created a retangular raster representing number of lightnings in a square. So I have this matrix raster created using ST_MapAlgebraFct and ST_MakeEmptyRaster.
This raster is not a tiled raster (tb_densitysurface), and when I Clip this raster using a polygon, the query below, the performance is not good.
Maybe because the raster is not tiled. So how do I tile this raster ? And I want the result to be a raster (not gval) to be exported as a file, and then colored using GDAL.
It worked with a different query only using ST_Clip (this returns a rast column and is very fast), but the edges aren´t smooth as using ST_Insersection. I am using Postgis 2.5.
SELECT
(gval).val
(gval).geom
FROM (
SELECT ST_Intersection(
ST_Clip(rast,ST_Envelope(area_country)),
1,
area_country
) As gval
FROM country, tb_densitysurface WHERE ST_Intersects(rast, area_country) and country.gid = 19
) As foo
Regards,
Liglio
Regina Obe
I’m not quite sure what you mean by edges aren’t smooth. Rasters are never really smooth. Vectors are smooth. Rasters may look smooth at the edges, but that’s more because pixels are square and the pixels in each tile line up with the pixels.
If you want it to look a little smoother, maybe expand your envelop ever so slightly so it lines up with a pixel in a raster. So each edge pixel is guaranteed to be fully inside the raster and no edge partly outside the raster or do the operation in raster space which I will explain shortly.
ST_Intersection yes is known to be very slow and as you noticed does not give you a raster back.
Is ST_Clip + ST_Intersects performance sufficient for your needs?
ST_Clip really is ST_Intersection in raster space for raster/geometry. So you could do the intersection in pure raster space by converting your geometry to a raster (making sure to align your new raster with your reference raster). That should make the edges smooth.
Like so – not tested so might have some syntax errors and also I haven’t used this in a very long time so may be wrong about what it returns.
SELECT ST_Intersection(r.rast, r2.rast) AS new_rast
FROM tb_densitysurface AS r, country,
-- create a raster to represent the country that has the same alignment as our density raster (pixels use the same grid)
ST_AsRaster( country.area_country, r.rast, '1BB', 0) AS r2(rast)
WHERE country.gid = 19 AND ST_Intersects(r.rast, r2.rast)
) As foo
As far as tiling goes, yah that would improve the smoothness of the inner tiles but not most outer ones.
You’d also need to ST_Union them all up to get back to a single tile which will add perhaps more overhead than just keeping as single tile.
If you want to tile your raster, you can use this function - https://postgis.net/docs/manual-2.5/RT_ST_Tile.html
Hope that helps,
Regina
liglio....@nexxa.com.br
Regina,
Thanks for your replay.
Your solution is fast like using ST_Clip, but I still have the problem that my pixels are 25km square (low definition) and the image look like this: (image b1.jpg)
And I want to produce something like this: (image b2.jpg)
I know that the second image cannot be reproduced as just raster with big pixels, the pixels cannot be cutted.
There is a way to export this image and associate a value for each square (polygon) to be colored accordingly ?
Regards,
Liglio
Regina Obe
Liglio,
b1 being choppier than b2, is because the rasterized polygon takes on the pixel size of your low res polygon.
I’d try resampling your tb_densitysurface to lower pixel size before you do the operation with ST_Resample.
https://postgis.net/docs/RT_ST_Resample.html
The width and height are measured in pixels. The idea is you want your tb_densitysurface to have more pixels than it does now so that the size of each pixel is smaller.
If you do something like
CREATE TABLE tb_densitysurface_x5 AS
SELECT rid, ST_Resample(rast, ST_Width(rast)*5, ST_Height(rast)*5 ) AS rast
FROM tb_densitysurface;
That should give you the same spatial ref sys, but the pixels will be smaller in size.
You can then use this to rerun the ST_Intersection in raster space.
I think it will be a little slower though, because the more pixels you have the slower things get.
Just keep on increasing that number 5 until you get something that meets your choppiness criteria.
As you get less choppy, operations will get slower, so you need to find a sweet spot there.
Regarding: There is a way to export this image and associate a value for each square (polygon) to be colored accordingly ?
Was that to resolve the above issue, or you just want to know.
Well there is ST_PixelAsPolygons which will convert each pixel to a square polygon. https://postgis.net/docs/RT_ST_PixelAsPolygons.html
There is also ST_DumpAsPolygons https://postgis.net/docs/RT_ST_DumpAsPolygons.html which tries to aggregate nearby pixels with the same value into a polygon.
Both return geomval rows similar to the ST_Intersection of raster, geom. As I recall I think those were completely rewritten in C so should still be faster than ST_Intersection(geometry,raster).
Regina Obe
Just occurred to me that the ST_Intersection I gave you earlier probably does much the same as ST_Clip.
That said, you should be able to achieve a less choppy result using ST_Resample and ST_Clip in unison.
Increase the 5 to as much as you need to get the less choppy effect you are looking for.
SELECT ST_Clip( ST_Resample(rast, ST_Width(rast)*5, ST_Height(rast)*5 ) , c. area_country)
FROM country AS c
INNER JOIN tb_densitysurface AS d
ON ( ST_Intersects(d.rast, c.area_country) )
WHERE c.gid = 19;
From: Regina Obe [mailto:l...@pcorp.us]
Sent: Sunday, December 4, 2022 12:10 PM
To: 'PostGIS Users Discussion' <postgi...@lists.osgeo.org>
Cc: 'liglio....@nexxa.com.br' <liglio....@nexxa.com.br>
Subject: RE: [postgis-users] RES: Clipping a raster and a polygon using ST_Intersection
Liglio,
b1 being choppier than b2, is because the rasterized polygon takes on the pixel size of your low res polygon.
I’d try resampling your tb_densitysurface to lower pixel size before you do the operation with ST_Resample.
https://postgis.net/docs/RT_ST_Resample.html
The width and height are measured in pixels. The idea is you want your tb_densitysurface to have more pixels than it does now so that the size of each pixel is smaller.
If you do something like
CREATE TABLE tb_densitysurface_x5 AS
SELECT rid, ST_Resample(rast, ST_Width(rast)*5, ST_Height(rast)*5 ) AS rast
FROM tb_densitysurface;
That should give you the same spatial ref sys, but the pixels will be smaller in size.
You can then use this to rerun the ST_Intersection in raster space.
I think it will be a little slower though, because the more pixels you have the slower things get.
Just keep on increasing that number 5 until you get something that meets your choppiness criteria.
As you get less choppy, operations will get slower, so you need to find a sweet spot there.
Regarding: There is a way to export this image and associate a value for each square (polygon) to be colored accordingly ?
Was that to resolve the above issue, or you just want to know.
Well there is ST_PixelAsPolygons which will convert each pixel to a square polygon. https://postgis.net/docs/RT_ST_PixelAsPolygons.html
There is also ST_DumpAsPolygons https://postgis.net/docs/RT_ST_DumpAsPolygons.html which tries to aggregate nearby pixels with the same value into a polygon.
Both return geomval rows similar to the ST_Intersection of raster, geom. As I recall I think those were completely rewritten in C so should still be faster than ST_Intersection(geometry,raster).
From: postgis-users [mailto:postgis-us...@lists.osgeo.org] On Behalf Of liglio....@nexxa.com.br
Sent: Wednesday, November 30, 2022 9:37 AM
To: 'PostGIS Users Discussion' <postgi...@lists.osgeo.org>
Regina Obe
> Each pixel is associated with a value in the original raster, and when you do resample, what value is associated to the pixels (in the docs “New pixel values are computed using the NearestNeighbor”).
So in your example x5, only one pixel has the original pixel value and the others 24 pixels has zero or nodata ? If I do colored scale of values it not seems right.
Hmm that is not what I was expecting. Then again I always use the function to reduce resolution, and not for this particular purpose. I had assumed it would apply the value to the next neighboring and keep on applying that way. But sounds like I was wrong about that.
I think I was confusing that with ST_Rescale. Try using ST_Rescale instead and see if that gives expected results.
Note ST_Rescale takes as input pixel sizes. In your case you want to reduce the size of each pixel. The below reduces the pixel size by 20%
https://postgis.net/docs/RT_ST_Rescale.html
SELECT ST_Clip( ST_Rescale(rast, ST_PixelWidth(rast)*0.2, ST_PixelHeight(rast)*0.2 ) , c. area_country)
FROM country AS c
INNER JOIN tb_densitysurface AS d
ON ( ST_Intersects(d.rast, c.area_country) )
WHERE c.gid = 19;
Another question: I do a retangular matrix larger than the country (creating an empty raster and dumping each pixel as polygon, and counting lightining inside each sqare). So I have a lot of squares to compute, and 20 years of lightining (about 2 billions).
It will be more efficient/faster if my empty raster has the shape of my country, so It will have less squares to compute. It is possible ?
Yes it would be more efficient both in space and time for computing.
Just clip your raster by the country boundaries kind of like you were doing.
So something like
CREATE TABLE tb_densitysurface _country_clip AS
SELECT s.rid, ST_Clip(s.rast, geom) AS rast
FROM tb_densitysurface AS s CROSS JOIN (SELECT ST_Union(area_country) AS geom FROM country_boundaries) AS c;
I recall you said you had just one raster, if you have multiple, then you should change the above an INNER JOIN
ST_Intersects(s.rast, c.geom). If you only care about one country, you can clip to that one country.
The only reason I am unioning is so you don’t end up with duplicate rasters at country boundaries.
Hope that helps,
Regina
From: liglio....@nexxa.com.br [mailto:liglio....@nexxa.com.br]
Sent: Tuesday, December 6, 2022 2:20 PM
To: 'Regina Obe' <l...@pcorp.us>
Subject: RES: [postgis-users] RES: Clipping a raster and a polygon using ST_Intersection
Regina,
Thank you very much for your reply, and by the way congratulations for your book “PostGis in action” third edition, I bought and it is very good.
Just occurred to me that the ST_Intersection I gave you earlier probably does much the same as ST_Clip
R: Yes is pretty much the same. The result image is the same.
Each pixel is associated with a value in the original raster, and when you do resample, what value is associated to the pixels (in the docs “New pixel values are computed using the NearestNeighbor”).
So in your example x5, only one pixel has the original pixel value and the others 24 pixels has zero or nodata ? If I do colored scale of values it not seems right.
Another question: I do a retangular matrix larger than the country (creating an empty raster and dumping each pixel as polygon, and counting lightining inside each sqare). So I have a lot of squares to compute, and 20 years of lightining (about 2 billions).
It will be more efficient/faster if my empty raster has the shape of my country, so It will have less squares to compute. It is possible ?
Regards,
Liglio
Regina Obe
ST_Resize might be a better option
https://postgis.net/docs/RT_ST_Resize.html
ST_Resize takes in raster width/height similar to ST_Resample.
From: Regina Obe [mailto:l...@pcorp.us]
Sent: Tuesday, December 6, 2022 2:53 PM
To: 'PostGIS Users Discussion' <postgi...@lists.osgeo.org>
Cc: 'liglio....@nexxa.com.br' <liglio....@nexxa.com.br>
Subject: RE: [postgis-users] RES: Clipping a raster and a polygon using ST_Intersection
> Each pixel is associated with a value in the original raster, and when you do resample, what value is associated to the pixels (in the docs “New pixel values are computed using the NearestNeighbor”).
So in your example x5, only one pixel has the original pixel value and the others 24 pixels has zero or nodata ? If I do colored scale of values it not seems right.
Hmm that is not what I was expecting. Then again I always use the function to reduce resolution, and not for this particular purpose. I had assumed it would apply the value to the next neighboring and keep on applying that way. But sounds like I was wrong about that.
I think I was confusing that with ST_Rescale. Try using ST_Rescale instead and see if that gives expected results.
Note ST_Rescale takes as input pixel sizes. In your case you want to reduce the size of each pixel. The below reduces the pixel size by 20%
https://postgis.net/docs/RT_ST_Rescale.html
SELECT ST_Clip( ST_Rescale(rast, ST_PixelWidth(rast)*0.2, ST_PixelHeight(rast)*0.2 ) , c. area_country)
FROM country AS c
INNER JOIN tb_densitysurface AS d
ON ( ST_Intersects(d.rast, c.area_country) )
WHERE c.gid = 19;
Another question: I do a retangular matrix larger than the country (creating an empty raster and dumping each pixel as polygon, and counting lightining inside each sqare). So I have a lot of squares to compute, and 20 years of lightining (about 2 billions).
It will be more efficient/faster if my empty raster has the shape of my country, so It will have less squares to compute. It is possible ?
Yes it would be more efficient both in space and time for computing.
Just clip your raster by the country boundaries kind of like you were doing.
So something like
CREATE TABLE tb_densitysurface _country_clip AS
SELECT s.rid, ST_Clip(s.rast, geom) AS rast
FROM tb_densitysurface AS s CROSS JOIN (SELECT ST_Union(area_country) AS geom FROM country_boundaries) AS c;
I recall you said you had just one raster, if you have multiple, then you should change the above an INNER JOIN
ST_Intersects(s.rast, c.geom). If you only care about one country, you can clip to that one country.
The only reason I am unioning is so you don’t end up with duplicate rasters at country boundaries.
Sent: Tuesday, December 6, 2022 2:20 PM
To: 'Regina Obe' <l...@pcorp.us>
Subject: RES: [postgis-users] RES: Clipping a raster and a polygon using ST_Intersection
Regina,
Thank you very much for your reply, and by the way congratulations for your book “PostGis in action” third edition, I bought and it is very good.
Just occurred to me that the ST_Intersection I gave you earlier probably does much the same as ST_Clip
R: Yes is pretty much the same. The result image is the same.
Each pixel is associated with a value in the original raster, and when you do resample, what value is associated to the pixels (in the docs “New pixel values are computed using the NearestNeighbor”).
So in your example x5, only one pixel has the original pixel value and the others 24 pixels has zero or nodata ? If I do colored scale of values it not seems right.
Another question: I do a retangular matrix larger than the country (creating an empty raster and dumping each pixel as polygon, and counting lightining inside each sqare). So I have a lot of squares to compute, and 20 years of lightining (about 2 billions).
It will be more efficient/faster if my empty raster has the shape of my country, so It will have less squares to compute. It is possible ?
Regards,
Liglio
De: Regina Obe <l...@pcorp.us>
Enviada em: segunda-feira, 5 de dezembro de 2022 13:25