[postgis-users] How do you use PostGIS Raster?
Pierre Racine
I'm preparing a presentation and I need some use cases for PostGIS Raster. I would be grateful if people already using the raster side of PostGIS would describe what they already do it. Bborie, Jorge, Regina, others? These uses cases are vital for the project.
On my side we are using PostGIS raster to do raster/vector analysis over large datasets. Basically determining mean values for temperature (raster), elevation (raster) for fauna observations (point buffers). Converting everything to vector and using desktop solutions proved impracticable for datasets covering the extent of Canada.
For those who did not try the raster extension yet, don't be shy to express your planned experiments or your expectations.
Thanks all for your contribution to the project,
Pierre
_______________________________________________
postgis-users mailing list
postgi...@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users
Jorge Arévalo
Hi,
I personally see PostGIS Raster from a developer's point of view (too
close to code problems). But taking some distance, I collaborate in
two use cases right now in my company:
- Geospatial database in the cloud. We have a big raster coverage (~
50GB) in several GeoTIFF files (~ 5-6 GB each one). We want to serve
them through a WMS interface using a cloud architecture with
PostgreSQL/PostGIS/PostGIS Raster storage. We may execute gdal2tiles
against each file and store tiles in PostGIS Raster (just a
possibility). Starting project.
- Vector/raster intersection. We want to cross public cadastral
information with a base layer of raster data (orthophotos, DEMs...) to
provide some value over the cadastral information. Useful for farmers.
Ongoing project.
More ideas welcome :-)
Best regards,
--
Jorge Arévalo
Internet & Mobilty Division, DEIMOS
jorge....@deimos-space.com
http://es.linkedin.com/in/jorgearevalo80
http://mobility.grupodeimos.com/
http://gis4free.wordpress.com
http://geohash.org/ezjqgrgzz0g
Bborie Park
> Hi all,
>
> I'm preparing a presentation and I need some use cases for PostGIS Raster. I would be grateful if people already using the raster side of PostGIS would describe what they already do it. Bborie, Jorge, Regina, others? These uses cases are vital for the project.
>
> On my side we are using PostGIS raster to do raster/vector analysis over large datasets. Basically determining mean values for temperature (raster), elevation (raster) for fauna observations (point buffers). Converting everything to vector and using desktop solutions proved impracticable for datasets covering the extent of Canada.
>
> For those who did not try the raster extension yet, don't be shy to express your planned experiments or your expectations.
>
> Thanks all for your contribution to the project,
>
> Pierre
Howdy all,
I use PostGIS Raster as a developer, implementer and end-user. At my
work, I have yet to put PostGIS Raster into production as I'm currently
waiting on PostGIS 2.0 to be released.
At CVEC, our goal is to provide California vector control and public
health agencies raw and computed climate, meteorological, vector
abundance and disease risk-assessment rasters. Some of these datasets
are large (1km scale for every day from 1965 to today) so I wanted a
mechanism that allowed me to easily manage the raster data, be able to
expose the data in SQL for use by modelers and researchers, intersect
the raster data with existing vector data and deliver on-the-fly
generated rasters to end-users in vector control and public health.
In the past, we converted all our raster data into vectors in PostGIS
but that became a storage and general management nightmare in addition
to the performance hit.
Currently, I've only done minimal performance testing using one year's
daily maximum temperature data where I wanted to aggregate the data by
different time units, e.g. by month, half-month, disease week. In
addition to the storage space savings of using PostGIS raster versus
vectors in PostGIS, a query that once took 30 - 40 seconds now completes
in ~10 seconds. I do need to do more testing to see if I could enhance
performance by playing with the size of the tiles.
My development contribution the project is because there are things I'd
like to see implemented before PostGIS 2.0 gets branched. I'd like to
see ST_Accum done so that I can bring the tiles for a single day
together in one raster to be outputted to the end-user as a GDAL raster.
Pierre, is it possible to get a copy of your presentation after you've
delivered it? It may be nice to use in attracting more participants in
the academic circles. I'm currently thinking of some MRI, CT or X-Ray
usage.
-bborie
--
Bborie Park
Programmer
Center for Vectorborne Diseases
UC Davis
530-752-8380
bkp...@ucdavis.edu
Pierre Racine
> Pierre, is it possible to get a copy of your presentation after you've delivered it?
> It may be nice to use in attracting more participants in the academic circles. I'm
> currently thinking of some MRI, CT or X-Ray usage.
Thanks Bborie. You can already get the FOSS4G presentations I did in the project page. I will send you the new one when I'm done.
Pierre
Paragon Corporation
I'd like to see it too. My plans for raster are much further down on the
horizon unfortunately mostly because I've got too much on my plate and I'm
not sure how attenable they are. But here it goes:
A) You've probably figured out by now is that one of my pet dreams is to be
able to extrude building footprints right in PostGIS from aerial images.
For example now it usually takes about 2-3 years from the production of
satellite imagery before we get building footprints out of it. So all new
project construction doesn't show up.
I'm not sure how involved the process is, I imagine very involved and
expensive giving how long it seems to take.
B) My more realistic project -- which this really is still a year or 2 down
-- is for plant alarm equipment -- being able to with thermal cameras
mounted in the buildings and plants be able to determine the health of the
equipment at any point in time by analyzing the thermal camera output that
is georeferenced to the locations of the equipment.
Hope that helps,
Regina
Bryce L Nordgren
I'm not using PostGIS raster yet. We've been using PostGIS for years to keep track of active fire and burn scar detections from the MODIS instruments on board the Aqua and Terra satellites. Part of our process is to resample the point detections onto a grid which aggregates burned area over the course of a fire season. Aggregating onto a grid has been performed using an external program, but we'd like to be able to use PostGIS raster.
More recently, we're comparing a vector fire perimeter from the MODIS instrument to a higher resolution Landsat image of the area. We select the areas of the Landsat raster which are inside the perimeter vs. outside the perimeter and calculate histograms on the separate image regions. Currently, we're using ENVI and IDL to do this more or less manually. We'd like to be able to perform this inside PostGIS, as PostGIS is the source of the vector data anyway.
Bryce
Pierre Racine
What are you missing to implement any of those in PostGIS raster?
From: postgis-us...@postgis.refractions.net [mailto:postgis-us...@postgis.refractions.net] On Behalf Of Bryce L Nordgren
Sent: Tuesday, June 07, 2011 3:12 PM
To: PostGIS Users Discussion
Subject: Re: [postgis-users] How do you use PostGIS Raster?
Bryce L Nordgren
| Pierre Racine <Pierre...@sbf.ulaval.ca>
Sent by: postgis-us...@postgis.refractions.net 06/07/2011 07:30 PM
|
|
|
What are you missing to implement any of those in PostGIS raster?
For the second thing: Landsat scenes are ~7000x8000 pixels; ST_DumpAsPolygons() gives a huge set of geometries. The MODIS fire perimeters are also huge. The first step is a "global" (image-wide) analysis. It was very slow when I tried it. We need an ST_Intersection that returns a clipped raster; and we also need an ST_Union() or ST_Collect() which returns an aggregated raster.
Bryce_______________________________________________
postgis-users mailing list
postgi...@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users
Pierre Racine
>that returns a clipped raster; and we also need an ST_Union() or ST_Collect() which returns
>an aggregated raster.
What if you tiled your landsat in smaller tiles (100x100)?
Pierre
Bryce L Nordgren
> The first step is a
> >"global" (image-wide) analysis. It was very slow when I tried it.
>
I didn't get past the global analysis. On my little tutorial thing, the step that took 8 hours before I killed it was:
DATABASE=> CREATE TABLE tmp_burned AS
DATABASE-> SELECT gid, ST_Collect(ST_Intersection(polygon, utm_geom)) as tmp_burned
DATABASE-> FROM nbr_polygons, modis
DATABASE-> WHERE ST_Intersects(polygon, utm_geom)
DATABASE-> GROUP BY gid ;
This is under "Calculate burned area in polygon coverage" on: https://collab.firelab.org/software/projects/modisfire/wiki/Calculating_histograms_with_PostGIS_Raster
Bryce
Sandro Santilli
> It would be convenient if there was a "rasterize" function
> which would serve as the inverse of a ST_DumpAsPolygons().
I agree on this, and I'd actually love to work on it.
It is one of the first things I implemented in the former
PostGIS Raster type (aka CHIP).
Don't know what's the name of that function in Pierre's plan.
I'd Just need to find some fundings to start.
--strk;
() Free GIS & Flash consultant/developer
/\ http://strk.keybit.net/services.html
Pierre Racine
1) For ST_Intersection to be fast the total area covered by nbr_polygons must be relatively smaller than the modis area. If the intersecting surface is similar to your modis area, ST_Intersection will try to vectorize all the raster and this takes generally too much time and produce too much vector data.
2) For ST_Intersection to be fast with ST_Intersects your raster must be tiled when you import it with raster2pgsql using the -k option
If the polygon area is relatively small comparing with the raster area AND your raster is tiled, this kind of query should be (relatively) fast.
Pierre
> -----Original Message-----
> From: postgis-us...@postgis.refractions.net [mailto:postgis-users-
> bou...@postgis.refractions.net] On Behalf Of Bryce L Nordgren
> Sent: Tuesday, June 07, 2011 5:05 PM
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] How do you use PostGIS Raster?
>
>
>
> <https://collab.firelab.org/software/projects/modisfire/wiki/Calculating_histog
> rams_with_PostGIS_Raster>
>
> Bryce
Mathieu Basille
Here, we are using PostGIS to analyze large datasets combining rasters
and vectors. Basically, we are looking at movement and habitat selection
in a predator-prey system, using GPS-collar data over large areas. The
major use of PostGIS Raster is for the intersection of prey steps (line
segments or buffers around these segments) with different raster:
Landsat (landcover type), slope, road density, relative probability of
occurrence of the predator, etc. Note that we first tried to use ArcGIS
for this step, without success, due to many bugs and the need to correct
them by hand (which was largely too time-consuming). We also used
PostGIS Raster to intersect predator locations with the Landsat map to
estimate Resource Selection Functions (RSF) in order to build maps of
relative probability of occurrence. In the end, information of the
intersections were used in R to characterize movements of the preys on
the landscape.
Best,
Mathieu Basille.
--
~$ whoami
Mathieu Basille, Post-Doc
~$ locate
Laboratoire d'Écologie Comportementale et de Conservation de la Faune
+ Centre d'Étude de la Forêt
Département de Biologie
Université Laval, Québec
~$ info
http://ase-research.org/basille
~$ fortune
``If you can't win by reason, go for volume.''
Calvin, by Bill Watterson.
Javier de la Torre
Santosh Gaikwad
Look at the following application of Ecological Sensitive Area (ESA) of western ghats (one of the biodiversity hotspot) of India. We have used PGRaster in this case.
http://www.wetlandsofindia.org/geoserver/www/esa/esa.html (It works well in Firefox)
Thanks,
Santosh Gaikwad
Geo-Informatics Officer,
Salim Ali Centre For Ornithology
& Natural History (SACON),
12-13-588/B, Nagarjunanagar Colony,
Tarnaka,Hyderabad- 500017,
Andhra Pradesh,India.
http://www.wetlandsofindia.org http://sacon.org
Mobile : 91-8106736257
Homepage: http://www.wetlandsofindia.org/santosh/index.html
Pierre Racine
Could you tell us more on what you do on the server side? "PGRaster" you mean PostGIS Raster? Or the old pgraster?
Pierre
> -----Original Message-----
> From: postgis-us...@postgis.refractions.net [mailto:postgis-users-
> bou...@postgis.refractions.net] On Behalf Of Santosh Gaikwad
> Sent: Thursday, June 09, 2011 4:54 AM
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] How do you use PostGIS Raster?
>
Pierre Racine
Nice application. What do you use now and why do you plan on using PostGIS? Is it possible to upload a shapefile and get the carbon sequestrated for the area covered by the shapefile?
Pierre
> -----Original Message-----
> From: Javier de la Torre [mailto:jat...@vizzuality.com]
> Sent: Thursday, June 09, 2011 3:42 AM
> To: PostGIS Users Discussion
> Cc: Mari...@lists.refractions.net; Pierre Racine; ude Labbé
> Subject: Re: [postgis-users] How do you use PostGIS Raster?
>
Pierre Racine
Could you provide us with some numbers? How many geometries do you intersect? What is the total area/length covered by those geometries? With what raster size? How long it takes? Which kind of machine are you using? I would really like to use your numbers in my presentation.
Pierre
> -----Original Message-----
> From: Mathieu Basille [mailto:bas...@ase-research.org]
> Sent: Wednesday, June 08, 2011 10:09 PM
> To: PostGIS Users Discussion
> Cc: Pierre Racine; Marie-Claude Labbé
> Subject: Re: [postgis-users] How do you use PostGIS Raster?
>
Javier de la Torre
What we do is have a server with the raster and a web server exposing an API. It accepts a polygon as geojson and in command line calculate the vsum of the carbon value using starspan.
In PostGIS we could do many more things like intersections and stuff like this before calculating the sum of the raster value. We could stored polygons and with simple sql statements calculate their values... well lot more flexibility in general.
Best,
Etienne Bellemare
I'm using postgis raster to perform intersection between rasters and vector features which are mainly derived from lidar data (which yields lots of data). It's a scientific environment rather than a production environment.
Etienne
Santosh Gaikwad
We have used WKTRaster for vector raster analysis for ESA. Then we have put the results on the web. If you click on search option the website which I have given then you will see a dialog box which has criteria where you can select ESA option and then select any of the western ghat state and provided 5 min and 1 min data (right now we just have 1 min data for Goa state) and after doing search it displays the result in coloured grid. If you click on it, then you get the rank values which shows how important that grid from ESA point of view.
I am also ready to share the methodology which we have used. Basically we have used your tutorial as a guideline. http://trac.osgeo.org/postgis/wiki/WKTRasterTutorial01 Its very useful tool though we have faced some little problems.
Thanks & Regards,
Santosh
Mathieu Basille
I am resurrecting this thread to honour an old promise to Pierre. This
might be of interest for those interested in using PostGIS for the study
of wildlife movements.
First, a summary of our use of PostGIS: We are using PostGIS to analyze
large datasets combining rasters and vectors. Basically, we are looking
at movement and habitat selection in a predator-prey system, using
GPS-collar data over large areas. The major use of PostGIS Raster is for
the intersection of prey steps (line segments or buffers around these
segments) with different raster: Landsat (landcover type), slope, road
density, relative probability of occurrence of the predator, etc. Note
that we first tried to use ArcGIS for this step, without success, due to
many bugs and the need to correct them by hand (which was largely too
time-consuming). We also used PostGIS Raster to intersect predator
locations with the Landsat map to estimate Resource Selection Functions
(RSF) in order to build maps of relative probability of occurrence. In
the end, information of the intersections were used in R to characterize
movements of the preys on the landscape.
Second, some numbers: we generally work on rasters (LandSat, road
density, DEM) of approximately 150e6 pixels (25m×25m), i.e. 12600×11900,
and occasionally on a much larger raster at the scale of Québec
(LandSat) of approximately 1700e6 (20000×850000) pixels, same
resolution. We are intersecting these maps with a total of 850 000 GPS
relocations of four species, i.e. slightly less line segments between
each relocation (so-called steps). We generally contrast each step with
10 randomly generated steps, which means that we are working on hundreds
thousand steps at a time (we generally process them by season/species).
Here are some processing times on a Intel Xeon station (4 cores @2.67
GHz) running Debian Linux, with 12 GB of RAM:
- Intersection of 1 000 000 steps (mean=200m) takes 7.3 h on the most
complex rasters (i.e. rasters which vary a lot along the steps, hence
increasing computation times). Speed seems fairly constant, with
approximately 130-140 thousand steps per hour.
- On simpler rasters (i.e. less variation along the steps), computation
time can be up to 10 times higher (1.5e6 steps per hour).
- Intersection of 57 000 buffers (i.e. polygons around points, radius =
300m) takes almost 1h to get the composition of the buffer.
Hope this is useful!
Best,
Mathieu Basille.