Feedback and Advice Needed on a Geotrellis Based App

[Kemal Ardıl Gülez]

Hello All,

We have been working on a weighted overlay app. You can find the link here:

http://harita.citysurf.com.tr/weightedoverlay/

There are a whole bunch of flaws that are easily noticable

1- Performance: So easily noticable. Every single tile takes whole seconds to load. I definitely would like that to be faster. Here is what we currently do:

    - The code and the endpoints are very similar to geotrellis-chatta-demo. I have a few single band layers, I assign them weights -> sum up their values * weight -> render png using a color map.
    - Tiles have 30 meters of resolution.
    - Layers are stored on local file system
    - App is working on a non-dedicated server with 4G java memory setting on a development server

    What measures can I take in order to make the stuff run faster other than throwing money at it? Is the low amount of performance normal for this setup?

2- Tiles: Let me explain you the tile generation process first.
    - Single band tiles that contain point information are obtained. The tiles are generated using a function: if there are points of interest in that pixel -> 1, else -> nodata.
    - We used ArcMap to perform point density analysis on the raw tiles that are obtained in the previous step
    - Cut the tiles and they are ready to be served.

    No matter how hard we tried to make this thing work, the look is always the same: Circles on top of circles on top of circles. Did anyone else build such an application? If yes, what was the process of obtaining data? Also, what kind of an analysis can we perform over raw tiles so that they look smoother than this?

Another problem that doesn't exist in the link I provided:

When I tried adding an NDVI layer using landsat 8 images (same 30 meters of resolution), the app stopped being able to respond to tile requests and gave me the error "Configured registration timeout of 1 second expired, stopping" or something along that line. I tried increasing the registration timeout but making it 30 seconds did no good for me either. I didn't gdalwarp the landsat images with the extent of all other layers (which are the same) before generating a tile out of them, so that might be a problem. I'll try again with gdalwarp. Any other tips about how to do this?

Thank you all in advance.

[Eugene Cheipesh]

Hello Kemal,

To address your question about performance, I would expect your setup to have very high performance, I don’t see your code but the basic components should be:

- File layer, which is appropriate for this scale of data and is very fast to query and read tiles

- FileValueReader, which does not require a SparkContext and has very low overhead.

- Render step, which is basic tile math

The fact that you don’t even need to use SparkContext on this type of endpoint makes expected latency very good.

Here are couple of things you should check:

- JVM memory, it defaults to 512MB, but you would probably want 1G-2G for a tile server like this. If you’re passing 4G in -Xmx argument to java than it should be more than enough. It would be good to double check that it is actually using this memory and the setting is not lost in the startup. For instance if you’re starting this from SBT I believe javaOptions are not considered unless you’re also forking.

- spray request parallelism: I assume you’re using spray-routing to make the endpoints. Spray does not handle requests asynchronously by default. You need to wrap your response in instance of Future. What it looks like here is that it’s almost handling the tile requests sequentially. In special cases you would even want to put the request completing on a separate thread pool, otherwise they will share the default thread pool with request routing actors. But that has not been needed for these type of tile-server applications.

-- 
Eugene Cheipesh

--
You received this message because you are subscribed to the Google Groups "geotrellis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geotrellis-us...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.