Performance of mapnik rendering

[Kai Weber]

I've setup a PostGIS-DB and mod_tile/Mapnik based renderer on separate servers.
The performance of the render process is unacceptable. Following is the output
from generate_image.py:

$ time ./generate_image.py
output image to image.png!

real 2m57.561s
user 0m59.188s
sys 0m1.448s

The DB disks are SSDs and fairly well equipped hardware, the network connection
between the two hosts is 1Gb/s.

How long should the process normally take? Any ideas how to find the
bottleneck?

Regards, Kai

[Frederik Ramm]

Hi,

On 01/02/2012 09:45 PM, Kai Weber wrote:
> How long should the process normally take?

That depends on what area you are rendering, with what stylesheet, what
Mapnik version, what PostgreSQL/PostGIS version, on the structure and
indexing of your database tables, the configuration of your PostgreSQL
database, and probably a number of further factors, so be a little more
verbose and we might have an idea ;)

Are you absolutely sure that you have a *working* Gigabit network (try
to copy a large file)?

Bye
Frederik

--
Frederik Ramm ## eMail fred...@remote.org ## N49�00'09" E008�23'33"

[Iván Sánchez Ortega]

It depends. I'd first look at SQL queries running with no indexes, i.e. look
at what SQL queries run during those two minutes (see
http://chrismiles.info/systemsadmin/databases/articles/viewing-current-
postgresql-queries/ ), then explain those queries.

And as Frederik said, what about the network load? If you have some kind of
monitoring facilities on those server (munin, etc), it's worth to have a look
at them during the renderings.

--
----------------------------------
Iván Sánchez Ortega <iv...@sanchezortega.es> <iv...@geonerd.org>

http://ivan.sanchezortega.es
MSN:iv...@void.sanchezortega.es
Jabber:ivans...@jabber.org ; ivans...@kdetalk.net
IRC: ivansanchez @ OFTC & freenode

[feverzsj]

hi, Kai

 

>How long should the process normally take? Any ideas how to find the

>bottleneck?

 

I think your timing is totally normal. Besides hardware/data/db, line label placement of road could also take a very long time, especially for long road.

You may check out the tiles that take a long time to render. Generlly, it would contain a small part of a very long road.

Possible solution: split the long road in to smaller parts.

 

 

regards, zsj

[Kai Weber]

* Frederik Ramm <fred...@remote.org>:

> >How long should the process normally take?
>
> That depends on what area you are rendering, with what stylesheet,
> what Mapnik version, what PostgreSQL/PostGIS version, on the
> structure and indexing of your database tables, the configuration of
> your PostgreSQL database, and probably a number of further factors,
> so be a little more verbose and we might have an idea ;)

I did a fresh install with the following components

Postgres 8.4
libmapnik 0.7.1
mapnik-utils from http://svn.openstreetmap.org/applications/rendering/mapnik/
renderd & mod_tile 0.5.1 from http://repository.azae.net/debian/squeeze/
Apache 2.2.3

I followed the various guides from the openstreetmap wiki. The first thing
I did after experiencing the slow map rendering was to tune postgres
(shared_mem ~ 3GB, higher workmem, fsync off...). I did not notice a big
difference.

Imported the full planet file with osm2psql in 16 hours (4x SSD, 12GB RAM).

> Are you absolutely sure that you have a *working* Gigabit network
> (try to copy a large file)?

I will have another look and get back later today.

FIY, this setup will be the new map server for gpsies.com