Mapnik on mobile

[Rinigus]

Hi,

I am working on using Mapnik as a rendering backend for mobile offline maps. Since the application is developed for Sailfish (Linux mobile distro), I have most Linux development tools available. At present, I used SQLite to store the data. With the adaptation of OSM Bright and tailored import scripts, I manage to get quite decent performance on mobile (about a second for a tile of 1024x1024 at most of the zoom levels, tiles rendered in parallel on all CPUs => 4 tiles per second on quad-core phone; measurements as judged-by-eye). RAM usage is also reasonably low, <100 MB RSS, and Mapnik probably uses only a fraction of it. 

The only issue is a size of SQLite database, an expected problem experienced by others as well. For DK, its about 1GB - haven't tried Germany yet. So, before I engage into compression of SQLite representation, maybe there is an alternative format that I am missing? There is https://github.com/mapbox/mapnik-vector-tile, but I must say that its not clear to me:

* would this format lead to smaller datasets (if used correctly)?;

* how to generate these tiles from PBFs ?

* how to use them with Mapnik? XML styles?

Any other formats recommended for my use case? There is also an option to write my own input plugin, but I haven't looked deeply into it yet. 

If anyone is interested in using Mapnik with SQLite, the style and import scripts are available at https://github.com/rinigus/osm-bright-sqlite

Best wishes,

Rinigus

[Blake Thompson]

Rinigus,

If you are building a rendering engine for mobile you would be much better off using a GL based mapping software such as Mapbox GL native.

I would recommend using Mapbox Vector Tiles for compression, this will work with GL or with mapnik. As for vector tile creation, there are lots of different ways to do this. https://github.com/mapbox/awesome-vector-tiles

 Probably the quickest to setup and get going would be Tippecanoe, but you would probably need to convert your datasets to another format before doing this. You could also look into tilelive or node-mapnik. 

https://github.com/mapbox/tippecanoe/

Thanks,

Blake Thompson

--
You received this message because you are subscribed to the Google Groups "mapnik" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapnik+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rinigus]

Hi Blake,

thank you for fast reply! Indeed, GL-based solution would be better in long run. We have rather small amount of developers on this platform, so we have to use solutions that are (almost) ready. Since we have several apps that can communicate with the map providers to fetch the map tiles, routes, and search for POIs/addresses, I have been developing a server that we can use as a drop-in replacement for online services. It works reasonably well, allows us to get offline maps on devices and re-use existing applications. We just have to run "server" and "client" on the devices at the same time. In future, when the GL visualization will be implemented by the clients, I could start just providing the corresponding data on the server as well. However, we have other parts that would have to be implemented first on the clients, so GL is not priority yet. At present, I am going through the code and polishing the components - hence the wish to improve map rendering via Mapnik.

Now, you mention that Mapnik can use vector tiles. Do you have an example somewhere on how to implement Mapbox Vector Tiles interaction with Mapnik allowing me to render MVTs into raster tiles on device?

Cheers,

Rinigus

--
You received this message because you are subscribed to a topic in the Google Groups "mapnik" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mapnik/S2V24zJpjEY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mapnik+unsubscribe@googlegroups.com.

[Blake Thompson]

Rinigus, 

I really think that you would be much better served to use https://github.com/mapbox/mapbox-gl-native/ first and right away here the more I hear about your application. It comes with a lot of features that you would have to integrate yourself, such as caching, etc. It also will just be a better look and feel. You will have to write probably more custom code to just make mapnik work and the performance would not be as good. Additionally, the binary size of the GL client and memory usage should be much lower. You might look into the Android bindings/SDK when doing this as well, as it might​ help guide you to how to integrate all the things you are talking about -- or might just work right away, I am not certain. 

If you still want to mapnik + vector tiles, you might look at the node-mapnik bindings for these things or the tests on mapnik-vector-tile. 

https://github.com/mapbox/mapnik-vector-tile/blob/master/test/vector_tile.cpp#L122

Mapnik-vector-tile provides ways to both render vector tiles as a mapnik datasource and render an image from a vector tile. 

Thanks,

Blake Thompson

[Rinigus]

Blake,

Thank you very much for your insight! I will look into Mapbox GL Native again. I did that earlier as well as on tangram-es, but was driven away due to the lack of clarity in offline support (Mapbox GL Native) or frequent crashes on Linux (Tanagram-ES). What was also not clear from Mapbox GL Native README was the target applications. It seems to be not just rendering library but also fetching data in the background, for example. At least that I managed to get only from reading the current and closed issues. But I will look again into it. In long term, we would probably switch to GL-based solutions on the client side and remove rendering from the server side that I develop. 

As for Mapnik, it fitted the bill quite well: great looking maps, clear target on rendering tiles from provided data. Since available client apps already have all logic implemented with caching, that's not an issue. Also, incorporation of Mapnik drawing is rather simple (maybe 10-20 lines of code), main issue was adopting a style and constructing corresponding SQLite import (as soon as you figure out what's going on).

In general, the platform I am working on supports C++11 (not 14), Qt/QML, and python with its Qt/QML bindings (python-QML is used by several popular map applications on our platform). We don't use Java, at least I am not aware of any native applications in it. Hence my look towards technologies that were implemented on server side of the things (mapnik, for example). You also use Mapbox GL on the server side and I would probably look at that in the beginning. Most probably its around some Java code though.

As for using vector tiles in Mapnik: I guess the best documentation is in example code, right? From looking through the issues (again, really hard to get any hands on documentation) it seems that one creates tiles based on SQL queries that would have to take into account the expected zoom level of the tile. However, its not clear how do I style the maps in Mapnik later. Do I use similar style/layer XML? What would be instead of Datasource in XML? 

Best wishes and thank you very much for your help and pointers!

Rinigus