Using online routable files (.rd5) for production

[Vianney Coppé]

Hello,

Can the routable files available at http://brouter.de/brouter/segments4/ be downloaded weekly/monthly for production?

And out of curiosity, why are the routable files much lighter than for other routing engines (GraphHopper, OSRM)? Because the routing profiles are applied at runtime and less pre-processing is made?

Best regards,

Vianney

[abrensch]

>> Can the routable files available at http://brouter.de/brouter/segments4/ be downloaded weekly/monthly for production?

The server is at 3 TB downstream per month, which is pretty much for a 15 Euro/month virtual server, so no guarantees that I will not limit the download somehow in the future.

But you can use the diff-files ( *.df5 ) for a delta-download. They are kept for 9 days, so this is good for anything between daily and weekly.

For how to apply the diff see: https://github.com/abrensch/brouter/blob/master/brouter-mapaccess/src/main/java/btools/mapaccess/Rd5DiffValidator.java

Or you can do the preprocessing all by yourself, it's not so much harder. Marcus is offerning a dockerized version for that, or you can setup yourself. It uses "osmupdate" which also uses a delta-download to apply diffs on an osm-planet-pbf file.

rd5-files are small in size because I spent quite some engeneering on that. they are organized as microtiles of 1/32 degree square in size, and these microtiles are compressed with a specially tailored compression algorithm that comes close to the entropy-limit.

regarrds, Arndt

[Vianney Coppé]

Hi,

Thanks for your answer!

Diff-files are a very nice feature. I have written some Java code which takes two directories as inputs : the one containing the segments and the one containing the diff folders. It then applies the diff files to the right segment and outputs the result in a specified output folder. If you find that interesting, I can create a pull request. (and also scripts to download all missing segments or diff files)

One question about it: is it correct to use the function "recoverFromDelta" to apply the changes?

Also, what are the requirements for a routing server serving let's say 100k requests/day?

If I was to use this setup, I would happily contribute to the cost of the server hosting the segments and diff files.

Best regards,

Vianney

[abrensch]

>> Also, what are the requirements for a routing server serving let's say 100k requests/day?

My frontend server is at > 100k requests/day (in summer), and it's a virtual server with 8 GB and 8 cores .Just make sure that there's enough memory left for the file-system-buffer to contain all the rd5-data

However, these 100k requests are mostly short routings from multi-waypoint-route planning. If you are talking about travelling-salesman-problem calculations (TSP) , it really depends on the average distance. For nation-wide TSP-matrices we would need some algortithmic additions.

The server that does the preprocessing is a physical server with 40 GB where I am just a guest, and the preprocessing-job uses at max 6 GB java-heap memory.