[OSM-talk] Statistics on road network length?
Matthias Meißer
currently I looking for some analysis on the length of the OSM
road/footway networks. Even this is just a very simple compare, it's a
very common way to compare data completeness in professional GIS scene.
I looked everywhere in the wiki but just come up with this global length
history: http://www.beyonav.com/openstreetmap
Does anybody has an idea if there is another platform, that splits up
this number by countries?
Thanks,
Matthias
(user:!i!)
_______________________________________________
talk mailing list
ta...@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk
Frederik Ramm
On 11/16/11 12:22, Matthias Meißer wrote:
> Does anybody has an idea if there is another platform, that splits up
> this number by countries?
A platform, no, but a means - illustrated for a small country:
% wget
http://svn.openstreetmap.org/applications/utils/filter/osm-length/osm-length-2.pl
% wget -O-
http://download.geofabrik.de/osm/europe/luxembourg.osm.bz2|bzcat|perl
osm-length-2.pl > /dev/null
highway length sums (metres):
track 4845387m
secondary 1982316m
residential 1629088m
primary 981455m
unclassified 787765m
service 606435m
footway 580123m
path 462068m
tertiary 444727m
cycleway 391501m
motorway 297223m
motorway_link 122052m
trunk 41977m
construction 33806m
pedestrian 19771m
steps 18720m
road 14388m
trunk_link 11592m
living_street 10476m
bridleway 8280m
primary_link 6816m
raceway 5100m
services 2152m
rest_area 877m
platform 661m
secondary_link 573m
proposed 305m
noway 164m
peer 78m
crossing 72m
turning_circle 31m
ford 24m
TOTAL 13306019m
%
Note that for comparison with official figures you will somehow have to
account for dual carriageways which are mapped, and counted, as distinct
roads.
Bye
Frederik
Martin Koppenhoefer
> Hi,
> currently I looking for some analysis on the length of the OSM road/footway
> networks. Even this is just a very simple compare, it's a very common way to
> compare data completeness in professional GIS scene.
For Italy there is detailed statistics available on gfoss.it :
http://www.gfoss.it/osm/stat/
AFAIK these are done by Diego Guidotti.
cheers,
Martin
Matthias Meißer
bye
Matthias
Maurizio Napolitano
> http://www.gfoss.it/osm/stat/
>
> AFAIK these are done by Diego Guidotti.
Diego is available to the release of the code :)
Hermann Peifer
> Thanks Frederik! I think this will do it :)
>
> bye
> Matthias
>
Just to let you know: I am occasionally running my own script which is
kind of similar to Frederik's Perl script. As OSM input data, I am using
the .pbf files from [1]
For the Haversine Formula, I followed the explanations at [2]. This
formula calculates distances between 2 points on a spherical Earth.
There can be 1% of differences compared to distances measured on an
ellipsoid. I am using 6367516.477 m as Earth radius, which is the mean
of a^2/b and b^2/a, where a and b are the semi-major/minor axis of the
WGS84 ellipsoid.
(As a side remark: the distance between 2 nodes of an OSM way is often
below 100m, so any considerations about distances on spheres vs.
ellipsoids are rather of an academical nature.)
Concerning the issue with dual carriageways, I am using a brute-force
approach and divide the calculated distance by 2 if these conditions are
true:
highway ~ /^(motorway|trunk|primary|secondary|tertiary)/ &&
oneway ~ /yes|true|1/
(If someone has a better algorithm: I would be interested.)
Hermann
[1] http://download.geofabrik.de/osm/europe/
[2] http://mathforum.org/library/drmath/view/51879.html
dig...@arcor.de
Too bad, cause I think a monthly run would be ok for everybody :/
Matthias
----- Original Nachricht ----
Von: Hermann Peifer <pei...@gmx.eu>
An: Matthias Meißer <dig...@arcor.de>
Datum: 17.11.2011 10:26
Betreff: Re: Statistics on road network length?
--
ThomasB
!i! wrote:
>
> Well even if it's bad style to ask if someone else can do the job, but
> couldn't we create an addon for existing statistic services, that can
> calculate this values? I contacted the people behind
> http://osmstats.altogetherlost.com but even if they hope to provide some
> country specific calculations, they say that exactly this value would make
> to heavy load for their servers.
> Too bad, cause I think a monthly run would be ok for everybody :/
>
> Matthias
>
>
I guess there are not so many people out there having a machine with more
than 20GB RAM. I tried to calculate Germany with a 2009 planet file but
failed, I guess because of RAM. So calculating large countries with 2011
data may require RAM well in excess of 20GB. Smaller countries work fine.
--
View this message in context: http://gis.638310.n2.nabble.com/Statistics-on-road-network-length-tp7000059p7009261.html
Sent from the General Discussion mailing list archive at Nabble.com.
Hermann Peifer
>
> I guess there are not so many people out there having a machine with more
> than 20GB RAM. I tried to calculate Germany with a 2009 planet file but
> failed, I guess because of RAM. So calculating large countries with 2011
> data may require RAM well in excess of 20GB. Smaller countries work fine.
>
One idea could be to use Geofabrik's OSM files by country. My rather
crude script needs 2 seconds for processing Albania and 6 minutes each
for France and Germany (on a machine with 8G RAM). See below.
Hermann
Fri Nov 18 19:36:28 CET 2011 +++ albania.osm.pbf
Fri Nov 18 19:36:30 CET 2011 +++ andorra.osm.pbf
Fri Nov 18 19:36:30 CET 2011 +++ austria.osm.pbf
Fri Nov 18 19:37:40 CET 2011 +++ azores.osm.pbf
Fri Nov 18 19:37:40 CET 2011 +++ belarus.osm.pbf
Fri Nov 18 19:37:52 CET 2011 +++ belgium.osm.pbf
Fri Nov 18 19:38:19 CET 2011 +++ bosnia-herzegovina.osm.pbf
Fri Nov 18 19:38:23 CET 2011 +++ bulgaria.osm.pbf
Fri Nov 18 19:38:29 CET 2011 +++ croatia.osm.pbf
Fri Nov 18 19:38:42 CET 2011 +++ cyprus.osm.pbf
Fri Nov 18 19:38:45 CET 2011 +++ czech_republic.osm.pbf
Fri Nov 18 19:39:37 CET 2011 +++ denmark.osm.pbf
Fri Nov 18 19:40:13 CET 2011 +++ estonia.osm.pbf
Fri Nov 18 19:40:19 CET 2011 +++ faroe_islands.osm.pbf
Fri Nov 18 19:40:19 CET 2011 +++ finland.osm.pbf
Fri Nov 18 19:40:54 CET 2011 +++ france.osm.pbf
Fri Nov 18 19:46:46 CET 2011 +++ germany.osm.pbf
Fri Nov 18 19:53:23 CET 2011 +++ great_britain.osm.pbf
Fri Nov 18 19:55:47 CET 2011 +++ greece.osm.pbf
Fri Nov 18 19:56:10 CET 2011 +++ hungary.osm.pbf
Fri Nov 18 19:56:20 CET 2011 +++ iceland.osm.pbf
Fri Nov 18 19:56:24 CET 2011 +++ ireland.osm.pbf
Fri Nov 18 19:56:41 CET 2011 +++ isle_of_man.osm.pbf
Fri Nov 18 19:56:41 CET 2011 +++ italy.osm.pbf
Fri Nov 18 19:59:07 CET 2011 +++ kosovo.osm.pbf
Fri Nov 18 19:59:13 CET 2011 +++ latvia.osm.pbf
Fri Nov 18 19:59:18 CET 2011 +++ liechtenstein.osm.pbf
Fri Nov 18 19:59:18 CET 2011 +++ lithuania.osm.pbf
Fri Nov 18 19:59:23 CET 2011 +++ luxembourg.osm.pbf
Fri Nov 18 19:59:26 CET 2011 +++ macedonia.osm.pbf
Fri Nov 18 19:59:27 CET 2011 +++ malta.osm.pbf
Fri Nov 18 19:59:28 CET 2011 +++ moldova.osm.pbf
Fri Nov 18 19:59:32 CET 2011 +++ monaco.osm.pbf
Fri Nov 18 19:59:32 CET 2011 +++ montenegro.osm.pbf
...
ThomasB
Hermann Peifer wrote:
>
> One idea could be to use Geofabrik's OSM files by country. My rather
> crude script needs 2 seconds for processing Albania and 6 minutes each
> for France and Germany (on a machine with 8G RAM). See below.
>
not sure we are using the same script. With osmosis --read-pbf
file=/home/wicking/osm/germany.osm.pbf --write-xml file=- | perl
osm-length-2.pl > /dev/null
I am 15 minutes after the start at 14GB RAM usage and still increasing.
--
View this message in context: http://gis.638310.n2.nabble.com/Statistics-on-road-network-length-tp7000059p7009497.html
Sent from the General Discussion mailing list archive at Nabble.com.
_______________________________________________
Frederik Ramm
On Fri, 18 Nov 2011 11:56:53 -0800 (PST)
ThomasB <toba...@yahoo.de> wrote:
> not sure we are using the same script. With osmosis --read-pbf
> file=/home/wicking/osm/germany.osm.pbf --write-xml file=- | perl
> osm-length-2.pl > /dev/null
> I am 15 minutes after the start at 14GB RAM usage and still
> increasing.
Yes, the osm-length perl script wastes a lot of memory by keeping all
nodes in a Perl hash. It would be relatively easy to write an efficient
road length calculator based on the Osmium framework; that one should
then work with a far smaller memory footprint.
Bye
Frederik
Jo
> approach and divide the calculated distance by 2 if these conditions are
> true:
>
> highway ~ /^(motorway|trunk|primary|secondary|tertiary)/ &&
> oneway ~ /yes|true|1/
>
You probably want to add oneway=-1 to that list. There are some
oddballs who used that indicate oneway is against the vector sense.
Also JOSM proposes to use it when reversing a way.
Jo
Hermann Peifer
>> Concerning the issue with dual carriageways, I am using a brute-force
>> approach and divide the calculated distance by 2 if these conditions are
>> true:
>>
>> highway ~ /^(motorway|trunk|primary|secondary|tertiary)/&&
>> oneway ~ /yes|true|1/
>>
>
> You probably want to add oneway=-1 to that list. There are some
> oddballs who used that indicate oneway is against the vector sense.
> Also JOSM proposes to use it when reversing a way.
>
> Jo
Thanks for the hint. I already thought about it but forgot to mention
that the above tests are using regular expressions. ^motorway also
matches motorway_link, etc. and 1 also matches -1.
Hermann
ThomasB
/# Fatal error in CALL_AND_RETRY_2
# Allocation failed - process out of memory/
although there is still sufficient (physical) memory available. A change of
osmjs -l array -> disk did not help.
> talk@
> http://lists.openstreetmap.org/listinfo/talk
>
--
View this message in context: http://gis.638310.n2.nabble.com/Statistics-on-road-network-length-tp7000059p7011662.html
Sent from the General Discussion mailing list archive at Nabble.com.
_______________________________________________
Jochen Topf
That error comes from the Javascript part, not from the node location store, so
changes to the -l parameter will not help. It's probably the same problem
Martijn reported on dev recently. There seems to be a memory leak somewhere in
the Javascript code of Osmium. What OSM extract did you try to run it on?
Jochen
On Sat, Nov 19, 2011 at 08:52:39AM -0800, ThomasB wrote:
> Date: Sat, 19 Nov 2011 08:52:39 -0800 (PST)
> From: ThomasB <toba...@yahoo.de>
> To: ta...@openstreetmap.org
> Subject: Re: [OSM-talk] Statistics on road network length?
--
Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298
Frederik Ramm
On 11/18/2011 10:58 PM, Frederik Ramm wrote:
> On Fri, 18 Nov 2011 11:56:53 -0800 (PST)
> ThomasB<toba...@yahoo.de> wrote:
>> not sure we are using the same script. With osmosis --read-pbf
>> file=/home/wicking/osm/germany.osm.pbf --write-xml file=- | perl
>> osm-length-2.pl> /dev/null
>> I am 15 minutes after the start at 14GB RAM usage and still
>> increasing.
>
> Yes, the osm-length perl script wastes a lot of memory by keeping all
> nodes in a Perl hash. It would be relatively easy to write an efficient
> road length calculator based on the Osmium framework; that one should
> then work with a far smaller memory footprint.
http://www.remote.org/frederik/tmp/osmium_road_length.cpp
Does all of Germany (from a PBF file) in less than 5 minutes and using
less than 2 GB of RAM. I haven't got the time to do this properly right
now - it just dumps a list of results for all highway types to stdout
and sadly includes all the tag rubbish:
* 0.2km
Access_Ramp 0.0km
Altmarkt 0.2km
FIXME 0.8km
Fußweg 0.0km
Hinterer Spielbach 0.1km
Hohndorfer Weg 1.2km
Privatstraße 0.1km
Sandstraße 0.1km
Weikersdorfer Weg 0.1km
abandoned 23.4km
abandonned 0.3km
abendoned 0.5km
access 0.8km
access_ramp 0.1km
agriculture 0.8km
almost_path 0.4km
animal_pass 0.1km
animal_trail 0.2km
area:<unterschiedlich> 0.1km
bicycle 0.1km
bollard 0.4km
bridge 0.0km
bridleway 2198.5km
bridleway, track 0.5km
bus_guideway 5.9km
...
(full result file for Germany here:
http://www.remote.org/frederik/tmp/osmium_road_length.result)
But anyone with minimal C++ skills should be able to polish that. (If
you do, be so kind and make your polished version available somewhere
stable, e.g. send it to Jochen for inclusion in his examples directory
or so.) A power user might even want to take Peter Koerner's history
splitter which is OSM based and has the ability to cut out OSM data
along polygon boundaries, and add the length formula from this script so
you can easily make a separate counting for lots of places.
Best wishes
Frederik
--
Frederik Ramm ## eMail fred...@remote.org ## N49°00'09" E008°23'33"
Hermann Peifer
>
> http://www.remote.org/frederik/tmp/osmium_road_length.cpp
>
> Does all of Germany (from a PBF file) in less than 5 minutes and using
Thanks for this one. I just managed to do germany.osm.pbf in 3 minutes
and 3 seconds. Memory usage only went up to 1.3G.
Hermann
