today: massive LiDAR holdings of National Survey of Finland become open data
605 views
Skip to first unread message
Martin Isenburg
May 9, 2012, 8:00:22 AM5/9/12
to LAStools - efficient tools for LiDAR processing
Hello
today the LiDAR holdings of the National Land Survey of Finland became
open data. That means massive amounts of LAS files went online and can
be freely downloaded. The files are provided exclusively as LASzip
compressed LAS files (i.e. in the LAZ format) which - according to the
guy in charge - compacted the total data amount from around 3.1
TeraBytes down to 475 GigaBytes. You can download the data here:
https://tiedostopalvelu.maanmittauslaitos.fi/tp/kartta?lang=en
it can be bit tricky to find those areas in Finland where data is
currently available, but here is an index that help you zoom in on the
relevant locations:
http://www.maanmittauslaitos.fi/sites/default/files/MML_laserkeilaukset_2008-2012_30042012.pdf
and here some more overall information on the open data policy of the
National Land Survey of Finland:
http://www.maanmittauslaitos.fi/en/tiedotteet/2012/05/national-land-survey-topographic-data-now-freely-available
Regards,
Martin @lastools
today the LiDAR holdings of the National Land Survey of Finland became
open data. That means massive amounts of LAS files went online and can
be freely downloaded. The files are provided exclusively as LASzip
compressed LAS files (i.e. in the LAZ format) which - according to the
guy in charge - compacted the total data amount from around 3.1
TeraBytes down to 475 GigaBytes. You can download the data here:
https://tiedostopalvelu.maanmittauslaitos.fi/tp/kartta?lang=en
it can be bit tricky to find those areas in Finland where data is
currently available, but here is an index that help you zoom in on the
relevant locations:
http://www.maanmittauslaitos.fi/sites/default/files/MML_laserkeilaukset_2008-2012_30042012.pdf
and here some more overall information on the open data policy of the
National Land Survey of Finland:
http://www.maanmittauslaitos.fi/en/tiedotteet/2012/05/national-land-survey-topographic-data-now-freely-available
Regards,
Martin @lastools
Terje Mathisen
May 9, 2012, 11:18:46 AM5/9/12
to last...@googlegroups.com, Martin Isenburg
Martin Isenburg wrote:
> Hello
>
> today the LiDAR holdings of the National Land Survey of Finland became
> open data. That means massive amounts of LAS files went online and can
> be freely downloaded. The files are provided exclusively as LASzip
> compressed LAS files (i.e. in the LAZ format) which - according to the
> guy in charge - compacted the total data amount from around 3.1
> TeraBytes down to 475 GigaBytes. You can download the data here:
>
> https://tiedostopalvelu.maanmittauslaitos.fi/tp/kartta?lang=en
I just tried this, quickly locating a 3x3 grid of interesting laz files:
> Hello
>
> today the LiDAR holdings of the National Land Survey of Finland became
> open data. That means massive amounts of LAS files went online and can
> be freely downloaded. The files are provided exclusively as LASzip
> compressed LAS files (i.e. in the LAZ format) which - according to the
> guy in charge - compacted the total data amount from around 3.1
> TeraBytes down to 475 GigaBytes. You can download the data here:
>
> https://tiedostopalvelu.maanmittauslaitos.fi/tp/kartta?lang=en
(Forests, lakes and building areas just outside Helsinki)
L4134E4.laz
L4134F3.laz
L4134F4.laz
L4134G2.laz
L4134G4.laz
L4134H1.laz
L4134H2.laz
L4134H3.laz
L4134H4.laz
My next step was to convert the first file to contours using las2iso:
las2iso -i L4134E4.laz -last_only -iso_every 1 -oshp
After 5-10 seconds Win7 told me: "las2iso has stopped working".
(I first tried this with all 9 input files, and using
-keep_classification 2 along with -last_only, and got a crash for every
input file. The simpler command line is sufficient to demonstrate the
problem.)
I am running the very latest lastools (downloaded an hour ago), I simply
unpacked everything on top of my previous (working) installation.
las2las have happily converted the laz files to las, lasview shows them
OK (the LIDAR data seems to be pretty well pre-classified into
ground/vegetation/buildings etc).
Any ideas?
Terje
>
> it can be bit tricky to find those areas in Finland where data is
> currently available, but here is an index that help you zoom in on the
> relevant locations:
>
> http://www.maanmittauslaitos.fi/sites/default/files/MML_laserkeilaukset_2008-2012_30042012.pdf
>
> and here some more overall information on the open data policy of the
> National Land Survey of Finland:
>
> http://www.maanmittauslaitos.fi/en/tiedotteet/2012/05/national-land-survey-topographic-data-now-freely-available
>
> Regards,
>
> Martin @lastools
>
- <Terje.M...@tmsw.no>
"almost all programming can be viewed as an exercise in caching"
Martin Isenburg
May 10, 2012, 6:43:11 AM5/10/12
to LAStools - efficient tools for LiDAR processing
Hello,
> I just tried this, quickly locating a 3x3 grid of interesting laz files:
> (Forests, lakes and building areas just outside Helsinki)
combining these 9 tiles may be a bit much. Whenever you use filters
that lower the number of input points (e.g. the '-first_only', ' -
last_only, -keep_class 2' ...) you may want to also add the '-
extra_pass' option so that las2iso first counts how many points will
actually be in the triangulation before it allocates the expected
memory for it. This allows you to work with larger inputs and stay
within the 2 GB memory limit on a 32 bit windows executable.
but thanks for reporting. i need to look into catching this memory
overflow so i can terminate with a nicer error message. (-:
in a follow-up email, Terje writes:
"I have done some more investigation, and it seems to be a resource
limitation that las2iso doesn't catch: Each of the Finn LIDAR files
seems to cover a 3x3 km area, so with 1m DEM resolution that's 9M
cells and well over the 2M limit of the unlicensed toolkit, right?
When I tried to generate only 10% of my contours, i.e. -iso_every 10,
the process did finish, and in not too many seconds at that."
it has nothing to do with the license protection. the tools always
work in the same manner. the only different is that unlicensed tools
add a certain amount of noise once you go above a certain threshold to
the output. the tool will always inform you when it is doing so.
"I also found an interesting issue with the LIDAR file itself: The
area is typical Finland, i.e. fairly flat (+5 to +40 or 50), with zero
mountains, but the LIDAR claims heights from -21 to +1830m! I am
guessing that that 1830m hit must be a spurious return from a bird or
water droplet(s)?"
yes. i just did the following lasinfo run (see below) on a different
tile from the finnish land survey data and you will notice the z
histogram suggests birds or dirt or scanner malfunction.
regards,
martin
C:\lastools\bin>lasinfo N4324D1.laz -histo z 100
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
global_encoding: 0
project ID GUID data 1-4: 0 0 0 ''
version major.minor: 1.2
system identifier: ''
generating software: 'EspaEngine'
file creation day/year: 0/0
header size: 227
offset to point data: 229
number var. length records: 0
point data format: 1
point data record length: 28
number of point records: 20255836
number of points by return: 0 0 0 0 0
scale factor x y z: 0.01 0.01 0.01
offset x y z: 0 0 0
min x y z: 434000.00 6900000.00 67.32
max x y z: 436999.99 6902999.99 658.43
the header is followed by 2 user-defined bytes
LASzip compression (version 2.1r0 c2 50000): POINT10 2 GPSTIME11 2
reporting minimum and maximum for all LAS point record entries ...
x 43400000 43699999
y 690000000 690299999
z 6732 65843
intensity 0 255
edge_of_flight_line 0 1
scan_direction_flag 0 1
number_of_returns_of_given_pulse 1 4
return_number 1 4
classification 1 14
scan_angle_rank -25 27
user_data 134 251
point_source_ID 25 108
gps_time 299162.864868 464211.811502
z coordinate histogram with bin size 100
bin [0,100) has 11983153
bin [100,200) has 8272571
bin [200,300) has 100
bin [400,500) has 2
bin [500,600) has 8
bin [600,700) has 2
average z coordinate 98.6991
number of points by return was not set in header: 17403168 2494071
340856 17741 0
overview over number of returns of given pulse: 14905813 4307014
971616 71393 0 0 0
histogram of classification of points:
2439694 Unclassified (1)
2906267 Ground (2)
809072 Low Vegetation (3)
1399 Low Point (noise) (7)
139348 Water (9)
32901 Reserved for ASPRS Definition (10)
13925375 Reserved for ASPRS Definition (13)
1780 Reserved for ASPRS Definition (14)
> I just tried this, quickly locating a 3x3 grid of interesting laz files:
> (Forests, lakes and building areas just outside Helsinki)
> My next step was to convert the first file to contours using las2iso:
> las2iso -i L4134E4.laz -last_only -iso_every 1 -oshp
> After 5-10 seconds Win7 told me: "las2iso has stopped working".
> (I first tried this with all 9 input files, and using
> -keep_classification 2 along with -last_only, and got a crash for every
> input file. The simpler command line is sufficient to demonstrate the
> problem.)
las2iso is memory limited to about 25-30 million input points.
> las2iso -i L4134E4.laz -last_only -iso_every 1 -oshp
> After 5-10 seconds Win7 told me: "las2iso has stopped working".
> (I first tried this with all 9 input files, and using
> -keep_classification 2 along with -last_only, and got a crash for every
> input file. The simpler command line is sufficient to demonstrate the
> problem.)
combining these 9 tiles may be a bit much. Whenever you use filters
that lower the number of input points (e.g. the '-first_only', ' -
last_only, -keep_class 2' ...) you may want to also add the '-
extra_pass' option so that las2iso first counts how many points will
actually be in the triangulation before it allocates the expected
memory for it. This allows you to work with larger inputs and stay
within the 2 GB memory limit on a 32 bit windows executable.
but thanks for reporting. i need to look into catching this memory
overflow so i can terminate with a nicer error message. (-:
in a follow-up email, Terje writes:
"I have done some more investigation, and it seems to be a resource
limitation that las2iso doesn't catch: Each of the Finn LIDAR files
seems to cover a 3x3 km area, so with 1m DEM resolution that's 9M
cells and well over the 2M limit of the unlicensed toolkit, right?
When I tried to generate only 10% of my contours, i.e. -iso_every 10,
the process did finish, and in not too many seconds at that."
it has nothing to do with the license protection. the tools always
work in the same manner. the only different is that unlicensed tools
add a certain amount of noise once you go above a certain threshold to
the output. the tool will always inform you when it is doing so.
"I also found an interesting issue with the LIDAR file itself: The
area is typical Finland, i.e. fairly flat (+5 to +40 or 50), with zero
mountains, but the LIDAR claims heights from -21 to +1830m! I am
guessing that that 1830m hit must be a spurious return from a bird or
water droplet(s)?"
yes. i just did the following lasinfo run (see below) on a different
tile from the finnish land survey data and you will notice the z
histogram suggests birds or dirt or scanner malfunction.
regards,
martin
C:\lastools\bin>lasinfo N4324D1.laz -histo z 100
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
global_encoding: 0
project ID GUID data 1-4: 0 0 0 ''
version major.minor: 1.2
system identifier: ''
generating software: 'EspaEngine'
file creation day/year: 0/0
header size: 227
offset to point data: 229
number var. length records: 0
point data format: 1
point data record length: 28
number of point records: 20255836
number of points by return: 0 0 0 0 0
scale factor x y z: 0.01 0.01 0.01
offset x y z: 0 0 0
min x y z: 434000.00 6900000.00 67.32
max x y z: 436999.99 6902999.99 658.43
the header is followed by 2 user-defined bytes
LASzip compression (version 2.1r0 c2 50000): POINT10 2 GPSTIME11 2
reporting minimum and maximum for all LAS point record entries ...
x 43400000 43699999
y 690000000 690299999
z 6732 65843
intensity 0 255
edge_of_flight_line 0 1
scan_direction_flag 0 1
number_of_returns_of_given_pulse 1 4
return_number 1 4
classification 1 14
scan_angle_rank -25 27
user_data 134 251
point_source_ID 25 108
gps_time 299162.864868 464211.811502
z coordinate histogram with bin size 100
bin [0,100) has 11983153
bin [100,200) has 8272571
bin [200,300) has 100
bin [400,500) has 2
bin [500,600) has 8
bin [600,700) has 2
average z coordinate 98.6991
number of points by return was not set in header: 17403168 2494071
340856 17741 0
overview over number of returns of given pulse: 14905813 4307014
971616 71393 0 0 0
histogram of classification of points:
2439694 Unclassified (1)
2906267 Ground (2)
809072 Low Vegetation (3)
1399 Low Point (noise) (7)
139348 Water (9)
32901 Reserved for ASPRS Definition (10)
13925375 Reserved for ASPRS Definition (13)
1780 Reserved for ASPRS Definition (14)
Jouko
May 15, 2012, 3:38:22 AM5/15/12
to LAStools - efficient tools for LiDAR processing
Some visualization examples from this data:
http://www.flickr.com/photos/65726513@N00/sets/72157622857163259/
Best Regards
Jouko Vanne
On 9 touko, 15:00, Martin Isenburg <martin.isenb...@gmail.com> wrote:
> Hello
>
> today the LiDAR holdings of the National Land Survey of Finland became
> open data. That means massive amounts of LAS files went online and can
> be freely downloaded. The files are provided exclusively as LASzip
> compressed LAS files (i.e. in the LAZ format) which - according to the
> guy in charge - compacted the total data amount from around 3.1
> TeraBytes down to 475 GigaBytes. You can download the data here:
>
> https://tiedostopalvelu.maanmittauslaitos.fi/tp/kartta?lang=en
>
> it can be bit tricky to find those areas in Finland where data is
> currently available, but here is an index that help you zoom in on the
> relevant locations:
>
> http://www.maanmittauslaitos.fi/sites/default/files/MML_laserkeilauks...
>
> Regards,
>
> Martin @lastools
http://www.flickr.com/photos/65726513@N00/sets/72157622857163259/
Best Regards
Jouko Vanne
On 9 touko, 15:00, Martin Isenburg <martin.isenb...@gmail.com> wrote:
> Hello
>
> today the LiDAR holdings of the National Land Survey of Finland became
> open data. That means massive amounts of LAS files went online and can
> be freely downloaded. The files are provided exclusively as LASzip
> compressed LAS files (i.e. in the LAZ format) which - according to the
> guy in charge - compacted the total data amount from around 3.1
> TeraBytes down to 475 GigaBytes. You can download the data here:
>
> https://tiedostopalvelu.maanmittauslaitos.fi/tp/kartta?lang=en
>
> it can be bit tricky to find those areas in Finland where data is
> currently available, but here is an index that help you zoom in on the
> relevant locations:
>
>
> and here some more overall information on the open data policy of the
> National Land Survey of Finland:
>
> http://www.maanmittauslaitos.fi/en/tiedotteet/2012/05/national-land-s...
> and here some more overall information on the open data policy of the
> National Land Survey of Finland:
>
>
> Regards,
>
> Martin @lastools
Reply all
Reply to author
Forward
0 new messages