Is there a size limit for Blast2dem to merge multiple asc files into a single tif

[Hogarth, Patrick]

Hi Martin,

 

Is there a maximum size limit that blast2dem can merge multiple ascii grids to create a single Geotiff. I’m trying to run it on a 26kmx26km block of data (1m resolution) comprising of 177 2km x 2km asc files, but it doesn’t seem to have created anything despite running for the last 3 hours.

 

Thanks

Patrick

 

Patrick Hogarth  Senior Geomatics Analyst

Environment Agency | Geomatics, National Monitoring Services

__________________________________________

Follow Environment Agency Geomatics on:

  

 

www.gov.uk/floodsdestroy

 

Information in this message may be confidential and may be legally privileged. If you have received this message by mistake, please notify the sender immediately, delete it and do not copy it to anyone else.

We have checked this email and its attachments for viruses. But you should still check any attachment before opening it.
We may have to make this message and any reply to it public if asked to under the Freedom of Information Act, Data Protection Act or for litigation.  Email messages and attachments sent to or from any Environment Agency address may also be accessed by someone other than the sender or recipient, for business purposes.
Click here to report this email as spam

[Terje Mathisen]

Hogarth, Patrick wrote:
>
> Hi Martin,
>
> Is there a maximum size limit that blast2dem can merge multiple ascii
> grids to create a single Geotiff. I’m trying to run it on a 26kmx26km
> block of data (1m resolution) comprising of 177 2km x 2km asc files,
> but it doesn’t seem to have created anything despite running for the
> last 3 hours.
>

There has to be size limits of course.

With your 26 000 x 26 000 grid you get 676 M pixels taking at least that
much memory, but more likely 2-8 bytes per pixel in which case you will
run out on a 32-bit application like LAStools.

Have you tried a few subsets to figure out where the actual limit lies?

Terje

--
- <Terje.M...@tmsw.no>
"almost all programming can be viewed as an exercise in caching"

[Martin Isenburg]

Hello.

Terje, not at all. The 32 bit memory limit does only apply to in-core algorithms. But the BLAST extension of LAStools 

http://rapidlasso.com/BLAST

is based on an out-of-core approach. Namely it is based on our streaming TIN algorithm and a multi-pass approach to generate the final raster that allows generating gigantic rasters while keeping only a fraction of it in memory. The overall approach is described in this publication:

http://www.cs.unc.edu/~isenburg/tin2dem/

Patrick, do you need to interpolate across holes in your ASC rasters? Because blast2dem will create a Delaunay triangulation of your input points it is a very slow way to merge rasters. Actually points on a grid are the *WORST* thing you can feed to a Delanuay triangulator because whenever 4 points are located on the same circle the algorithm slows down a lot. If you merely want to merge many small rasters into one large raster then you should use a raster software such as GDAL that will do this much more efficiently than blast2dem. Also lasgrid would be a faster option.

Note also blast2dem does not support BigTiff, so the resulting TIF file may be too big. Use IMG instead where larger rasters are supported.

Regards,

Martin @rapidlasso

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe

[Terje Mathisen]

Martin Isenburg wrote:
> Hello.
>
> Terje, not at all. The 32 bit memory limit does only apply to in-core
> algorithms. But the BLAST extension of LAStools
>
> http://rapidlasso.com/BLAST
>

Martin, I am of course aware of that, i.e. the BLAST streaming versions
are there specifically to be able to work on really big data sets, so I
guess my reply was unclear:

It was the size of the resulting TIF file which I guessed could cause
these problems.

If regular (not BIG) TIF has the same limits as Photoshop PSD used to
have (i.e. 32Kx32K) then his 26Kx26K raster output should have been OK?

Terje

> is based on an out-of-core approach. Namely it is based on our
> streaming TIN algorithm and a multi-pass approach to generate the
> final raster that allows generating gigantic rasters while keeping
> only a fraction of it in memory. The overall approach is described in
> this publication:
>
> http://www.cs.unc.edu/~isenburg/tin2dem/

> <http://www.cs.unc.edu/%7Eisenburg/tin2dem/>

> - <Terje.M...@tmsw.no <mailto:Terje.M...@tmsw.no>>

> "almost all programming can be viewed as an exercise in caching"
>
>
> --
> Download LAStools at
> http://lastools.org
> http://rapidlasso.com
> Be social with LAStools at
> http://facebook.com/LAStools
> http://twitter.com/LAStools
> http://linkedin.com/groups/LAStools-4408378
> <http://linkedin.com/groups/LAStools-4408378>
> Manage your settings at
> http://groups.google.com/group/lastools/subscribe

> <http://groups.google.com/group/lastools/subscribe>