lasclip with multiple files

[hyd...@gmail.com]

Hey again :)

We're trying to clip multiple LAZ files with a single shapefile while using the multiprocessing capabilities within lastools.

Naturally, each LAZ file holds some of the relevant geometries of the shapefile, some geometries might cross multiple LAZ files (e.g. along the border), etc.

We're running something along the lines of:

lasclip -lof my_files.txt -poly geoms.shp -split uid -otxt -oparse xyzi -odir clipped/ -cores 40

The issue we're running into is a race condition. Since lasclip creates the names using only the uid field, then the clipped point cloud from e.g. LAZ file #1 and LAZ file #2 will attempt to write to the same file, overwriting successful clipping.

The simplest approach to solve this would be if the generated clipped file would also include the LAZ file name (similar to the -odix flag), but I couldn't see how to have both the uid from the shapefile and the basename of the LAZ file be present in the resulting clipped file.

So this is perhaps a feature request / clarification if these race conditions were (not) considered? :)

Thanks for your help!

Best,

Idan

[Jochen Rapidlasso]

Hi Idan,

thanks for the suggestion.

We add it to our todo list. This seems to be not very complicated, so we will implement this soon.

Best,

Jochen @rapidlasso

[Jochen Rapidlasso]

Hi Idan,

we implemented your request for lasclip64.

There are 2 new arguments:

    -filename_as_prefix
    -filename_as_suffix

If set, a prefix or suffix will be added to the output filename additional to the attribute set by the split argument.

You can download the beta versions at

    https://downloads.rapidlasso.de/beta/lasclip64.7z
    https://downloads.rapidlasso.de/beta/lasclip64.tar.gz

Best regards,

Jochen @rapidlasso

[James Carracher]

Hi Jochen,

This is great. I've wanted a change like this for ages now. I've bodged together a way to do so using FME but this is better.
 I might have another suggestion too, or maybe I'm missing something, I'd like to be able to customise the tile numbering. As far as I understand the numbers start at 0 and count upwards based or the ID in the shp file. Frequently I need a different starting value. It's usually 1 instead of 0, but it has been different before. If that was possible too that would be ideal

Regards,

James

[Idan Tene]

Thanks Jochen!

James, you can include an identifier attribute in your shapefile and specify lasclip to use that in the naming convention (similarly to how we used -split uid). That should work nicely :)

--
Download LAStools at
https://rapidlasso.de
Manage your settings at
https://groups.google.com/g/lastools/membership
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/b0f5a754-fdbd-4e88-871b-d5b5ac6cf6c9n%40googlegroups.com.

[Matthew Baughan]

Hi, Jochen

Been waiting for this thanks!

Upon testing:

in:

10001_1_256_2023_3065_C-FVVS_utm10.laz

out:

#1 using -filename_as_prefix

lasclip64 -i *.laz -poly 20k_clip.shp -split MAP_TILE -filename_as_prefix -olaz -odir \output
10001_1_256_2023_3065_C-FVVS_utm1_093B053.laz'. total time 5.645 sec.

#2 using -filename_as_suffix

lasclip64 -i *.laz -poly 20k_clip.shp -split MAP_TILE -filename_as_suffix -olaz -odir \output
093B053_10001_1_256_2023_3065_C-FVVS_utm1.laz'. total time 5.723 sec.

Prefix and suffix need to be switched as they are working opposite to what they are named, and for some reason in both cases the last character of the filename is deleted in the output

Hope this helps,

Thanks,

Matt

[Jochen Rapidlasso]

Hi Matt,

thanks for your comments. About prefix/suffix: Isn't that correct?

-filename_as_prefix

gives you 

    10001_1_256_2023_3065_C-FVVS_utm1_093B053.laz

where 
    10001_1_256_2023_3065_C-FVVS_utm1 is the prefix of the output file.

The missing char will be fixed - sorry about this.

We will release a new version today with some minor bugfixes and improvements like this and give a note in the group when it is online.

Also the full documentation about the changes will be released very soon.

Best,

Jochen @rapidlasso

[Idan Tene]

Hey Jochen,

If I may make a suggestion, as a fellow developer and lastools user - to maintain the same "feel" as other lastools, it seems to me that the `-filename_as_prefix` should be a default setting (similar to how other tools append to the base filename, whether by running index or by `-odix`). I do understand this is a major behavior change for lasclip though. If implemented this way, I suppose there should be a `-omit_filename` flag instead, though I doubt it will be a useful flag, and is more offered as a means for backwards compatibility.

Then, while I understand the sentiment snd desire to generalize tools and allow as much freedom as possible, I don't believe there's a need to also support `-filename_as_suffix`. It just adds one more flag to the CLI, readme, etc. The parsing of filenames will remain identical in most cases.

Just my two cents on this one.

Happy to have this in place, thank you so much!

--
Download LAStools at
https://rapidlasso.de
Manage your settings at
https://groups.google.com/g/lastools/membership
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/f73a58a0-0116-4934-9c97-9f16f23ca8b9n%40googlegroups.com.

[Jochen Rapidlasso]

Hi Idan,

thanks for your suggestions.

The filename *IS* the default setting.

So if you enter just "-split" you get files like

    [inputfilename]_[no_of_shape].laz

Only if you give the argument -split *PLUS* an attribute index or name you get the name of this attribute as filename.

So the 

 -filename_as_prefix
 -filename_as_suffix

arguments are refinements for the specific split option "- split [n]".

They should be called "-filename_as_prefix_of_attribute_name" which is admittedly a bit long.

To avoid surprises we add some additional hints about the naming, if someone combines "-split" with "-filename_as...".

To add the filename by default if -split [n] is given would break the downwards-compatibility rule, which is one of our major rules:

A running script should never change behavior by a new version (unless it was a error before and now the error is fixed).

The " filename_as_suffix" argument was our 2nd use case of lasclip: It is very handy, to have the files sorted by attributes, and have the filename just as distinguishing key/feature of identical shape attributes.

Best regards,

Jochen @rapidlasso

[Idan Tene]

Hey Jochen,

That all makes a lot of sense, and of course the backwards compatibility is a very understandable concern.

Thanks for going over the decision-making process!

Best

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/cbf02560-24af-475b-9696-6cfc3337098fn%40googlegroups.com.

[Matthew Baughan]

Hi Jochen,

Has the new version of the beta been released yet?

I'm suffering from some data loss with the current version, and have some other issues to address.

Thank you,

Matt