Limitation on the number of files in batch processing

46 views
Skip to first unread message

Hsin-Ju Li

unread,
Apr 30, 2024, 6:41:38 AMApr 30
to LAStools - efficient tools for LiDAR processing
Hi,


I'm curious about whether there's a limit to the number of input laz files that the lastools functions can handle. I've tried using lasindex and lastile, but whenever there are over about 200 laz files inside a folder, I get the following error message in the terminal:

0024:err:environ:build_command_line command line too long (59229)

Here's an example of running lastile:

lastile -i /filepath/*.laz -tile_size 500 -buffer 25 -flag_as_withheld -odir /out_filepath -olaz -cores 16

Thank you!

Kirk Waters - NOAA Federal

unread,
Apr 30, 2024, 6:53:59 AMApr 30
to last...@googlegroups.com
That tends to be an issue of the operating system and not the individual program. I assume from the direction of your slashes that you're on a linux system. The shell will expand that *.laz to build the command line, which comes into the program as the argument list. There's a limit on how many characters it can be, though the limit is pretty big. To get around it, there is a list of files argument (-lof) in all the LAStools that takes a text file containing a filename, one per line. You might do:

find /filepath -name '*.laz' > filelist.txt
lastile -lof filelist.txt -tile_size 500 -buffer 25 -flag_as_withheld -odir /out_filepath -olaz -cores 16

Kirk Waters, PhD 
NOAA Office for Coastal Management
Applied Sciences Program      




--
Download LAStools at
https://rapidlasso.de
Manage your settings at
https://groups.google.com/g/lastools/membership
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/a890e8d4-c935-40f9-b02d-dd9b2e7f58b8n%40googlegroups.com.

Hsin-Ju Li

unread,
May 2, 2024, 1:57:01 AMMay 2
to LAStools - efficient tools for LiDAR processing
Hi Krik,


Thank you. Your code works! 
However, I'm wondering, when using -lof, wouldn't the files be fed one by one into lastile? In that case, does the -cores function get ignored?

Kirk Waters - NOAA Federal

unread,
May 2, 2024, 6:40:39 AMMay 2
to last...@googlegroups.com
It shouldn't be any different. Simply two ways of getting a list of all the files needed. Splitting the work across cores comes after getting the list.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/f9352730-b9d1-42f7-b157-214ff9da8cf6n%40googlegroups.com.


--
Sent from a mobile device subject to autocorrect errors.
Reply all
Reply to author
Forward
0 new messages