how to parse xyz.zip files ?
502 views
Skip to first unread message
Jean-Roc Morreale
Apr 23, 2012, 10:18:53 AM4/23/12
to last...@googlegroups.com
Hi,
I'm trying to convert *.xyz.zip files into *.laz, the file are composed like this :
/* Headers
'ID' 'X' 'Y' 'Z' 'Echo' 'Date' 'Hour'
000000001 612665.94 7093999.92 27.23 P 2008/05/07 18/45/16
000000002 612665.94 7093999.92 27.17 D 2008/05/07 18/45/16
Here is a command I tried :
txt2las-i "D:\611000_7094000_p01.xyz.zip" -iparse xyz.zip -parse pxyzruu -olaz
Here is the error message I get :
WARNING: cannot parse '000000002 612665.94 7093999.92 27.17 D 2008/05/07 18/45/16' with 'pxyzruu'. skipping...
The goal is to convert more than 43.000 files without decompressing or modifying them as it would take too much disk space to be achieved.
regards,
Morreale Jean-Roc
I'm trying to convert *.xyz.zip files into *.laz, the file are composed like this :
/* Headers
'ID' 'X' 'Y' 'Z' 'Echo' 'Date' 'Hour'
000000001 612665.94 7093999.92 27.23 P 2008/05/07 18/45/16
000000002 612665.94 7093999.92 27.17 D 2008/05/07 18/45/16
Here is a command I tried :
txt2las-i "D:\611000_7094000_p01.xyz.zip" -iparse xyz.zip -parse pxyzruu -olaz
Here is the error message I get :
WARNING: cannot parse '000000002 612665.94 7093999.92 27.17 D 2008/05/07 18/45/16' with 'pxyzruu'. skipping...
The goal is to convert more than 43.000 files without decompressing or modifying them as it would take too much disk space to be achieved.
regards,
Morreale Jean-Roc
Antonio Ruiz
Apr 24, 2012, 3:16:51 AM4/24/12
to last...@googlegroups.com
Jean-Roc,
Some input files could be grouped into one output LAZ. You should first decide what is the result you want and finding how to get it should be the next step.
How are the points distributed in the input files? The current naming of the input fiiles could indicate that they are strips split according to a regular tiling. Do the time and date information refer to the starting time of each strip? It seems you don't have the decimal part of the GPS time. Is there any other information coded in the input file or directory name?
Unzipping the files one by one, converting to LAZ and then deleting the unzipped files won't be a bad solution.
Toni
2012/4/23 Jean-Roc Morreale <jea...@gmail.com>
--
Download LAStools at
http://lastools.org/
Visit the LAStools group at
http://groups.google.com/group/lastools/
Be social with LAStools at
http://www.facebook.com/LAStools
http://www.twitter.com/LAStools
Jean-Roc Morreale
Apr 25, 2012, 2:47:43 AM4/25/12
to last...@googlegroups.com
Hi Toni,
There is the structure of my data files, each of them having the same columns :
S01
> S01_P01
> ZIP
> 611000_7094000_p01.xyz.zip
> 611000_7095000_p01.xyz.zip
> [...]
> S01_P02
> S01_P03
> [...]
S02
S03
[...]
Each of them contains a similar header with the name of the campaign, the software used (terraimaging) and an explanation for each columns.
My aim is to convert each tiles to LAZ for exchange purpose and then merge each campaign (S01, S2,...) into only LAZ file to ease the use. The problem is that I can't get through the parsing stage, unzipped or not.
There is the structure of my data files, each of them having the same columns :
S01
> S01_P01
> ZIP
> 611000_7094000_p01.xyz.zip
> 611000_7095000_p01.xyz.zip
> [...]
> S01_P02
> S01_P03
> [...]
S02
S03
[...]
Each of them contains a similar header with the name of the campaign, the software used (terraimaging) and an explanation for each columns.
My aim is to convert each tiles to LAZ for exchange purpose and then merge each campaign (S01, S2,...) into only LAZ file to ease the use. The problem is that I can't get through the parsing stage, unzipped or not.
Terje Mathisen
Apr 24, 2012, 6:19:07 PM4/24/12
to last...@googlegroups.com
Antonio Ruiz wrote:
> How are the points distributed in the input files? The current naming of
> the input fiiles could indicate that they are strips split according to
> a regular tiling. Do the time and date information refer to the starting
> time of each strip? It seems you don't have the decimal part of the GPS
> time. Is there any other information coded in the input file or
> directory name?
>
> Unzipping the files one by one, converting to LAZ and then deleting the
> unzipped files won't be a bad solution.
I agree:
> How are the points distributed in the input files? The current naming of
> the input fiiles could indicate that they are strips split according to
> a regular tiling. Do the time and date information refer to the starting
> time of each strip? It seems you don't have the decimal part of the GPS
> time. Is there any other information coded in the input file or
> directory name?
>
> Unzipping the files one by one, converting to LAZ and then deleting the
> unzipped files won't be a bad solution.
I would be very tempted to write a small script, something like convert.pl
It starts by picking up all the input filenames, then converting them
one by one (possibly grouping M input files into a single output file).
Run it with a '-cores N' parameter, and it would start N-1 secondary
copies of itself, each receiving the relevant file list as a parameter,
then run the final group in the master task.
Terje
>
> Toni
>
> 2012/4/23 Jean-Roc Morreale <jea...@gmail.com <mailto:jea...@gmail.com>>
- <Terje.M...@tmsw.no>
"almost all programming can be viewed as an exercise in caching"
Jean-Roc Morreale
Apr 26, 2012, 2:38:47 AM4/26/12
to last...@googlegroups.com
As I previously said, I'm not at this stage yet as I can't get to convert even one single file :)
I made a try using "-parse sxyzrss" but I get the same error, am I missing one option to pass in the command or is this format not parseable ?
I made a try using "-parse sxyzrss" but I get the same error, am I missing one option to pass in the command or is this format not parseable ?
Rick W
Apr 27, 2012, 7:37:33 AM4/27/12
to LAStools - efficient tools for LiDAR processing
Hi there,
The data is parseable. I created a 2 line file from the points you
supplied and can parse the file but not with any of the options you
have tried. From your command syntax you have a couple of issues.
First the command itself has to be properly structured. Command line
options or flags are prefixed with a dash (-) there are dozens of
options for any given command. One option is the input file (-i)
another is output file (-o) which are almost always required. The
order of the options in this case does not matter but as a personal
standard I like to put the command first followed by the input file
then the output file.
At a minimum you would need txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz notice how the options are separated by
a space(The default is a space you can change this) on each side. To
parse the file you also need the -parse option. Then the command would
look like txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz -parse <parse options>. Here is where
you are running into your second issue. You need to map the 7 fields
in your file to valid options of the parse command.
For example fields 2,3,4 are the x y z coordinate values so txt2las -i
611000_7094000_p01.xyz.zip -o 611000_7094000_p01.xyz.zip.laz -parse
sxyzsss will work fine skipping the other fields with the s. You need
to figure out what the rest of the columns are and use the correct
parse option.
From the help docs the supported entries are:
x - x coordinate as a double
y - y coordinate as a double
z - z coordinate as a double
X - x coordinate as unscaled integer
Y - y coordinate as unscaled integer
Z - z coordinate as unscaled integer
a - scan angle
i - intensity
n - number of returns for given pulse
r - number of this return
c - classification number
C - classification name
u - user data
p - point source ID
e - edge of flight line
d - direction of scan flag
R - red channel of RGB color
G - green channel of RGB color
B - blue channel of RGB color
M - vertex index number
As far as I can tell txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz -parse pxyzsss is as far as I can get.
I don't know how to flag the last 3 columns of your file the Echo,
Date and Hour as I am not familiar with the format.
Good Luck
The data is parseable. I created a 2 line file from the points you
supplied and can parse the file but not with any of the options you
have tried. From your command syntax you have a couple of issues.
First the command itself has to be properly structured. Command line
options or flags are prefixed with a dash (-) there are dozens of
options for any given command. One option is the input file (-i)
another is output file (-o) which are almost always required. The
order of the options in this case does not matter but as a personal
standard I like to put the command first followed by the input file
then the output file.
At a minimum you would need txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz notice how the options are separated by
a space(The default is a space you can change this) on each side. To
parse the file you also need the -parse option. Then the command would
look like txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz -parse <parse options>. Here is where
you are running into your second issue. You need to map the 7 fields
in your file to valid options of the parse command.
For example fields 2,3,4 are the x y z coordinate values so txt2las -i
611000_7094000_p01.xyz.zip -o 611000_7094000_p01.xyz.zip.laz -parse
sxyzsss will work fine skipping the other fields with the s. You need
to figure out what the rest of the columns are and use the correct
parse option.
From the help docs the supported entries are:
x - x coordinate as a double
y - y coordinate as a double
z - z coordinate as a double
X - x coordinate as unscaled integer
Y - y coordinate as unscaled integer
Z - z coordinate as unscaled integer
a - scan angle
i - intensity
n - number of returns for given pulse
r - number of this return
c - classification number
C - classification name
u - user data
p - point source ID
e - edge of flight line
d - direction of scan flag
R - red channel of RGB color
G - green channel of RGB color
B - blue channel of RGB color
M - vertex index number
As far as I can tell txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz -parse pxyzsss is as far as I can get.
I don't know how to flag the last 3 columns of your file the Echo,
Date and Hour as I am not familiar with the format.
Good Luck
Rick W
Apr 27, 2012, 7:59:34 AM4/27/12
to LAStools - efficient tools for LiDAR processing
Further to my prevoius post after rereading the LAS specification use
txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz -parse uxyzsss
"p" is intended for a different purpose not sure there is a flag for a
unique ID so "u" should work.
http://www.asprs.org/a/society/committees/standards/LAS_1_3_r11.pdf
txt2las -i 611000_7094000_p01.xyz.zip -o
611000_7094000_p01.xyz.zip.laz -parse uxyzsss
"p" is intended for a different purpose not sure there is a flag for a
unique ID so "u" should work.
http://www.asprs.org/a/society/committees/standards/LAS_1_3_r11.pdf
Jean-Roc Morreale
May 2, 2012, 4:18:29 AM5/2/12
to last...@googlegroups.com
Hi Rick, thanks for the replies. The LAS specification is surprisingly clear to read !
These data have been heavily adapted to the original customer demand and I do not have access to the raw data, 'Echo' is an equivalent of 'Return Number' where P is return 1 and D is return 5.
I can keep 'Echo' by using the User Data's flag but 'Date' and 'Hour' cannot fit into this 1 byte field. I will proceed to the conversion anyway, the positive sides of las/laz outweighing these issues (turning a 73Mo file into a 468Ko one = awesome).
These data have been heavily adapted to the original customer demand and I do not have access to the raw data, 'Echo' is an equivalent of 'Return Number' where P is return 1 and D is return 5.
I can keep 'Echo' by using the User Data's flag but 'Date' and 'Hour' cannot fit into this 1 byte field. I will proceed to the conversion anyway, the positive sides of las/laz outweighing these issues (turning a 73Mo file into a 468Ko one = awesome).
Reply all
Reply to author
Forward
0 new messages