Waveforms and their format

[Narsimha]

Hi,

I'm just wondering that is there any standard format for waveform data?

Is it .wdp, .wvs, ASCII or others ? what is the ASPRS standard?

Can we switch the waveform data formats from one to another?

Also keen to know what would be the preferred or ideal waveform format to be smoothly synced with LAS1.4 point clouds?

Regards,

Narsimha

[Martin Isenburg]

Hello,

there is a "standard" that is part of the LAS format specification 1.3 and 1.4 using point types 4, 5, 9 or 10. Here the points are "decorated" with their waveforms that are ideally stored in a separate WDP file (rather than being appended to the LAS file) and indexed by the "offset" in the wave form packet that is part of each LAS point record 4, 5, 9 or 10.

We created the PulseWaves format because the LAS extension to include waveform storage has a few short-comings: no storage of the out-going waveform, only single-segment returning waveforms, only single-channel returning waveforms, high redundancy of data for multi-return data sets. Vendors have also had trouble to correctly interpret the format as several previous posts in this forum will tell you. An illustrative graphic that I had proposed to clarify how the direction vector for different returns along the waveform is to be understood has so far been rejected by the "boss" of the least-transparently-managed open geospatial standard we have ... (-; But that may change soon as I hear the LAS format will soon be placed under some form of oversight by a proper standarization body via special agreement by the OGC ... 

What sensor is your data from? For both RIEGL and Optech waveform data you can also request "PulseWaves" output for your data. Not sure what the status for Leica is. At least their AHAB systems should allow for similarly nice waveform outputs as we see for RIEGL and Optech. Some more info about your particular data would help ...

Regards,

Martin 

--
--
Post to "PulseWaves" by email to pulse...@googlegroups.com
Unsubscribe by email to pulsewaves+...@googlegroups.com
Visit this group's message archives at http://pulsewaves.org

---
You received this message because you are subscribed to the Google Groups "PulseWaves - no pulse left behind" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pulsewaves+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anh Vu Vo]

Hello,

I am not sure whether you can consider it is a standard format, but Sorted Pulse Data (see the source paper below) is an alternative open, vendor-neutral format to LAS and PulseWaves. The format is not that popular compared to LAS and PulseWaves but it seems to deserve a place in the list (?).

---

Bunting, P., Armston, J., Lucas, R. M., & Clewley, D. (2013). Sorted pulse data (SPD) library. Part I: A generic file format for LiDAR data from pulsed laser systems in terrestrial environments. Computers & Geosciences, 56, 197-206.

Vu Vo

[Pete Bunting [pfb]]

Hi all, 

With SPD in mind it is worth highlighting that there is an updated version of the SPD format (v4), details are here http://pylidar.org/en/latest/spdv4format.html

SPD v4 is implemented in the new pylidar library that is aiming to make it easier to access lidar data (discrete returns and waveforms) through python http://pylidar.org. 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

--------------------------------------------------------------------
Astudio gyda ni yn Aberystwyth - Prifysgol Gyntaf Cymru https://www.aber.ac.uk/cy/undergrad/

Study at Aberystwyth – Wales’ First University https://www.aber.ac.uk/en/undergrad/

[Narsimha]

Thank you all, I was in an impression that. wdp is the ASPSRS standard.

 

However, the FWL data is provided as an ASCII product and delivered per flight path. Further details are as follows.

 

Sensor:LiDAR

Device Name : Trimble AX60

INS/IMU: Trimble AP50 GNSS/IMU

 

Q1. Can we convert the waveforms to one or other (SPD or Pulse waves?)? Is there any way to split the FW data as per the LAS tile index?

Q2. What sort of FWL format would be ideal to work with LAS1.4 data?

 

Sample data:

99896375|365596.69281418|-35.269207|148.994020|571.69|-35.269165|148.994026|579.13|59000|1|0|32|48|32|16|16|16|16|32|64|64|48|32|16|16|96|256|592|1008|1264|1232|960|576|288|144|80|64|48|64|64|80|80|64|48|32|32|48|48|32|32|32|32|48|32|16|0|0|0|0|32|16|0|16|16|32|32|16|16|0

 

ASCII product of decomposed waveform (including all information not present in the Las files, i.e. range of sensor to the target, width of return, amplitude, etc,,,,) delivered as flight path.

Each return is a separate line:

1. Pulse ID

2. Time (origin)

3. Easting

4. Northing

5. Elevation (origin; AusGeoid09)

6. Elevation (origin; ellipsoid)

7. Time (return)

8. Easting (return)

9. Northing (return)

10. Elevation (return; Ausgeoid09)

11. Elevation (return; ellipsoid)

12. Class

13. Return number

14. Number of return per pulse

15. Intensity

16. Gaussian amplitude (if different from the “intensity”)

17. Gaussian width (FWHM)

18. Range

19. Pulse zenith angle

20. Pulse azimuth angle

Appreciate your taught on this.

Regards,

Narsimha

[Martin Isenburg]

Hello,

your sample data is not conform with the description. See below

99896375|365596.69281418|-35.269207|148.994020|571.69|-35.269165|148.994026|579.13|59000|1|0|32

1. Pulse ID: 99896375
2. Time (origin): 365596.69281418
3. Easting: -35.269207
4. Northing: 148.994020
5. Elevation (origin; AusGeoid09): 571.69 only one of 5 / 6 is here ???
6. Elevation (origin; ellipsoid): 571.69 only one of 5 / 6 is here ???
7. Time (return): missing???
8. Easting (return): -35.269165
9. Northing (return):  148.994026

10. Elevation (return; Ausgeoid09): 579.13  only one of 10 / 11 is here ???

11. Elevation (return; ellipsoid): 579.13  only one of 10 / 11 is here ???

12. Class: 59000 ?!?!?!?!

13. Return number: 0 ?!?!?!?!

14. Number of return per pulse: 32 ?!?!?!?!

...

I am utterly confused by the above. Why is the origin of the pulse (or maybe the start of the waveform recoding) lower in elevation than the return?

One goal of the PulseWaves format to eliminate exactly what is happening here: wasting the time of researchersin full waveform LiDAR are spending on understanding and parsing the smorgasboard of formattings that FWF data can be delivered in. Do you have a contact at Trimble who can be asked to add PulseWaves support? I believe that the full waveform scanners of Trimble are re-branded RIEGL scanners. Then maybe someone with access to RiPROCESS can turn the originally recorded SDF files into PulseWaves for you?

Also ... your data was stored in long/lat with only six decimal digits which corresponds to approximately decimeters. This is bad as this will introduce a small quantization error. I think seven decimal digits are standard for airborne LiDAR as this gets around centimeter resolution. This is what I remember from a while ago and matches the detailed discussion on the topic in this thread here:

http://gis.stackexchange.com/questions/8650/how-to-measure-the-accuracy-of-latitude-and-longitude

Sorry that there are not better news. But your (bad) experience really underscores the benefits of easy to parse standards such as LAS or PulseWaves for our community. 

Regards,

Martin

[Martin Isenburg]

Hello,

Narsimha had answered the following. I had to remove Narsimha's original message due to the excessively large original attachement of 6 MB. I truncated this down to a few lines of that file and those lines are attached. They look like this. And yes, they do contain full waveform-derived data but no actual waveform samples:

53247521|346780.82441624|685272.99|6093915.66|1063.50|1082.89|346780.82441757|685241.75|6093513.68|511.80|531.18|4|1|1|243|384|4|683|0.6311|1.4945

53247522|346780.82441894|685272.99|6093915.66|1063.50|1082.89|346780.82442027|685241.67|6093513.82|511.16|530.54|4|1|2|194|352|3|683|0.6304|1.4943

53247522|346780.82441894|685272.99|6093915.66|1063.50|1082.89|346780.82442027|685241.55|6093512.33|509.11|528.49|2|2|2|57|176|3|686|0.6304|1.4943

1. Pulse ID:  53247521

2. Time (origin): 346780.82441624

3. Easting: 685272.99

4. Northing: 6093915.66

5. Elevation (origin; AusGeoid09): 1063.50

6. Elevation (origin; ellipsoid): 1082.89

7. Time (return): 346780.82441757

8. Easting (return): 685241.75

9. Northing (return): 6093513.68

10. Elevation (return; Ausgeoid09): 511.80

11. Elevation (return; ellipsoid): 531.18

12. Class: 4

13. Return number: 1

14. Number of return per pulse: 1

15. Intensity: 243

16. Gaussian amplitude (if different from the “intensity”): 384

17. Gaussian width (FWHM): 4 

18. Range: 686

19. Pulse zenith angle: 0.6311

20. Pulse azimuth angle: 1.4943

This makes much more sense. This kind of data allows you, to for example, create a *richer* LAS file that contains information of the "echo width" that can be stored as an extra byte similarly to how RIEGL (*) is doing it. You can probably do also some kind of intensity correction using the distance of the target from the scanning system as well as the incidence angle of the beam that can be computed from the two points given as origin and return. But there is are no actual waveform samples stored in this representation. I think seeing it an *enriched* LAS file is probably closest to the truth. Using txt2las you should be able to generate a LAS file with "extra bytes" to store the additional attributes derived from the waveform. But you do not have the information about the actual digitzed waveform samples to convert this to PulseWaves (or LAS 1.3 FWF) formats.

Regards,

Martin @rapidlasso

(*) http://www.riegl.com/uploads/tx_pxpriegldownloads/Whitepaper_-_LAS_extrabytes_implementation_in_Riegl_software_01.pdf

---------- Forwarded message ----------

From: Narsimha <narsimha....@gmail.com>

To: PulseWaves - no pulse left behind <pulse...@googlegroups.com>

Cc: 

Date: Mon, 25 Jul 2016 20:59:53 -0700 (PDT)

Subject: Re: [PulseWaves] Re: Waveforms and their format

Hi all,

My apologies...I have just noticed that the earlier sample was from raw waveform data.

Now I have attached the sample decomposed waveform and hopefully it will comply with the description provided.

The data provider confirms that data delivered is fully compliant with the project specifications. 

Moving further and also as you can expect I need to work my way to make best use of this data.

Hope the forum can shed some light on my technical issues and also with a affordable solution.

Regards,

Narsimha